CN104166834B

CN104166834B - Pavement detection method and apparatus

Info

Publication number: CN104166834B
Application number: CN201310187630.XA
Authority: CN
Inventors: 陈超; 游赣梅; 师忠超; 鲁耀杰; 王刚
Original assignee: Ricoh Co Ltd
Current assignee: Ricoh Co Ltd
Priority date: 2013-05-20
Filing date: 2013-05-20
Publication date: 2017-10-10
Anticipated expiration: 2033-05-20
Also published as: CN104166834A

Abstract

A kind of pavement detection method and apparatus are proposed, can be included：Acquisition includes the disparity map and gray-scale map on road surface；Detection can be identified for that the Sign for road of road surface position from gray-scale map；The full V disparity maps based on full figure are built from disparity map；First via millet cake is selected from full V disparity maps；The road instruction V disparity maps based on the Sign for road detected are built from disparity map；The second road surface point is selected from road instruction V disparity maps；And road surface is extracted based on first via millet cake and the second road surface point.Utilize pavement detection method according to the above embodiment of the present invention and road surface checking device, simultaneously using the V disparity maps based on whole disparity map and the V disparity maps based on such as Sign for road of left and right lane line, carry out road surface extraction, detection for tilted road surface is very effective, and the road surface extracted more tallies with the actual situation.In addition, the primary operational processing of the inventive method is all 2-D data, thus characteristic point is strengthened, and amount of calculation significantly reduces simultaneously.

Description

Pavement detection method and apparatus

Technical field

The present invention relates to image procossing, relate more specifically to pavement detection method and apparatus.

Background technology

The application of drive assist system is increasingly popularized.Road or lane-departure warning system (Lane/Road detection Warning, LDW/RDW) be drive assist system subsystem, steering direction etc. can more accurately be determined with collision free.Road Road detection is very crucial for LDW/RDW systems, and further located only is only possible on the basis of road information is aware of Reason, for example, alert.Moreover, pavement of road detection is also the very important part of 3D driving environment understanding technologies, 3D is driven Roadside guardrail detection in environment understanding technology, vanishing Point Detection Method, vehicle identification etc. has a great impact.

However, existing road surface algorithm for estimating there is also some problems：They can carry out accurate road in simple environment Face detection, but may be it occur frequently that error detection in complex environment.

In entitled " Image Processing Apparatus and Method " U.S. Patent Publication In US20090041337A1, estimate based on 3D infomation detection roadmarkings, and then using this roadmarking and 3D information progress road surface Meter.The invention directly carries out lane detection in disparity map using 3D information, and road surface estimation is then carried out in disparity map, because And it is comparatively computationally intensive, complexity is high.

It is Zhencheng Hu, Francisco Lamosa, Keiichi Uchimura, entitled " AComplete in author U-V-Disparity Study for Stereovision Based3D Driving Environment Analysis.” Proceedings of the5th international Conference on3-D Digital Imaging and In Modeling, Jun.13,2005. document, it is proposed that a kind of road scene parser based on stereoscopic vision.The technology By constructing V disparity maps from original disparity map, to the road plane in 3D road scenes, non-rice habitats region, barrier etc. enters Row classification.Technology complex road condition for there is a situation where to tilt such as road surface may make a mistake detection.

The content of the invention

The present invention is proposed based on said circumstances.

According to an aspect of the present invention, it is proposed that a kind of pavement detection method, it can include：Acquisition includes regarding for road surface Difference figure and gray-scale map；Detection can be identified for that the Sign for road of road surface position from gray-scale map；Built from disparity map based on complete The full V disparity maps of figure；First via millet cake is selected from full V disparity maps；Built from disparity map based on the Sign for road detected Road instruction V disparity maps；The second road surface point is selected from road instruction V disparity maps；And based on first via millet cake and 2 road surface points extract road surface.

According to an aspect of the present invention, it is proposed that a kind of road surface checking device, it can include：Image obtains part, uses Include the disparity map and gray-scale map on road surface in obtaining；Sign for road detection part, can be identified for that for being detected from gray-scale map The Sign for road of road surface position；Full V disparity maps build part, for building the full V disparity maps based on full figure from disparity map； First via millet cake alternative pack, for selecting first via millet cake from full V disparity maps；Road instruction V disparity maps build part, use In building the road instruction V disparity maps based on the Sign for road detected from disparity map；Second road surface point selection part, For selecting the second road surface point from road instruction V disparity maps；And road surface extracting parts, for based on first via millet cake and 2 road surface points extract road surface.

Using pavement detection method according to the above embodiment of the present invention and road surface checking device, while using based on whole The V disparity maps of disparity map and the V disparity maps based on such as Sign for road of left and right lane line, carry out road surface extraction, for inclining The detection in wrong path face is very effective, and the road surface extracted more tallies with the actual situation.In addition, using according to the above embodiment of the present invention Pavement detection method and road surface checking device, detect the Sign for road of such as lane line in gray-scale map, and based on detection Road surface straight line estimation is carried out to Sign for road construction V disparity maps, the two step process are all 2-D datas, thus feature Point is strengthened, and amount of calculation significantly reduces simultaneously.

According to another aspect of the present invention, it is proposed that a kind of pavement detection method, it can include：Acquisition includes road surface Disparity map；V disparity maps are built from disparity map；Select road surface point；And road surface is extracted based on road surface point.Wherein, road surface is selected Point includes：Obtain the architectural feature of each pixel in V disparity maps, the architectural feature of a pixel is based on being in the pixel In center, region with given shape and predefined size the accumulation of all pixels point and obtained from, the shape in the region Situation about being projected with road surface in V disparity maps is matched；Based on the architectural feature of the pixel of each column in V disparity maps, the row are determined Dynamic threshold；Wherein selected road surface point meets its Structural Eigenvalue It is highly minimum in the pixel of all dynamic thresholds more than the row in the row.

According to another aspect of the present invention, it is proposed that a kind of road surface checking device, it can include：Image obtains part, Acquisition includes the disparity map on road surface；V disparity maps build part, for building V disparity maps from disparity map；Road surface point selection portion Part, for selecting road surface point；And road surface extracting parts, for extracting road surface based on road surface point.Wherein, road surface point selection part Selection road surface point can include：The architectural feature of each pixel in V disparity maps is obtained, the architectural feature of a pixel is to be based on In region centered on the pixel, with given shape and predefined size the accumulation of all pixels point and obtained from, The situation that the shape in the region is projected with road surface in V disparity maps is matched；Structure based on the pixel of each column in V disparity maps is special Levy, determine the dynamic threshold of the row；The architectural feature of wherein selected road surface point meet the dynamic threshold that is more than affiliated row and The road surface point be all dynamic thresholds more than the row in the row pixel in it is highly minimum.

Using pavement detection method according to the above embodiment of the present invention and road surface checking device, as previously described, because picture What the architectural feature of vegetarian refreshments considered is the overall condition rather than an independent pixel of the pixel in a region, so compared with Weak road surface has obtained corresponding enhancing.At the same time, due to calculating accumulated value, noise can effectively be removed.

Further, since determining the structure spy for calculating a pixel according to projection of shape of the road surface in V disparity maps The shape in the region levied, therefore resulting architectural feature is more suitable for pavement detection application.

In being chosen according to the road surface of the present embodiment point, because taking the dynamic threshold for being suitable for regional area such as each column Value, therefore, it is possible to adaptively determine dynamic threshold, therefore can preferably select the road surface point of each regional area.

Brief description of the drawings

Fig. 1 contributes to understand the onboard system schematic diagram of of the invention, as application environment of the present invention example.

Fig. 2 is the overall flow figure of pavement detection method 1000 according to an embodiment of the invention.

Fig. 3 schematically shows the result example of a lane detection.

(a) in Fig. 4 schematically shows the example that vehicle-mounted stereoscopic camera shoots the road area disparity map obtained.Figure (b) in 4 schematically shows the V disparity maps that disparity map conversion as shown in Figure 3 is obtained, namely full V specifically described herein Disparity map.

Fig. 5 shows according to an embodiment of the invention built from disparity map based on the Sign for road detected The flow chart of the illustrative methods 1500 of road instruction V disparity maps.

Fig. 6（a）It is identical with Fig. 3, the lane detection result in gray-scale map in an example is shown, Fig. 6 (b) is shown Corresponding detection area-of-interest in the disparity map determined in one example, illustrated therein is two regions as the sense of detection Region of interest（ROI）.

(a), (b), (c) in Fig. 7 shows the V disparity map organigrams based on left and right lane line of an example. (a) in Fig. 7 is identical with Fig. 6 (b), shows corresponding detection area-of-interest in the disparity map determined in one example, (b) in Fig. 7 shows that (d) in the left side road V disparity map schematic diagrames constructed based on left-lane line ROI, Fig. 7 shows base (c) in the right side road V disparity map schematic diagrames constructed in right-lane line ROI, Fig. 7 is shown based on left and right lane line ROI's V disparity maps correspond respectively to corresponding part in the V disparity maps based on full figure.

Fig. 8 is shown according to an embodiment of the invention extracts showing for road surface based on first via millet cake and the second road surface point The flow chart of example method 1700.

Fig. 9 shows that each road surface line segment extracted in Sign for road is by the example of left and right lane line is unified in one Schematic diagram in individual V disparity maps.

Figure 10 is given based on the second road surface of left side line segment, right side the second road surface line segment and complete first road surface shown in Fig. 9 The schematic diagram for the road surface envelope that line segment is drawn.

Figure 11 shows the flow chart of road surface point extracting method 1400 according to an embodiment of the invention.

The testing result that road surface historical frames are representatively illustrated in Figure 12 and the detection zone determined therefrom that, and schematically Ground shows the detection zone determined based on camera parameters.

Figure 13 shows showing for the region according to an embodiment of the invention for being used to calculate pixel Pi architectural feature It is intended to.

Figure 14 shows the schematic diagram for the road surface candidate point tentatively chosen for the V disparity maps based on full disparity map.

Figure 15 (a) schematically shows the road surface point example tentatively chosen, and Figure 15 (b) is diagrammatically illustrated to be gone by discrete point The road surface point example of final selection after removing.

Figure 16 is schematically illustrated in set of the selection result of each V disparity maps Road millet cake in a V disparity map.

Figure 17 (a1)-(c1) and (a2)-(c2) are shown for comparing conventional pav point choosing method and according to the present invention The schematic diagram of the result of the road surface point choosing method of embodiment.

Figure 18 shows the overview flow chart of pavement detection method 1000 ' according to a second embodiment of the present invention.

Figure 19 shows that the road surface extracted in V disparity maps is the exemplary road surface inverse mapping method 1800 in the case of envelope Flow chart.

Figure 20 (a) and Figure 10 are same, it is schematically shown that the road extracted according to one embodiment of the invention in V disparity maps Envelope, Figure 20 (b) schematically shows regarding for the road surface envelope according to an embodiment of the invention based on extraction Road surface point inverse mapping result in poor figure.

Figure 21 (a1)-(c1) and (a2)-(c2) show the road surface result that different detection methods are obtained.

Figure 22 shows the flow chart of pavement detection method 2000 according to a third embodiment of the present invention.

Figure 23 shows the block diagram of road surface checking device 4000 according to an embodiment of the invention.

Figure 24 shows the block diagram of road surface checking device 5000 according to another embodiment of the present invention.

Figure 25 is the concept map for the hardware configuration for showing the pavement detection system 6000 according to the embodiment of the present invention.

Embodiment

In order that those skilled in the art more fully understand the present invention, with reference to the accompanying drawings and detailed description to this hair It is bright to be described in further detail.In addition, the main points in order to avoid obscuring the present invention, will not for some technologies well known in the art Elaborate.

It will be described in the following order：

1st, basic conception introduction

2nd, application environment is illustrated

3rd, the first embodiment of pavement detection method

3.1st, the overall flow of pavement detection method

3.2nd, the detailed example of road surface point extracting method

4th, the second embodiment of pavement detection method

5th, the 3rd embodiment of pavement detection method

6th, the first embodiment of road surface checking device

7th, the second embodiment of road surface checking device

8th, system hardware configuration

9th, summarize

1st, basic conception introduction

Basic conception is described below, in order to understand.

Parallax, actually refer to from a certain baseline two ends respectively draw a straight line to it is same compared with far object when, formed angle therebetween.One As refer to direction difference from two points for having certain distance produced by same target.In terms of target between two points Angle, is called the parallactic angle of the two points, and the distance between 2 points are referred to as baseline.Only it is to be understood that parallactic angle degree and baseline length, The distance between target and observer can just be calculated.

Disparity map (disparity map) is that on the basis of appointing piece image, its size is the size of the benchmark image, member Element value is the image of parallax value.Disparity map contains the range information of scene.The left figure that disparity map can be shot from binocular camera Calculate and obtain in picture and right image, or obtained by the depth map calculating in three-dimensional view.

Certain point coordinates in ordinary two dimensional disparity map with（x,y,d）Represent, wherein x is abscissa, and y is ordinate, and d is represented Parallax value at the pixel.

For ease of understanding, figuratively, V disparity maps can be considered as the side view of disparity map, and U disparity map can be considered as The top view of poor figure.V disparity maps can be calculated from disparity map and obtained.The gray value of any point (d, y) is in V- disparity maps The ordinate of correspondence disparity map is equal to the number of d point for parallax value in y row.

Road surface line or road surface line segment：The broadside lines of pavement of road is represented, in the real 3D worlds, road is substantially One 3D plane, but pass through after side projection, the road surface side profile formed in V disparity maps substantially 2D line segment, Here road surface line or road surface line segment are called.

2nd, application environment schematic diagram

Fig. 1 contributes to understand the onboard system schematic diagram of of the invention, as application environment of the present invention example.This hair Bright software or hardware can be realized as Road Detection part therein or one part.

3rd, the first embodiment of pavement detection method

3.1st, the overall flow of pavement detection method

Pavement detection method according to an embodiment of the invention is described referring to Fig. 2.

As shown in Fig. 2 in step S1100, acquisition includes the disparity map and gray-scale map on road surface.

In one example, it can shoot to obtain including the gray-scale map in road surface region by vehicle-mounted stereoscopic camera（Such as a left side Eye gray-scale map）, and obtain corresponding disparity map from such as left eye gray-scale map and right eye gray-scale map by calculating.Stereoscopic camera is for example There are binocular camera, many mesh cameras etc..

But, in another example, to shoot respectively gray-scale map and disparity map can be obtained by different cameras, as long as The Conversion Relations between the coordinate system between image be can determine so that both may switch to a unified coordinate In system.For example, the general camera that can be carried on automobile obtains gray-scale map to shoot, and the binocular phase carried on automobile Machine obtains disparity map to shoot and calculate, in this case, gives the position relationship and each of general camera and binocular camera Performance parameter, then can determine the coordinate transformation relation between gray-scale map and disparity map.

In one example, the image capture device of shooting image and image procossing is carried out to detect the signal transacting on road surface Device can mutually for be located locally, even integrate.In another example, the camera of shooting image With carry out image procossing with detect road surface information processor can mutually for be remote, for example, by wired or nothing Gray-scale map and/or disparity map are delivered to signal processing apparatus by line communication.

Hereinafter, for convenience of description, it is equipped on vehicle with video camera and for the information processor of Road Detection It is described exemplified by upper.

Next, in step S1200, detection can be identified for that the Sign for road of road surface position from gray-scale map.

Here, for example, Sign for road can be lane line, the position of lane line is the position for identifying road surface.But, Sign for road not limited to this, but can be the fence on the anything that can be identified for that road surface position, such as road, curb Stone, road is on both sides of the road or middle grove etc..Specifically, for example, the bottom position of the fence on road indicates road surface position.Class As, the crowd that a group is advanced on road can also be as Sign for road, because the bottom position of crowd indicates road surface position.

But, because detecting that lane line is relatively easy and more directly indicates road surface position in gray-scale map, because This may be particularly advantageous as Sign for road using lane line.Hereinafter, for convenience of description, retouched by taking lane line as an example State.

On the lane detection based on gray-scale map, the method that many lane detections are disclosed in the prior art, these Existing method for detecting lane lines may be incorporated for the present invention.

The illustrative methods of a progress lane detection according to embodiments of the present invention are given below.First, car is determined The substantially detection zone of diatom, such as, because video camera is integrally fixed on vehicle, in vehicle travel process, lane line is located at Specific region in image, therefore the specific region can be set to detect ROI（Area-of-interest）.Secondly, in detection ROI In, gray level image is handled by such as Sobel etc edge detection operator, so that edge graph is created, wherein in view of track Line has specific angle, and the marginal point that only Grad can be within the angular range, which is remained, subsequently to be located Reason；In addition, when carrying out lane detection, having many features can be for carrying out lane detection, such as lane width is long Degree, color etc..Finally, sectional straight line fitting can be carried out in edge graph using Hough transform.The lane line information of output is such as Following formula（1）It is shown：

Left-lane line:y·sin(θ_L)+x·cos(θ_L)=ρ_L (1)

Right-lane line:y·sin(θ_R)-x·cos(θ_R)=ρ_R

Wherein, x, y are the coordinate in gray-scale map, θ_L,ρ_L,θ_R,ρ_RRespectively left and right lane line parameter.If do not had on road surface There is lane line, the module will be judged that track output parameter is set into 0 is identified by the accumulated value of Hough transform.Figure 3 schematically show the result example of a lane detection.

Next, in step S1300, the full V disparity maps based on full figure are built from disparity map.

As it was previously stated, can be by the way that a vertical plane will be mapped to a little in original disparity map（V- vertical dimensions, Δ-depth dimension）, to generate V disparity maps.Each point in former disparity map represents a point in three-dimensional world coordinate system（U- Level is tieed up, V- vertical dimensions, Δ-depth dimension）.After mapping, among new image, level dimension is disappeared, this new image It is exactly so-called V disparity maps.The intensity each put in V disparity maps is exactly there is same disparity value to be sat with identical y in former disparity map Mark（V- vertical dimensions）Point in x coordinate（U- levels are tieed up）Accumulation number on direction.In this V disparity map, road surface is increased By force, and put quantity substantially reduce, therefore based on V disparity maps carry out pavement detection can significantly improve the effect of pavement detection Rate and accuracy.

Next, in step S1400, first via millet cake is selected from full V disparity maps.

If carrying out line segment by line fitting method to V disparity maps untreatedly is used as road surface, because noise is very Many, testing result is inaccurate；And amount of calculation is very big.

According to one embodiment of the invention, filtering V disparity maps, to remove Null Spot, are carried with carrying out the selection of road surface point Computationally efficient, prevents noise jamming.

According to an example, when selecting road surface point, it may be considered that the intensity of road surface point should be more than certain threshold value, and Road surface point is normally at the bottom of V disparity maps.

According to one embodiment of present invention, it can include from V disparity maps selection road surface point：Obtain each picture in V disparity maps The architectural feature of vegetarian refreshments, the architectural feature of a pixel be based on it is centered on the pixel, with given shape and The accumulation of all pixels point is with obtained from the region of predefined size, and shape and the road surface in the region are projected in V disparity maps Situation matching, this is in order to which when architectural feature is extracted, the architectural feature of the pixel on road surface can be as more as possible Neighbouring road surface point is included by ground.In one example, binocular camera level be equipped on vehicle and basic horizontal to In the case of preceding shooting, the region is shaped as approximate parallelogram, as shown in figure 13, and such region shape exists with road surface The approximate line style projection matching at 135 degree of angles is approximately tilted in V disparity maps.I.e., generally, it is mounted in vehicle in camera level In the case that upper and horizontal forward lower section is shot, the situation in the non-fully-flattened in road surface（Most of situation is such in reality）, Projection approximation of the road surface in V disparity maps is parallelogram shape.In another example, the shape in region can be flat square Shape, and when carrying out structure-pixel extraction to a pixel, the rectangle major axis is tilted approximately along 135 directions, and the rectangle is long in other words Axle is approximately parallel in the projection of V disparity maps with road surface.But, for example when camera shoots road surface vertically downward, road surface is in V parallaxes Projection line in figure will no longer be incline direction, but approximate 90 degree, perpendicular to trunnion axis, then now selected to extract architectural feature The shape in the region taken can be adjusted accordingly, for example, rectangle of the major axis perpendicular to trunnion axis.Above-mentioned carry out pixel accumulation The shape in region（Including inclined angle, the size of shape etc.）It can be adjusted according to actual conditions.Because structure is special Levy consideration is the overall condition rather than an independent pixel of the pixel in a region, so weaker road surface can Strengthened accordingly.At the same time, due to calculating accumulated value, noise can effectively be removed.

In one example, the threshold value for being used to filter road surface point of each row can be dynamically determined.For example, V can be based on The architectural feature of the pixel of each column in disparity map, determines the dynamic threshold of the row.

In one example, selected road surface point should meet its Structural Eigenvalue be more than affiliated row dynamic threshold and The road surface point be all dynamic thresholds more than the row in the row pixel in it is highly minimum.

In one example, selection road surface point also includes carrying out road surface point selection only in particular detection region, and direct Pixel outside the particular detection region is excluded in outside the point candidate of road surface.For example, in one example, can be based on Historical trace information determines the first detection zone in V disparity maps and based on the video camera for obtaining disparity map and gray-scale map Parameter the second detection zone in V disparity maps is determined such as setting angle；And use the first detection zone and second The common factor of detection zone is as detection zone, wherein only carrying out road surface point selection in the detection area.So, it can significantly reduce Amount of calculation and exclusion noise jamming.

In one example, in the point selection of road surface, it is also contemplated that remove discrete point.For example, calculating road surface point away from elder generation Before the distance of road surface line that detects, and the distance for the road surface line for excluding and being previously detected from the point of road surface is more than predetermined threshold Point.

Hereinafter, Figure 11 will be referred to, the example of carry out road surface according to an embodiment of the invention point selection is described in detail.

In addition, in the application for a patent for invention by the old superfine Application No. CN201210194074.4 made of identical inventor In application for a patent for invention with Application No. CN201210513215.4, the method example of road surface point selection is also given, these Method can apply to the present invention.It is merged into herein incorporated herein by by above-mentioned patent document.

Next, in step S1500, the road instruction based on the Sign for road detected is built from disparity map V disparity maps.

The difference of road sign V disparity maps and previously described full V disparity maps is that full V disparity maps are based on whole Width anaglyph, and road sign V disparity maps are only based on the corresponding disparity map in relevant with Sign for road region Part.

According to an embodiment of the invention built from disparity map based on the road road sign detected is described below with reference to Fig. 5 The illustrative methods of the road instruction V disparity maps of will thing.

As shown in figure 5, this method using the lane line information that detects with original disparity map as input 1510, in step In S1520, according to the position of lane line, corresponding detection region of interest ROI is determined in original disparity map.The number of region of interest Mesh corresponds to the number of lane line.

Specifically, because disparity map and the coordinate of gray-scale map are corresponding or can determine Conversion Relations, therefore Its position in disparity map can be determined by the position of lane line in gray-scale map.At the same time, according to camera parameters, also Real distance can be determined in disparity map, the information such as width.Accordingly, centered on lane line, set according to actual range ROI.Actually it is determined that ROI be a rectangle in the 3 d space, but be shown as in anaglyph a trapezoidal shape.

In step S1530, based on the region of interest ROI determined in original disparity map, corresponding V disparity maps are constructed.

Hereinbefore exemplified by it there are two lane lines, illustrate to detect two lane line region of interest, so that correspondingly Construct two lane line V disparity maps.But lane line is in a unlimited number in two, and can be for one, three or more Bar.And the number of lane line may also be zero, in this case, then corresponding lane line V disparity maps will not be constructed, And then follow-up pavement detection will be carried out according only to full V disparity maps.

Built in step S1500 after the completion of road instruction V disparity maps, proceed to step S1600.

In step S1600, the second road surface point is selected from road instruction V disparity maps.

The operation of the second road surface point is selected to be similar in step from road instruction V disparity maps in step S1600 The operation that first via millet cake is selected from full V disparity maps in S1400.But it should be noted that in step S1600 from road Indicative V disparity maps select the operation of the second road surface point can be with selecting the first road surface from full V disparity maps in step S1400 The operation of point is different, can be the details of operation difference that integrated operation step is different or wherein has, or even both It is different.

In the case where there is the indicative V disparity maps of multiple Roads, as shown in Fig. 6 (a), (b) and Fig. 7 (a)-(d) In the case of, select respective road surface point from the indicative V disparity maps of each Road respectively.

In step S1600 step S1700 is proceeded to after road instruction V disparity maps the second road surface point of selection.

In step S1700, road surface is extracted based on first via millet cake and the second road surface point.

Describe according to an embodiment of the invention based on first via millet cake and the second road surface point extraction road below with reference to Fig. 8 The example of the method in face.

As shown in figure 8, first via millet cake and second road surface point 1710 of the input of this method for foregoing extraction.

In step S1720, the first via upper thread section for representing road surface is extracted based on first via millet cake；And based on the second tunnel Millet cake extracts the second road surface line segment for representing road surface.About based on point come be fitted or extract line segment method can for example using Either least square method or the combination using both approaches of Hough transformation method.Foregoing is old superfine by identical inventor The Application No. CN201210194074.4 made application for a patent for invention and Application No. CN201210513215.4 invention The road surface line segment approximating method provided in patent application can apply to the present invention.

Explanation is needed exist for, the second road surface point potentially includes multigroup road surface point, such as in foregoing detection left and right track In the case of line, then as it was previously stated, including one group of second road surface point of left side and one group of second road surface point of right side.Certainly existing Lane line not only two, but in the case of more a plurality of, then to there is multigroup second road surface point of corresponding number, Huo Zhe In the case that lane line is one, then in the presence of one group of second road surface point, and the second road surface is therefore extracted based on the second road surface point Line segment also can just include being based respectively on the second road surface of each group point to extract each corresponding the second road surface line segment.

Fig. 9 shows that each road surface line segment extracted in Sign for road is by the example of left and right lane line is unified in one Schematic diagram in individual V disparity maps, wherein the line segment indicated by numeral 1 and associated arrows is indicated based on the corresponding left side of left-hand lane line The second road surface of side point and be fitted obtained the second road surface of left side line segment, by numeral 2 and associated arrows indicate line segment indicate be based on Corresponding the second road surface of the right side point of right-hand lane line and be fitted obtained the second road surface of right side line segment, by numeral 3 and associated arrows The line segment of instruction indicates the full first via upper thread section obtained based on the corresponding first via millet cake fitting of full V disparity maps.

Extracted in step S1720 obtain first via upper thread section and（One or more）After second road surface line segment, advance To step S1730.

In step S1730, the envelope for extracting first via upper thread section and the second road surface line segment is used as road surface.I.e. based on upper Each road surface line segment for being fitted and obtaining in step S1720 is stated, their envelope is extracted as the road surface detected.

For example, continuing in case of being described above in association with Fig. 9, it is assumed that the second road surface of left side line segment, the tunnel of right side second Upper thread section and full first via upper thread section can use following formula respectively（2）、（3）、（4）Represent:

h_L(d)=k_L·d+b_L…(2)

h_R(d)=k_R·d+b_R…(3)

h_W(d)=k_W·d+b_W…(4)

Wherein, h_L(d),h_R(d),h_W(d) it is the pavement-height in each V figures, i.e. vertical dimension coordinate in V disparity maps, d It is apart from D（That is actual physics distance）Horizontal dimension coordinate in the parallax value at place, i.e. V disparity maps, k and b are the road surfaces estimated Line parameter.Then can as following formula (5), (6),（7）The shown envelope for extracting each road surface line segment is used as pavement-height scope h_E (d)：

h_E(d)=[h_Emin(d),h_Emax(d)] (5)

h_Emax(d)=max (h_L(d),h_R(d),h_W(d)) (6)

h_Emin(d)=min (h_L(d),h_R(d),h_W(d)) (7)

Wherein, wherein h_Emax(d) maximum of each road surface line segment height at parallax value d is represented, is left side second The pavement-height h of road surface line segment, right side the second road surface line segment and full first via upper thread section at parallax value d_L(d),h_R(d),h_W (d) maximum in.Wherein h_Emin(d) minimum value of each road surface line segment height at parallax value d is represented, is left side second The pavement-height h of road surface line segment, right side the second road surface line segment and full first via upper thread section at parallax value d_L(d),h_R(d),h_W (d) minimum value in.Thus, envelope is to represent the pavement-height minimum value h by each parallax value d_EminAnd pavement-height (d) Maximum h_Emax(d) the boundary line h of composition_Emax(d).And pavement-height scope h_E(d) it is in this pavement-height minimum value h_Emin (d) with pavement-height maximum h_Emax(d) between.

This method 1700 is output as the road surface envelope 1740 extracted.

Figure 10 is given based on the second road surface of left side line segment, right side the second road surface line segment and complete first road surface shown in Fig. 9 The schematic diagram for the road surface envelope that line segment is drawn.As shown in Figure 10, generally, real road surface broadside lines is not Preferable line segment, but have certain thickness, because generally road surface is in x directions（Horizontal direction）It is not severity , but have certain inclination.In Figure 10 example, each section fits the line come and describes the different part in road surface：Base The road surface line fitted in the V figures of full disparity map（Full first via upper thread section i.e. described above）The whole road of rough measurement Side height；Based on a left side, the road surface line fitted in the V figures of right-lane line（The second road surface of left side line segment i.e. described above With the second road surface of right side line segment）Respectively describe the road surface profile height in left side and right side.So embodiments in accordance with the present invention, Extract the envelope of line segment is fitted in each V figures come describe in practice it is inclined, with certain thickness road surface side profile.Should Envelope is the outline of all V figures Road upper threads, and this conforms better to actual road conditions.

In one example, lane line is two and substantially symmetrical on road surface, herein using under example, utilizes this Two groups of the left and right road surface point based on left and right V disparity maps and the road surface based on the extraction of full V disparity maps that invention above-described embodiment is extracted The extracted road surface of point can conform better to the actual conditions of road.Certainly, lane line can be asymmetrical, and lane line It is in a unlimited number in two, and can be arbitrary number.

In the examples described above, it is based respectively on different groups of road surface point to be fitted road surface line segment respectively, and extracts each road surface line The envelope of section is used as road surface.But this is only preferred exemplary, but and nonrestrictive.In another example, as alternative Implementation, first via millet cake and the second road surface point can be merged without distinction as the set of road surface point, and based on road surface Point set is used as road surface to extract road surface line segment；Or in another example, can also be combined based on whole road surfaces point and come straight The outer envelope for connecing road surface point set as extracting is used as road surface

In addition, there is lane line in the road scene being outlined above, lane line is not present in road scene In the case of, can be only with the line being fitted in the V figures based on full disparity map as final according to one embodiment of the invention Road surface broadside lines, so will be to some unstructured roads, such as backroad, rural road etc., well adapting to property.

Fig. 2 is returned to, after step S1700 terminates, pavement detection method terminates.

It described above is the overall flow of pavement detection method according to a first embodiment of the present invention.

Using pavement detection method according to a first embodiment of the present invention, while using the V parallaxes based on whole disparity map Figure with the V disparity maps based on such as Sign for road of left and right lane line, carry out road surface extraction, for tilted road surface detection very Effectively, and extract road surface more tally with the actual situation.

In addition, the embodiment of the present invention detects the Sign for road of such as lane line in gray-scale map, and based on detecting Sign for road construction V disparity maps carry out road surface straight line estimation, and the two step process are all 2-D datas, thus characteristic point Strengthened, and amount of calculation significantly reduces simultaneously.

3.2nd, the detailed example of road surface point extracting method

The example of road surface point extracting method according to an embodiment of the invention is described below with reference to Figure 11.

Figure 11 shows the flow chart of road surface point extracting method 1400 according to an embodiment of the invention.The road surface point is carried Method 1400 is taken to can apply to the point extraction step S1400 of the road surface shown in Fig. 2 and similar road surface point extraction step S1600, it is that the road instruction V of Sign for road that full V disparity maps are also based on detecting is regarded that both differences, which are to input, Difference figure.It will be described without distinction with inputting for V disparity maps below.

As shown in figure 11, input as V disparity maps 1410.

In step S1420, detection zone is limited.

By limiting detection zone, so as to subsequently only carry out road surface in the detection area rather than in view picture V disparity maps Point detection, can be substantially reduced amount of calculation, mitigate influence of noise.

In one embodiment, the detection zone in V disparity maps can be determined based on the historical trace information of road surface frame.Change Sentence is talked about, and the detection zone in V disparity maps can be limited according to the testing result of road surface historical frames.In vehicle travel process, Road surface will not undergo mutation, so there is very high probability to detect the road surface line segment of present frame near history detection zone.Figure The solid line that the testing result that road surface historical frames are representatively illustrated in 12 and the detection zone determined therefrom that, wherein label 1 are indicated The road surface line that former frame is detected is indicated, the region that the solid line that label 2 is indicated is confined indicates the inspection determined according to the testing result Region is surveyed, the region that the chain-dotted line that label 3 is indicated in addition is confined indicates the detection zone determined based on camera parameters.

In another embodiment, V can be determined based on the parameter of the video camera for obtaining disparity map and gray-scale map Detection zone in disparity map.Generally, such as in-vehicle camera parameter of the video camera during road is shot is solid It is fixed, therefore the road surface of the shot by camera is also normally in image FX.The parameter of video camera includes intrinsic parameter With outer parameter.The outer parameter of video camera for example has height of the video camera to road surface, in left and right camera in the case of binocular camera The angle on the distance of heart point, the plane of delineation and road surface etc..The intrinsic parameter of video camera is such as having camera focus.Utilize these priori Knowledge, those skilled in the art can determine detection zone of the road surface in V disparity maps based on mathematical operation.Label in Figure 12 The scope that the 3 chain-dotted line institute frames indicated take schematically shows the detection zone determined based on camera parameters.

In a further embodiment, can be determined based on historical trace information the first detection zone in V disparity maps and The second detection zone in V disparity maps is determined based on the parameter such as setting angle of the video camera for obtaining disparity map and gray-scale map Domain；And the common factor of the first detection zone of use and the second detection zone is as detection zone, wherein only entering in the detection area Walking along the street millet cake is selected.Computationally intensive big reduction can so be made, while the interference of many noises can be excluded.Need explanation It is that in the schematic diagram in Figure 12, the first detection zone determined based on historical trace information is included in based on camera parameters Inside the second detection zone determined, therefore now the common factor of the two is the first detection zone.But example is only for, actually First detection zone and the second detection zone can intersect, or the second detection zone is inside the first detection zone, this Shi Erzhe common factor is different from the first detection zone.

Defined in step S1420 after detection zone, proceed to step S1430.

In step S1430, for each pixel in detection zone, architectural feature is extracted.

As it was previously stated, in embodiments of the present invention, the architectural feature for defining a pixel is based on the pixel Centered on, in region with given shape and predefined size the accumulation of all pixels point and obtained from, the shape in the region The situation that shape is projected with road surface in V disparity maps is matched, and this is the pixel on road surface in order to when architectural feature is extracted Architectural feature can be as much as possible by neighbouring road surface point.For any pixel Pi, its architectural feature describes its neighborhood Interior pixel distribution characteristics.Figure 13 shows the architectural feature according to an embodiment of the invention for being used to calculate pixel Pi Region schematic diagram, the region be pixel Pi parallelogram neighborhood.Can be by calculating its parallelogram neighborhood The intensity level sum of interior pixel as pixel Pi architectural feature value.

Due to the architectural feature of pixel consider be pixel in a region overall condition rather than independent one Individual pixel, so weaker road surface has obtained corresponding enhancing.At the same time, can be effective due to calculating accumulated value Remove noise.

In the examples described above, in the case where binocular camera level is equipped on vehicle and basic horizontal is shot forward, The shape in the region can be taken as approximate parallelogram.As shown in figure 13, such region shape and road surface are in V disparity maps Approximately tilt the approximate line style projection matching at 135 degree of angles.More specifically, this considers following situation and set：In camera water In the case that flat mounted on a vehicle and horizontal forward lower section is shot, the situation in the non-fully-flattened in road surface（Big portion in reality The situation of dividing is such）Under, projection approximation of the road surface in V disparity maps is parallelogram shape.In another example, the shape in region Shape can be taken as flattened rectangular, and when carrying out structure-pixel extraction to a pixel, the rectangle major axis is taken as approximately along 135 directions Tilt, the rectangle major axis is approximately parallel in the projection of V disparity maps with road surface in other words.But, in another example, for example in phase When machine shoots road surface vertically downward, projection line of the road surface in V disparity maps will no longer be incline direction, but approximate 90 degree, i.e., It perpendicular to trunnion axis, then now can accordingly be adjusted to extract the shape in the region that architectural feature is chosen, for example, be taken as major axis and hang down Directly in the rectangle of trunnion axis..The shape in the region of above-mentioned carry out pixel accumulation（Including inclined angle, the size of shape etc.） It can be adjusted according to actual conditions.Due to architectural feature consider be pixel in a region overall condition and It is not an independent pixel, so weaker road surface can be strengthened accordingly.At the same time, due to calculating accumulation Value, can effectively remove noise.

Due to determining the area of the architectural feature for calculating a pixel according to projection of shape of the road surface in V disparity maps The shape in domain, therefore resulting architectural feature is more suitable for pavement detection application.

Calculated in step S1430 and obtain in detection zone after the architectural feature of each pixel, proceeding to step S1440。

In step S1440, dynamic threshold is determined.

In the prior art, fixed threshold value generally is used to whole region in the candidate point of selection detection object etc..

But, usually, the problem of determination inherently one of threshold size is relatively difficult.And such threshold value is not It must be suitable in view picture region everywhere.

Therefore, according to one embodiment of present invention, for each column of detection zone, being adaptively dynamically determined the row Threshold value.Specifically, in detection zone, each row can be scanned, the Structural Eigenvalue of each pixel is calculated respectively, are found every Maximum Structural Eigenvalue in one row.Then it is dynamic using a certain ratio such as 0.6 of the row max architecture characteristic value as the row State threshold value.In threshold value, substitute and consider max architecture characteristic value, or in addition to max architecture characteristic value additionally, It is contemplated that the average value of architectural feature, dispersion etc. carry out threshold value.

According to another embodiment of the invention, be not for each column, but for several columns such as 3 row or 4 row come One threshold value of unified setting, the threshold value is determined according to the Structural Eigenvalue of the several columns.

Determined in step S1440 after dynamic threshold, proceed to step S1450.

In step S1450, according to the architectural feature of each pixel and the dynamic threshold of its altitude feature and affiliated row Value, carries out road surface point primary election.In other words, road surface point candidate or background dot are referred to.

For each row, only retain dynamic threshold of its Structural Eigenvalue more than the row and the pixel in bottommost, As road surface candidate point, other points will be all excluded.

After tentatively choosing road surface point based on architectural feature and altitude feature in step S1450, step is proceeded to S1460。

In step S1460, discrete point is removed from the road surface point tentatively chosen.

In one embodiment, this can be by calculating distance of the road surface point away from the road surface line being previously detected, and from road The distance for the road surface line for excluding and being previously detected in millet cake is more than the point of predetermined threshold to realize.It is relevant distance for example can be Euclidean distance.Predetermined threshold can for example be taken as all Euclidean distances of the road surface point away from the road surface line previously detected tentatively chosen Average.Thus, it is possible to exclude the point away from previous road surface line.Remaining point will be used as final road surface point set.

Figure 15 (a) schematically shows the road surface point example tentatively chosen, and Figure 15 (b) is diagrammatically illustrated to be gone by discrete point The road surface point example of final selection after removing.Point in rectangle frame in Figure 15 (a) is noise, by Figure 15 (b) and Figure 15 (a) Comparison it is visible, by discrete point removal processing after, eliminated the noise.

For the V disparity maps based on full disparity map and based on a left side, the V disparity maps of right-lane line, can be respectively adopted with Upper method carries out the selection of road surface point.The selection result that Figure 16 is schematically illustrated in each V disparity maps Road millet cake is regarded in a V Set in poor figure.

Figure 17 (a1)-(c1) and (a2)-(c2) are shown for comparing conventional pav point choosing method and according to the present invention The schematic diagram of the result of the road surface point choosing method of embodiment, wherein, conventional method is based only upon altitude feature progress road surface and clicked Take, and above-mentioned road surface point choosing method according to embodiments of the present invention employs architectural feature extraction and should determine that dynamic threshold with adaptive Value measure.Specifically, two groups are given using conventional pav point choosing method and the road surface point choosing method of the embodiment of the present invention Results contrast.Figure 17 (a1) shows a V parallax illustrated example based on full disparity map, and figure (b1) is shown for figure (a1) The road surface point that shown V disparity map application conventional pav point choosing methods are obtained, figure (c1), which is shown, applies the embodiment of the present invention The obtained road surface point of road surface point choosing method（Wherein incorporate based on full V disparity maps choose road surface point and based on left and right car The road surface point that the left and right V disparity maps of diatom are chosen）, wherein caused by the point in figure (b1) in rectangle frame is the guardrail on road surface Point, from scheming the comparison of (b1) and (c1), the road surface point choosing method of the embodiment of the present invention eliminates guardrail noise spot, and It is to extract architectural feature so that road surface point is enhanced by the present invention to scheme the point in the rectangle frame of (c1), is thus retained Road surface point, it is seen that the road surface point choosing method of the embodiment of the present invention can strengthen road surface point.Similarly, Figure 17 (a2) is shown One V parallax illustrated example based on full disparity map, figure (b2) is shown for the V disparity map application conventional pavs shown in figure (a2) The road surface point that point choosing method is obtained, figure (c2) shows the road surface obtained using the road surface point choosing method of the embodiment of the present invention Point, wherein the point in figure (b2) in rectangle frame is noise, from scheming the comparison of (b2) and (c2), the road surface of the embodiment of the present invention The point that point choosing method eliminates presentation two lines section form in noise, and a rectangle frame of figure (c2) is inclination situation Under obtained road surface point, another rectangle frame shows the road surface point obtained on more accurately road surface line segment, it is seen that the present invention is implemented The road surface choosing method of example can retain inclined road surface.

It can be seen that, in the road surface choosing method according to the present embodiment, because the architectural feature of pixel is extracted, so that road surface point Strengthened, can preferably retain road surface.

In the road surface choosing method according to the present embodiment, because limiting detection zone, so that amount of calculation is greatly reduced, Improve efficiency.

In the road surface choosing method according to the present embodiment, by removing discrete point, such as fence or irrigation canals and ditches can be suppressed Deng the influence of noise.

In the road surface choosing method according to the present embodiment, because taking the dynamic threshold for being suitable for regional area such as each column Value, therefore, it is possible to adaptively determine dynamic threshold, therefore can preferably select the road surface point of each regional area.

It should be noted that in the point choosing method in road surface according to an embodiment of the invention shown in Figure 11, using Detection zone restriction, architectural feature are extracted, dynamic threshold is determined, discrete point removes all means to obtain optimal road surface point Choose result.It should be understood that, this is not offered as that road surface point selection, phase must be carried out using all these means Instead, can only take as needed in these means some or it is some carry out road surface point selection, in addition can also basis Need to add extra means or the tactful performance improve road surface point choosing method.

4th, the second embodiment of pavement detection method

Below with reference to the pavement detection method 1000 ' of Figure 18 descriptions according to a second embodiment of the present invention.

The pavement detection method of the pavement detection method of second embodiment shown in Figure 18 and the first embodiment shown in Fig. 2 Difference be road surface inverse mapping step S1800 many.Step S1800 is described in detail below.Other step S1100-S1700 can With reference to the description above in conjunction with Fig. 2, to repeat no more here.

According to one embodiment of present invention, after V disparity maps extract one or more line segment as road surface, it is desirable to The road surface extracted point is showed in disparity map, i.e., so-called road surface point inverse mapping.

In other words, what road surface point inverse mapping here referred to is reflected relevant road surface point based on the road surface determined in V disparity maps It is mapped in disparity map, so as to reflect road surface point in disparity map.

It is that inverse mapping method in road surface is shown in the case of envelope to describe the road surface extracted in V disparity maps with reference to Figure 19 Example.

As shown in figure 19, the input of road surface inverse mapping method is the road surface envelope and original disparity map in V disparity maps 1810。

In step S1820, according to the road surface envelope line computation pavement-height of extraction.For road surface envelope altitude range h_E(d) it is the h as shown in formula (5) above_E(d)=[h_Emin(d),h_Emax(d)] situation.For at D（With parallax value d）, its pavement-height should its be in h_Emin(d) between hEmax (d).

In step S1830, real road surface point is confirmed according to pavement-height in V disparity maps.Because road surface line It is fitted the road surface point of institute's foundation（The road surface point namely selected in Fig. 2 or Figure 18 step S1400 and S1600）Only it is in fact Road surface candidate point, possible its is not real road surface point.This step is after road surface envelope is extracted, according to road surface envelope Line confirms which real road surface point has, specifically, in V disparity maps a bit (d, y), according to its parallax value d, base Its h is calculated in formula (5)_E(d) value：If its y-coordinate is in h_EminAnd h (d)_Emax(d) between, then it is pavement-height point.

Finally in step S1840, the road surface point that will confirm that（Point with pavement-height）Former disparity map is returned in inverse mapping.Will It is prior art that disparity map is returned in point inverse mapping in V figures, is repeated no more here.

Return to road surface inverse mapping method in Figure 19, Figure 19 and be output as the road surface detected shown in disparity map.

The road surface point inverse mapping method of the road surface envelope based on extraction shown in Figure 19 is merely illustrative, and the present invention is not It is limited to this.For example, in another embodiment, road surface point verification step S1830 and the road in the V disparity maps in Figure 19 can be saved The step S1840 of disparity map is returned in millet cake inverse mapping, and is directly based upon formula（5）Whether a bit (x, y, d) in checking disparity map For road surface point, specifically, for a pixel (x, y, d) in disparity map, according to its parallax d, calculated using formula (5) Its h_E(d) value：If its y-coordinate is in h_EminAnd h (d)_Emax(d) between, then it is the road surface point on disparity map, and vice versa.

Figure 21 (a1)-(c1) and (a2)-(c2) show the road surface result that different detection methods are obtained.Figure 21 (a1) is shown The original gradation road image of one example；Figure 21 (b1) shows and obtained on this original gradation road image conventional method Pavement detection result（Conventional method is only based on disparity map and chooses road surface point based on altitude feature and extract the side of road surface line Method）, from Figure 21 (b1), it is seldom that its road surface point retains；Figure 21 (c1) shows Figure 18 institutes according to embodiments of the present invention Show method（Wherein, architectural feature and dynamic threshold are employed in road surface point selection step）Obtained testing result, wherein Figure 21 (c1) lane line is included in the rectangle frame in, it can be seen that it remains the part of lane line.Figure 21 (a2) shows that one is shown The original gradation road image of example, Figure 21 (b2) shows the road surface inspection obtained on this original gradation road image conventional method Result is surveyed, Figure 21 (c2) shows the testing result that method shown in Figure 18 according to embodiments of the present invention is obtained, by Figure 21 (b2) And Figure 21（c2）Middle rectangle frame part is more visible, and the conventional method part road surface is very faint, and the road of the embodiment of the present invention Face detection method has obtained enhanced road surface.As can be seen from Figure 21, because the pavement detection method of the embodiment of the present invention is in base In the selection of the enterprising walking along the street millet cake of V disparity maps of Sign for road such as left and right lane line, and road surface envelope is extracted, therefore inclined Oblique road surface is retained.Additionally, due to architectural feature and dynamic threshold is used, road surface point is strengthened.Experimental result is demonstrate,proved Understand the validity of this method.

5th, the 3rd embodiment of pavement detection method

Below with reference to the pavement detection method of Figure 22 descriptions according to a third embodiment of the present invention.

Figure 22 shows the flow chart of pavement detection method 2000 according to a third embodiment of the present invention.The present invention the 3rd is real The pavement detection method characteristic for applying example is road surface point selection part, wherein selecting road surface based on architectural feature and dynamic threshold Point.

As shown in figure 22, in step S2100, acquisition includes the disparity map on road surface.Realization about step S2100 can be with With reference to the description of the part about obtaining disparity map in step S1100 in Fig. 2, repeat no more here.

In step S2200, V disparity maps are built from disparity map.Realization about step S2200 may be referred to relevant figure The description of part in 2 in step S1300, is repeated no more here.

In step S2300, road surface point is selected.

Wherein, selection road surface point can include：

The architectural feature of each pixel in V disparity maps is obtained, the architectural feature of a pixel is based in the pixel Centered on, in region with given shape and predefined size the accumulation of all pixels point and obtained from, the shape in the region The situation that shape is projected with road surface in V disparity maps is matched；

Based on the architectural feature of the pixel of each column in V disparity maps, the dynamic threshold of the row is determined；

Wherein selected road surface point meets its Structural Eigenvalue more than the dynamic threshold of affiliated row and the road surface point is this It is highly minimum in the pixel of all dynamic thresholds more than the row in row.

It should be noted that selecting the operation of road surface point to be not precluded from the utilization of other means in step S2300.For example, It can be used in step S2300 and combine the detection zone restriction step S1420's and discrete point removal step S1460 that Figure 11 is described Operation, and other means suitable for road surface point selection can also be used.

The realization determined with dynamic threshold is extracted about the architectural feature in step S2300 in the present embodiment, be may be referred to Step S1440 description is determined previously in conjunction with the S1430 of architectural feature extraction step shown in Figure 11 and dynamic threshold.Here no longer go to live in the household of one's in-laws on getting married State.

In step S2400, road surface is extracted based on road surface point.Operation on extracting road surface based on road surface point can be such as Utilize either least square method or the combination using both approaches of Hough transformation method.Foregoing is old by identical inventor The superfine Application No. CN201210194074.4 made application for a patent for invention and Application No. N201210513215.4 hair The road surface line segment approximating method provided in bright patent application can apply to the present invention.

As previously described, because the architectural feature of pixel consider be pixel in a region overall condition without It is an independent pixel, so weaker road surface has obtained corresponding enhancing.At the same time, can due to calculating accumulated value Effectively to remove noise.

6th, the first embodiment of road surface checking device

As shown in figure 23, road surface checking device 4000 can include：Image obtains part 4100, includes road surface for obtaining Disparity map and gray-scale map；Sign for road detection part 4200, road surface position is can be identified for that for being detected from gray-scale map Sign for road；Full V disparity maps build part 4300, for building the full V disparity maps based on full figure from disparity map；The first via Millet cake alternative pack 4400, for selecting first via millet cake from full V disparity maps；Road instruction V disparity maps build part 4500, For building the road instruction V disparity maps based on the Sign for road detected from disparity map；Second road surface point selection portion Part 4600, for selecting the second road surface point from road instruction V disparity maps；And road surface extracting parts 4700, for based on Millet cake and the second road surface point extract road surface all the way.

It should be noted that the arrow between part shown in Figure 23 only represents a kind of logical relation.Even if two Arrow is not drawn between individual part, two parts are also not offered as in the absence of logical relation.

The flow chart that operation and realization on each part of road surface checking device 4000 may be referred to reference to shown in Fig. 2 is carried out Description, repeat no more here.

7th, the second embodiment of road surface checking device

As shown in figure 24, the road surface checking device can include：Image obtains part 5100, and acquisition includes the parallax on road surface Figure；V disparity maps build part 5200, for building V disparity maps from disparity map；Road surface point selection part 5300, for selecting Road surface point；And road surface extracting parts 5400, for extracting road surface based on road surface point.Wherein, road surface point selection part 5300 is selected Routing millet cake includes：The architectural feature of each pixel in V disparity maps is obtained, the architectural feature of a pixel is based in the picture In region centered on vegetarian refreshments, with given shape and predefined size the accumulation of all pixels point and obtained from, the region Shape matched with the situation that road surface is projected in V disparity maps；Based on the architectural feature of the pixel of each column in V disparity maps, really The dynamic threshold of the fixed row；The architectural feature of wherein selected road surface point meets the dynamic threshold for being more than affiliated row and the road surface Point be all dynamic thresholds more than the row in the row pixel in it is highly minimum.

It should be noted that the arrow between part shown in Figure 24 only represents a kind of logical relation.Even if two Arrow is not drawn between individual part, two parts are also not offered as in the absence of logical relation.

The flow chart that operation and realization on each part of road surface checking device 5000 may be referred to reference to shown in Figure 22 enters Capable description, is repeated no more here.

8th, system hardware configuration

The present invention can also be implemented by a kind of system on detection road surface.Figure 25 is shown according to the embodiment of the present invention The concept map of the hardware configuration of pavement detection system 6000.As shown in figure 25, pavement detection system 6000 can include：Input is set Standby 6100, for such as from outside input, by image to be processed, the left images of binocular camera shooting, stereoscopic camera are shot Stereo-picture gray level image that even general camera is shot etc., the input equipment can for example include keyboard, Genius mouse, Yi Jitong Communication network and its remote input equipment that is connected etc.；Processing equipment 6200, it is above-mentioned according to present invention implementation for implementing The pavement detection method of example, or the above-mentioned pavement detection equipment according to the embodiment of the present invention is embodied as, it can such as include The central processing unit of computer or other chips with disposal ability etc., it may be connected to the network of such as internet （It is not shown）, according to the need for processing procedure and from Network Capture data such as left images, gray level image, anaglyph Deng；Output equipment 6300, for implementing the result obtained by above-mentioned pavement detection process to outside output, for example, can include display Device, printer and communication network and its remote output devices that are connected etc.；And storage device 6400, for easy Lose or non-volatile mode stores gray level image involved by pavement detection process, anaglyph, V disparity maps, road surface point, road surface Line segment parameter etc., for example, can include random access memory（RAM）, read-only storage（ROM）, hard disk or semiconductor deposit The various volatile and nonvolatile property memories of reservoir etc..

9th, summarize

Described above is pavement detection method according to embodiments of the present invention and road surface checking device.

According to an aspect of the present invention, pavement detection method can include：Acquisition includes the disparity map and gray scale on road surface Figure；Detection can be identified for that the Sign for road of road surface position from gray-scale map；The full V parallaxes based on full figure are built from disparity map Figure；First via millet cake is selected from full V disparity maps；The road instruction based on the Sign for road detected is built from disparity map V disparity maps；The second road surface point is selected from road instruction V disparity maps；And extracted based on first via millet cake and the second road surface point Road surface.

According to an aspect of the present invention, road surface checking device can include：Image obtains part, includes road for obtaining The disparity map and gray-scale map in face；Sign for road detection part, the road of road surface position is can be identified for that for being detected from gray-scale map Road mark；Full V disparity maps build part, for building the full V disparity maps based on full figure from disparity map；First via millet cake is selected Part is selected, for selecting first via millet cake from full V disparity maps；Road instruction V disparity maps build part, for from disparity map Build the road instruction V disparity maps based on the Sign for road detected；Second road surface point selection part, for referring to from road The property shown V disparity maps select the second road surface point；And road surface extracting parts, for based on first via millet cake and the extraction of the second road surface point Road surface.

According to another aspect of the present invention, pavement detection method can include：Acquisition includes the disparity map on road surface；From regarding V disparity maps are built in poor figure；Select road surface point；And road surface is extracted based on road surface point.Wherein, selection road surface point includes：Obtain V The architectural feature of each pixel in disparity map, the architectural feature of a pixel be based on it is centered on the pixel, have The accumulation of all pixels point is with obtained from the region of given shape and predefined size, and shape and the road surface in the region are regarded in V The situation matching projected in poor figure；Based on the architectural feature of the pixel of each column in V disparity maps, the dynamic threshold of the row is determined； Wherein selected road surface point meets that its Structural Eigenvalue is more than the dynamic threshold of affiliated row and the road surface point is that own in the row It is highly minimum more than in the pixel of the dynamic threshold of the row.

According to another aspect of the present invention, road surface checking device can include：Image obtains part, and acquisition includes road surface Disparity map；V disparity maps build part, for building V disparity maps from disparity map；Road surface point selection part, for selecting road Millet cake；And road surface extracting parts, for extracting road surface based on road surface point.Wherein, road surface point selection subassembly selection road surface point can With including：Obtain the architectural feature of each pixel in V disparity maps, the architectural feature of a pixel is based on being in the pixel In center, region with given shape and predefined size the accumulation of all pixels point and obtained from, the shape in the region Situation about being projected with road surface in V disparity maps is matched；Based on the architectural feature of the pixel of each column in V disparity maps, the row are determined Dynamic threshold；The architectural feature of wherein selected road surface point meets the dynamic threshold for being more than affiliated row and the road surface point is this It is highly minimum in the pixel of all dynamic thresholds more than the row in row.

The description of the embodiment of the present invention is merely illustrative, those skilled in the art can be changed as needed, substitute or Combination.

In description above, it is described in case of Sign for road is lane line, but is only for showing Example, Sign for road not limited to this, but can be the fence on the anything that can be identified for that road surface position, such as road, Curb stone, road is on both sides of the road or middle grove etc..Specifically, for example, the bottom position of the fence on road indicates road surface position Put.Similarly, the crowd that a group is advanced on road can also be as Sign for road, because the bottom position of crowd indicates road surface Position.

In description above, it is described so that lane line is two as an example, but is only for example, the number of lane line Two can be fewer of more than.

In description above, it is applied to describe the present invention, Bu Guoben under the situation of drive assist system in pavement detection The pavement detection method and apparatus of inventive embodiments can apply to other situations for needing to carry out pavement detection.

The general principle of the present invention is described above in association with specific embodiment, however, it is desirable to, it is noted that to this area For those of ordinary skill, it is to be understood that the whole or any steps or part of methods and apparatus of the present invention, Ke Yi Any computing device（Including processor, storage medium etc.）Or in the network of computing device, with hardware, firmware, software or Combinations thereof is realized that this is that those of ordinary skill in the art use them in the case where having read the explanation of the present invention Basic programming skill can be achieved with.

Therefore, the purpose of the present invention can also by run on any computing device a program or batch processing come Realize.The computing device can be known fexible unit.Therefore, the purpose of the present invention can also be included only by offer Realize that the program product of the program code of methods described or device is realized.That is, such program product is also constituted The present invention, and the storage medium for such program product that is stored with also constitutes the present invention.Obviously, the storage medium can be Any known storage medium or any storage medium developed in the future.

It may also be noted that in apparatus and method of the present invention, it is clear that each part or each step are to decompose And/or reconfigure.These decompose and/or reconfigured the equivalents that should be regarded as the present invention.Also, perform above-mentioned series The step of processing can order naturally following the instructions perform in chronological order, but and need not necessarily sequentially in time Perform.Some steps can be performed parallel or independently of one another.

Above-mentioned embodiment, does not constitute limiting the scope of the invention.Those skilled in the art should be bright It is white, depending on design requirement and other factors, can occur various modifications, combination, sub-portfolio and replacement.It is any Modifications, equivalent substitutions and improvements made within the spirit and principles in the present invention etc., should be included in the scope of the present invention Within.

Claims

1. a kind of pavement detection method, including：

Acquisition includes the disparity map and gray-scale map on road surface；

Detection can be identified for that the Sign for road of road surface position from gray-scale map；

The full V disparity maps based on full figure are built from disparity map；

First via millet cake is selected from full V disparity maps；

The road instruction V disparity maps based on the Sign for road detected are built from disparity map；

The second road surface point is selected from road instruction V disparity maps；And

Road surface is extracted based on first via millet cake and the second road surface point,

Wherein, it is described to select first via millet cake from full V disparity maps and/or select the second road surface point from road instruction V disparity maps Including：

Obtain the architectural feature of each pixel in V disparity maps, the architectural feature of a pixel be based on using the pixel as In center, region with given shape and predefined size the accumulation of all pixels point and obtained from, the shape in the region Situation about being projected with road surface in V disparity maps is matched；

Wherein selected road surface point meets its Structural Eigenvalue and is more than the dynamic threshold of affiliated row and during the road surface point is the row It is highly minimum in the pixel of all dynamic thresholds more than the row.

2. pavement detection method according to claim 1, described to be included based on first via millet cake and the second road surface point extraction road surface：

The first via upper thread section for representing road surface is extracted based on first via millet cake；

The the second road surface line segment for representing road surface is extracted based on the second road surface point；And

The envelope for obtaining first via upper thread section and the second road surface line segment is used as road surface.

3. pavement detection method according to claim 2, wherein the Sign for road is substantially symmetrical on road Sign for road,

Wherein from gray-scale map detection can be identified for that road surface position Sign for road include detection on the left of Sign for road and The Sign for road on right side；

The road instruction V disparity maps based on the Sign for road detected are wherein built from disparity map to be included building based on inspection The indicative V disparity maps of left side road of the Sign for road in the left side measured and the Sign for road based on the right side detected The indicative V disparity maps of right side road；

Wherein the second road surface point is selected to include from the indicative V disparity maps selection left side of left side road from road instruction V disparity maps Second road surface point and from the second road surface point of right side road indicative V disparity maps selection right side；

The the second road surface line segment for representing road surface is wherein extracted based on the second road surface point including extracting left based on the second road surface of left side point Side the second road surface line segment and based on the second road surface of right side point extract right side the second road surface line segment and；And

Wherein obtain the envelope of first via upper thread section and the second road surface line segment includes obtaining first via upper thread section, a left side as road surface The envelope of side the second road surface line segment and the second road surface of right side line segment is used as road surface.

4. pavement detection method according to claim 1, wherein selection road surface point also includes：

The first detection zone in V disparity maps is determined and based on for obtaining disparity map and gray-scale map based on historical trace information The parameter of video camera determine the second detection zone in V disparity maps；And

Using the common factor of the first detection zone and the second detection zone as detection zone, wherein only entering walking along the street in the detection area Millet cake is selected.

5. pavement detection method according to claim 1, wherein selection road surface point also includes：

Calculate distance of the road surface point away from the road surface line being previously detected, and the road surface line for excluding and being previously detected from the point of road surface Distance be more than predetermined threshold point.

6. pavement detection method according to claim 2, described to be included based on first via millet cake and the second road surface point extraction road surface：

According to the road surface envelope line computation pavement-height scope detected；

Road surface point is selected in V disparity maps based on pavement-height scope；And

Disparity map is returned into road surface point inverse mapping in V disparity maps.

7. a kind of pavement detection method, including：

Acquisition includes the disparity map on road surface；

V disparity maps are built from disparity map；

Select road surface point；And

Road surface is extracted based on road surface point,

Wherein, selection road surface point includes：

The architectural feature of each pixel in V disparity maps is obtained, the architectural feature of a pixel is based in the pixel is In the heart, region with given shape and predefined size the accumulation of all pixels point and obtained from, the shape in the region with The situation matching that road surface is projected in V disparity maps；

8. a kind of road surface checking device, including：

Image obtains part, includes the disparity map and gray-scale map on road surface for obtaining；

Sign for road detection part, the Sign for road of road surface position is can be identified for that for being detected from gray-scale map；

Full V disparity maps build part, for building the full V disparity maps based on full figure from disparity map；

First via millet cake alternative pack, for selecting first via millet cake from full V disparity maps；

Road instruction V disparity maps build part, for building the road based on the Sign for road detected from disparity map Indicative V disparity maps；

Second road surface point selection part, for selecting the second road surface point from road instruction V disparity maps；And

Road surface extracting parts, for extracting road surface based on first via millet cake and the second road surface point,

Wherein, first via millet cake alternative pack and the second road surface point selection part are configured as：

9. a kind of road surface checking device, including：

Image obtains part, and acquisition includes the disparity map on road surface；

V disparity maps build part, for building V disparity maps from disparity map；

Road surface point selection part, for selecting road surface point；And

Road surface extracting parts, for extracting road surface based on road surface point,

Wherein, road surface point selection subassembly selection road surface point includes：

The architectural feature of wherein selected road surface point meets the dynamic threshold for being more than affiliated row and the road surface point is institute in the row Have highly minimum in the pixel more than the dynamic threshold of the row.