CN110335322A - Roads recognition method and road Identification device based on image - Google Patents
Roads recognition method and road Identification device based on image Download PDFInfo
- Publication number
- CN110335322A CN110335322A CN201910614644.2A CN201910614644A CN110335322A CN 110335322 A CN110335322 A CN 110335322A CN 201910614644 A CN201910614644 A CN 201910614644A CN 110335322 A CN110335322 A CN 110335322A
- Authority
- CN
- China
- Prior art keywords
- road
- doubtful
- line
- line segment
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000000605 extraction Methods 0.000 claims abstract description 24
- 238000001914 filtration Methods 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims abstract description 12
- 238000011049 filling Methods 0.000 claims abstract description 10
- 230000003447 ipsilateral effect Effects 0.000 claims abstract description 10
- 239000000203 mixture Substances 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 2
- 239000000284 extract Substances 0.000 abstract description 7
- 230000000694 effects Effects 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 9
- 238000012549 training Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 241000209094 Oryza Species 0.000 description 2
- 235000007164 Oryza sativa Nutrition 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 239000010426 asphalt Substances 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 239000004568 cement Substances 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000011017 operating method Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 235000009566 rice Nutrition 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 239000002023 wood Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of roads recognition method based on image and road Identification devices, and the method comprising the steps of: according to the rgb value of pixel each in images to be recognized, judging doubtful road area and non-doubtful road area；Super-resolution rebuilding is carried out using trained SRGAN model to the doubtful road area, obtains the doubtful road area image of higher resolution, and gaussian filtering process is carried out to non-doubtful road area；Wavelet threshold denoising processing is carried out to the doubtful road area image after super-resolution rebuilding；The edge line in image exported after wavelet threshold denoising is handled using Canny operator extraction；The edge line extracted is filled up and organized into groups, realization connects together the ipsilateral penumbra line of same road, and the two sides of same road are bound together；The filling road region between edge line on both sides of the road, obtains closed road.The resolution ratio of image can be improved in the method for the present invention, accelerates edge line extraction speed, and relatively accurately extract complete road.
Description
Technical field
This application involves depth learning technology field, in particular to a kind of roads recognition method and road based on image is known
Other device.
Background technique
The type of traditional road extraction is broadly divided into two kinds.The first is directly to look for road, i.e., profile is extracted from image
Edge or frame find a pair of of parallel lines as roadside from image, but are constrained to the office of image resolution ratio and recognizer
Limit, extraction accuracy be not high；Second is the method for utilizing image segmentation, extracts mesh by gray value, threshold value and image information
Mark region, but since imaging surface is smooth, the result of different zones output may with other linear ground objects, as gully,
The mixing such as river.
Summary of the invention
The purpose of the present invention is to provide a kind of roads recognition method based on image and road Identification devices, to improve
The precision of road identification.
In order to achieve the above-mentioned object of the invention, the embodiment of the invention provides following technical schemes:
A kind of roads recognition method based on image, comprising the following steps:
According to the rgb value of pixel each in images to be recognized, doubtful road area and non-doubtful road area are judged；
Super-resolution rebuilding is carried out using trained SRGAN model to the doubtful road area, obtains more high-resolution
The doubtful road area image of rate, and gaussian filtering process is carried out to non-doubtful road area；
Wavelet threshold denoising processing is carried out to the doubtful road area image after super-resolution rebuilding；
The edge line in image exported after wavelet threshold denoising is handled using Canny operator extraction；
The edge line extracted to be filled up and organized into groups, realization connects together the ipsilateral penumbra line of same road,
And the two sides of same road are bound together；
The filling road region between edge line on both sides of the road, obtains closed road.
In the scheme advanced optimized, after the step of edge line that described pair is extracted is filled up and organized into groups,
The line segment for extending the road edge line after marshalling is further comprised the steps of:, final road edge line is obtained.
In the scheme advanced optimized, after judging doubtful road area and non-doubtful road area, further include
Step: using each pixel for belonging to doubtful road area as origin, using several pixels as radius, by the area within the scope of this
Domain is defined as doubtful road area.
In the scheme advanced optimized, described the step of gaussian filtering process is carried out to non-doubtful road area, comprising:
It is first filtered using the first convolution kernel, reuses the second convolution kernel and be filtered, the first convolution kernel is less than second
Convolution kernel.
On the other hand, the embodiment of the invention also provides a kind of road Identification devices, comprising: judgment module is used for basis
The rgb value of each pixel in images to be recognized judges doubtful road area and non-doubtful road area；Module is rebuild, is used for
Super-resolution rebuilding is carried out using trained SRGAN model to the doubtful road area, obtains the doubtful of higher resolution
Road area image, and gaussian filtering process is carried out to non-doubtful road area；Module is denoised, after to super-resolution rebuilding
Doubtful road area image carry out wavelet threshold denoising processing；Edge line extraction module, for using Canny operator extraction to pass through
Cross the edge line in the image exported after wavelet threshold denoising processing；Module is organized into groups, for filling out to the edge line extracted
It mends and organizes into groups, realization connects together the ipsilateral penumbra line of same road, and the two sides of same road are bound together；It fills out
Mold filling block obtains closed road for being filled to the region between the line of road two edges.
The embodiment of the invention also provides a kind of computer readable storage medium including computer-readable instruction, the meters
The step of calculation machine readable instruction makes processor realize any roads recognition method when executing described program when executed.
The embodiment of the invention also provides a kind of electronic equipment, the electronic equipment includes processor and is stored in storage
On device and the computer program that can run on a processor, the processor realize any road Identification side when executing described program
The step of method.
Compared with prior art, image beneficial effects of the present invention: is improved in certain limit by super-resolution technique
Resolution ratio, rather than must Price-dependent high high-resolution satellite image；On extraction effect, using fast speed
Edge extracting can also preferably extract most of road (can extract comparatively fine road) in image, simultaneously
By the improvement of GAN algorithm, can accomplish to extract road surface, rather than just a line segment.
Detailed description of the invention
It, below will be to use required in embodiment in order to illustrate more clearly of the technical solution of the application embodiment
Attached drawing be briefly described, it should be understood that the following drawings illustrates only some embodiments of the application, therefore is not to be seen as
It is the restriction to range, it for those of ordinary skill in the art, without creative efforts, can be with root
Other relevant attached drawings are obtained according to these attached drawings.
Fig. 1 is the flow diagram for the roads recognition method based on image that present invention implementation provides；
Fig. 2 is original remote sensing image figure provided in an embodiment of the present invention；
Fig. 3 is super-resolution striograph provided in an embodiment of the present invention；
Fig. 4 is to extract the effect picture behind edge using Canny operator to Fig. 2；
Fig. 5 is to extract the effect picture behind edge using Canny operator to Fig. 3；
Fig. 6 a-c is the different disposal process schematic of road edge line；
Fig. 7 is the flow chart that edge line segment extends step in roads recognition method；
Fig. 8 is the result figure of road extraction provided in this embodiment；
Fig. 9 is the functional block diagram of road Identification device in the present embodiment；
Figure 10 is the structural schematic diagram of the electronic equipment provided in the embodiment of the present invention.
Specific embodiment
To keep the purposes, technical schemes and advantages of the application embodiment clearer, implement below in conjunction with the application
The technical solution in the application embodiment is clearly and completely described in attached drawing in mode, it is clear that described reality
The mode of applying is a part of embodiment of the application, rather than whole embodiments.Based on the embodiment in the application, ability
Domain those of ordinary skill every other embodiment obtained without creative efforts, belongs to the application
The range of protection.
Therefore, requirement is not intended to limit to the detailed description of the presently filed embodiment provided in the accompanying drawings below to protect
The scope of the present application of shield, but it is merely representative of the selected embodiment of the application.Based on the embodiment in the application, ability
Domain those of ordinary skill every other embodiment obtained without creative efforts, belongs to the application
The range of protection.
Embodiment
The type of road extraction traditional at present can be broadly divided into two kinds.The first is directly to look for road, i.e., from image
Contour edge or frame are extracted, a pair of of parallel lines are found from image as roadside, but be constrained to image resolution ratio and identification
The limitation of algorithm, extraction accuracy be not high；Second is the method for utilizing image segmentation, passes through gray value, threshold value and image information
Extract target area, but since imaging surface is smooth, the result of different zones output may with other linear ground objects,
Such as gully, river mix, and there is also the low problems of road Identification precision.
Inventors have found that road extraction can also according to the thought of study, machine can from the initial data of Pixel-level to
Abstract semantic concept successively extracts information, automatic learning objective feature, thus find the right road, but this method needs
Enough training set and test set and good learning algorithm are wanted, therefore also there is certain difficulty and challenge.
Referring to Fig. 1, a kind of roads recognition method based on image provided in the present embodiment, comprising the following steps:
Step S1: according to the rgb value of pixel each in images to be recognized, judge doubtful road area and non-doubtful road
Region.
Road Identification described in the present embodiment is mainly the two class roads extracted in image, and one kind is the cement in cities and towns
Road, another kind of is the asphalt roads between cities and towns and city.By to a large amount of different remote sensing image (shooting time phase
Together, when such as being the morning nine to ten) road RGB information acquisition, find the rgb value of cement road be (165-185,
170-190,170-190), the rgb value of asphalt roads is (95-115,105-115,110-125), and therefore, this step is judging
When doubtful road area, meets rgb value and be the pixel of (165-185,170-190,170-190), and meet rgb value and be
The pixel of (95-115,105-115,110-125) is judged as belonging to road sections, i.e., all pixels for meeting condition
The region of point composition is doubtful road area.For example, the rgb value of a pixel be (169,182,170) because 169
The section 165-185,182 in the section 170-190, and 170 in the section 170-190, so the pixel is judged as belonging to doubtful road
Road region.
In addition, inevitably have tree shade, vehicle in road or because of roadway pollution etc. due to cause pixel uneven, can
Can occur biggish with threshold value (i.e. (165-185,170-190,170-190) and (95-115,105-115,110-125)) difference
Therefore region is missed in order to avoid belonging to the pixel of real roads, as an embodiment, can be with each
The pixel for being chosen to be road is origin, using several (such as 15) pixels as radius, by the region within the scope of this
It is defined as doubtful road area.
Step S2: super-resolution rebuilding is carried out using trained SRGAN model to the region for being identified as doubtful road, is obtained
Gaussian filtering process is carried out to the area image of the doubtful road of higher resolution, and to non-doubtful road area.
Training SRGAN model when, use 1200 to (be only for example herein, the quantity of training sample can be freely arranged,
It is theoretically The more the better) remote sensing image of different satellite shooting is used as training sample, is divided into two parts, and a part is as training
Training set is exactly inputted initial model as test set, training process, then carries out effect by test set by collection, another part
Judgement, constantly adjust neural network parameter when reaching good effect preservation model.Be exactly when use to
The model that the image input of super-resolution rebuilding saves, output result are the image of higher resolution.A pair of of remote sensing image
Including a high-definition picture and a low-resolution image, wherein high-definition picture is original remote sensing image, low resolution
Rate image is to please refer to Fig. 2 and Fig. 3 at random using a variety of filtering algorithms to original remote sensing image treated image, Fig. 2 shows
The original remote sensing image figure in Sichuan Province, region, the town Jiangyou City Ma Jiaoba, Fig. 3 are the corresponding super-resolution striograph of Fig. 2.
It should be noted that being improved original GAN algorithm, being compared in training SRGAN model in the present embodiment
Wasserstein distance is introduced in original GAN algorithm, is specifically sentenced compared to 4 points: 1. of original GAN algorithm improvement
Other device the last layer removes sigmoid.2. the loss of generator and arbiter does not take log.3. updating the parameter of arbiter every time
Their absolute value is truncated to no more than one fixed constant C afterwards.4. using RMSProp optimization algorithm.Wasserstein away from
From effect be exactly solve gradient disappear and reduce balance generator and arbiter training degree difficulty.Gradient is solved to disappear
Principle to be Wassertein distance have good flatness compared to KL divergence and JS divergence, use mathematic(al) manipulation will
Wassertein distance is write as the form that can be solved, and the arbiter neural network being limited using a parameter values range is come most
Change this form greatly, so that it may approximate W asserstein distance.Optimize generator under this near-optimization arbiter to make
Wasserstein distance reduces, and can effectively further generation distribution and true distribution, preferably resolves asking for gradient disappearance
Topic is more nearly so that generating distribution with true distribution, the also more adjunction of the road edge for the higher resolution image rebuild
It is bordering on the edge of true picture, traditional super-resolution is considerably reduced and rebuilds bring burr.
It will (be directly doubtful road in the present embodiment to the image of super-resolution rebuilding after trained SRGAN model
Region) the trained SRGAN model of input, i.e., the image of exportable higher resolution.
Gaussian filtering process is carried out to non-rice habitats part (i.e. being judged as non-doubtful road area in step S1).This implementation
It in example, is operated when specific operation using two steps, the first step is first filtered using the convolution kernel of such as 3*3, and second step reuses
Such as the convolution kernel of 7*7 is filtered, i.e., is filtered again in the result after first step filtering, twice using different big
Small convolution kernel is filtered, to improve filtration efficiency.The reason of being filtered operation is to reduce non-rice habitats part to rear
The interference generated when continuous road extraction, and the speed of road extraction can be greatly improved.Successively use 3*3, the convolution kernel of 7*7
Purpose be to first pass through the convolution kernel of 3*3 to be filtered operation, small range reduces the lines letter on high-frequency information, such as house
Breath；The bigger convolution kernel for passing through 7*7 again, so that the line information between house is weakened.Use convolution kernel of different sizes twice
It is filtered compared to one group of convolution kernel is only used, the influence in dominant interference information such as house can be greatly reduced.
It is readily comprehensible, in step sl, an image is divided into doubtful road area and non-doubtful road area,
It is to handle respectively doubtful road area and non-doubtful road area, i.e., same image will handle two in this step S2
It is secondary, it is once to handle doubtful road area, another time is handled non-doubtful road area.
Step S3: denoising behaviour is carried out using wavelet threshold denoising method to the doubtful road area image after super-resolution rebuilding
Make.
In this step, concrete operations are as follows:
A) the doubtful road area image after super-resolution rebuilding is considered as 2D signal and carries out wavelet transformation, obtain one group
Wavelet decomposition system ωJ, k；
B) by coefficient of wavelet decomposition ωJ, kHard threshold function processing is carried out, estimation wavelet coefficient u is obtainedJ, k, so that
ωJ, k-uJ, kIt is minimum.The expression formula of hard threshold function are as follows:Wherein threshold tau=3 α, α are that noise criteria is poor.
The threshold value assumes that the probability that the normally distributed variable of zero-mean is fallen in except [- 3 α, 3 α] is 0, i.e., general absolute value is less than 3 α
Coefficient be to be generated by noise.
C) the wavelet coefficient ω of estimation is utilizedJ, kCarry out wavelet reconstruction, the signal after being denoised.
Step S4: the edge line for the image that Canny operator extraction exports after step S3 processing is used.Please refer to Fig. 4
And Fig. 5, Fig. 4 are to extract the effect picture behind edge using Canny operator to Fig. 2, Fig. 5 be to Fig. 3 using Canny operator into
Row extracts the effect picture behind edge.
This step casts out the gaussian filtering operation of former Canny operator, specifically includes the following steps:
A) gradient intensity of each pixel and direction in the image exported after step S3 processing were calculated.
B) non-maxima suppression is applied, to eliminate edge detection bring spurious response.
C) it detects using dual threshold to determine true and potential edge.
D) by inhibiting isolated weak edge to be finally completed edge detection.
The gaussian filtering operation that this step casts out former Canny operator is because having carried out wavelet threshold to image in step S3
Denoising, the experiment proved that, for the image after super-resolution rebuilding, the method for wavelet threshold denoising is better than gaussian filtering.
Step S5: the step S4 edge line extracted is filled up and is organized into groups.There is step simple screening behaviour before filling up
Make, the marginal point number for including using edge line replaces the screening of edge line physical length, and being exactly will include marginal point number
30% gives up after entirety.Because can generate many short and small edge lines by edge extracting (is the woods or filling mostly
The edge of wood etc.), and these edge lines are not have effective to road extraction, even the road edge being scattered, still can not
It uses, therefore road Identification efficiency can be improved in operation of giving up herein.
It is filling up specific steps are as follows: Canny operator extraction goes out edge after, the pixel of marginal point is labeled as 0, non-side
The pixel of edge point is labeled as 255.Each marginal point is traversed, being tracked to its 8 neighborhood (is exactly centered on a marginal point
3*3 range, eight pixels are 8 neighborhoods around central pixel point, and tracking finds that be located at the pixel value of 8 neighborhoods be 0
Point), the point that pixel is 0 is found, then carry out next step tracking in its (point found) 8 neighborhood, until can not find pixel is 0
Until point.So go down to find each edge line, then takes its front and back endpoint to find edge on its 8 neighborhood boundary the edge line
Point fills up point all on straight line if having.
Marshalling specific steps are as follows: there are two the purposes of marshalling, a is to connect the ipsilateral penumbra line of same road
To together, b is to be bound together the two sides of same road.It will form many line segments after filling up operation, according to these lines
The relative position of section is organized into groups, and first carries out step a, then carry out step b.It is given below with reference to Fig. 6 a, Fig. 6 b, Fig. 6 c and Fig. 7
With explanation.
The marshalling rule of step a is that line segment is divided into following several situations:
A1, two line segments are adjacent (adjacent is usually the road of turning, more rare): line segment L2 is with endpoint P1 (or endpoint
P2) be the center of circle, formed respectively using R1 and R2 as the concentric circles of radius, wherein the density of the length of R1, R2 and surrounding line segment and
The length of L2 is related, such as the line segment around line segment L2 is more, then R1, and R2 is just shorter, is once linear between length and density
Function, maximum value (limiting R2) is the 1/3 of line segment L2, and minimum value (limiting R1) is the 1/10 of L2, and wherein R1 is always than R2
It is short.It is to have some tiny line segments in step S5 screening that the reason of two radiuses of different sizes look for line segment L1, which is arranged,
Do not screened out, but they are not belonging to road again, the circle at this moment requiring two radiuses to draw all intersects to sentence with target segment
Fixed condition, so that it may more accurately find the ipsilateral penumbra of same road.If line segment L1 is just and using R1 and R2 as radius
Circle all intersects, and all only one intersection points, then L2 is compiled as one group and connected its nearest endpoint by line segment L1.If not yet
L1 is found, then expands radius and continues to look for, until R2 reaches maximum value (such as the 1/3 of line segment L2), as shown in Figure 6 a.
A2, two line segments are conllinear (being collinearly used for general straight way road, most of is this): the direction of line segment L1 and line segment L2
Difference and lateral distance are both less than threshold value, angle theta of the direction difference between two line segments, and prescribed threshold is such as 3 °, and lateral distance is
To the vertical range of L1, the threshold value of vertical range is, for example, 10 pixels at the midpoint of L2.Judge just to compile after two line segments are conllinear as one
Group, and its nearest endpoint is linked up.
The marshalling rule of step b is that the L1 quartering is obtained three quartering points, including two endpoint number consecutivelies 1,2,
3,4,5.Using each endpoint as the center of circle, neighborhood search is carried out since 3 pixels are radius, until being gradually expanded to 10 pictures
Vegetarian refreshments, or stop search after meeting the following conditions: wherein 1 and No. 5 has an intersection point that counting (record can be added with line segment L3
The number of intersection point), remaining 2,3, No. 4 must count with line segment L3 there are two intersection point and in ipsilateral be just added, and have 3 in five points
It is a to reach count condition, and intersection point can determine that line segment L3 and line segment L1 is the two sides of same road at the same side
Line, it is one group that line segment L3 and line segment L1, which is compiled, as shown in Figure 6 b.
It should be noted that above-mentioned steps a and step b do not have point of sequencing.
Repeat the above steps the operation of a and step b, until all marshalling is completed.
It is two big group that grouping activity, which simultaneously compiles two edge lines of road, wherein being located at the edge of left side and downside
Line is A group, and the edge line of right side and upside is B group, it is therefore an objective to prevent from generating interference when road extends.The b of marshalling is operated
The two sides of same road are bound together, left and right (upper and lower) is the positional relationship at same both sides of the road edge.Such as left and right
The judgment method of relationship is two pixel x coordinates being located on two sidelines on horizontal line direction.
Step S6: road seeds extend, and obtain road edge line.
By step S5 processing after, at a distance of closer line segment warp knit be one group obtain the main body of road, if still there is phase
It then needs accurately to be extended away from farther away line segment, obtains final road edge line.Two edge lines of road will be prolonged
It stretches, B group is not involved in when A group is extended, and extension method is identical.For this sentences one edge line, the treatment process of this step
It is as follows:
A) endpoint for enabling side (edge) line is A point, and the range points that distance A point four is had a lot of social connections again on sideline are B point, forms one
New line segment L4 (i.e. BA) obtains new point C according to the length that the position of L4 extends half of L4, if C point has exceeded image side
Boundary then extends L4 to image boundary backed off after random circulation；If C point in image capturing range, carries out in next step.
B) line segment AC is set as line segment L5, and using C point as dot, the length of line segment BC is radius, neighborhood search is carried out, if finding
Point with being intersected using line segment BC as the circle that radius is drawn, then judge whether line segment belonging to crosspoint and the line segment where AB are conllinear,
The position of C point order as new A point if conllinear, repeats the above steps and a) b) is operated with this step, until with another line
Section connection, circulation all reduces the step-length of neighborhood search (radius of search is other than first time is BC, such as each searches every time
Rope radius is the 2/3 of original search radius, that is, BC)；If do not find the point intersected with it or crosspoint it is not conllinear if abandon prolonging
It stretches.
After A group, B group edge line are completed to extend, judge that the line segment relationship of elongated area (judges that A group and B group extend
Whether region line segment is parallel, and as shown in 6c, region shown in dotted line is elongated area), if exceeding the threshold value of parallel condition, repeat
Step S6, until meeting condition, i.e., without departing from the threshold value of parallel condition.The determination of the threshold value of parallel condition be with different images and
Variation, after carrying out grouping activity, count line segment length preceding 80% line segment angle, clustered.Each threshold value
Range be exactly cluster in by paracentral 70% line segment angle difference.It can be understood as the 70% of normal distribution, only not
The clustering used here as machine learning is crossed to do, such benefit is that threshold value setting can be made more flexible more acurrate.
Step S7: the filling road region between line on both sides of the road, extracted as a result, i.e. side sideline endpoint do it is another
The vertical line of side line obtains closed road, referring to Fig. 8, Fig. 8 is the result figure of road extraction.
Referring to Fig. 9, being based on identical inventive concept, the embodiment of the present invention provides a kind of road Identification device simultaneously,
Including consisting of module:
Judgment module is judged doubtful road area and non-is doubted for the rgb value according to pixel each in images to be recognized
Like road area；
Module is rebuild, for carrying out super-resolution rebuilding using trained SRGAN model to the doubtful road area,
The doubtful road area image of higher resolution is obtained, and gaussian filtering process is carried out to non-doubtful road area；
Module is denoised, for carrying out wavelet threshold denoising processing to the doubtful road area image after super-resolution rebuilding；
Edge line extraction module, the image for being exported after wavelet threshold denoising is handled using Canny operator extraction
In edge line；
Module is organized into groups, for the edge line extracted to be filled up and organized into groups, is realized the ipsilateral penumbra of same road
Line connects together, and the two sides of same road are bound together；
Extension of module obtains final road edge line for extending the line segment of the road edge line after organizing into groups；
It fills module and obtains closed road for being filled to the region between the line of road two edges.
The above-mentioned specific implementation procedure of modules may refer to the corresponding description in aforementioned roads recognition method, to save a piece
Width, details are not described herein again.
As shown in Figure 10, the present embodiment provides a kind of electronic equipment simultaneously, which may include processor 51
With memory 52, wherein memory 52 is coupled to processor 51.It is worth noting that, the figure is exemplary, can also use
The structure is supplemented or substituted to other kinds of structure, realizes that data are extracted, chart is redrawn, communicates or other function.
As shown in Figure 10, which can also include: input unit 53, display unit 54 and power supply 55.It is worth note
Meaning, the electronic equipment are also not necessary to include all components shown in Fig. 5.In addition, electronic equipment can also wrap
The component being not shown in Fig. 5 is included, the prior art can be referred to.
Processor 51 is sometimes referred to as controller or operational controls, may include microprocessor or other processor devices and/
Or logic device, the processor 51 receive the operation of all parts of input and controlling electronic devices.
Wherein, memory 52 for example can be buffer, flash memory, hard disk driver, removable medium, volatile memory, it is non-easily
The property lost one of memory or other appropriate devices or a variety of, can store configuration information, the processor 51 of above-mentioned processor 51
The instruction of execution, record the information such as image data.Processor 51 can execute the program of the storage of memory 52, to realize information
Storage or processing etc..It in one embodiment, further include buffer storage in memory 52, i.e. buffer, with the intermediate letter of storage
Breath.
Input unit 53 can be for example document reading apparatus, for providing road image to be identified to processor 51.
Display unit 54 is used to show that the processing image during road Identification, the display unit for example can be LCD display, but this
Invention is not limited to this.Power supply 55 is used to provide electric power for electronic equipment.
The embodiment of the present invention also provides a kind of computer-readable instruction, wherein when executing described instruction in the electronic device
When, described program makes electronic equipment execute the operating procedure that the roads recognition method based on image as shown in Figure 1 is included,
Or a part of step in method as shown in Figure 1.
The embodiment of the present invention also provides a kind of storage medium for being stored with computer-readable instruction, wherein the computer can
Reading instruction makes electronic equipment execute the operating procedure that roads recognition method as shown in Figure 1 is included, or side as shown in Figure 1
A part of step in method.
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware
With the interchangeability of software, each exemplary composition and step are generally described according to function in the above description.This
A little functions are implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Specially
Industry technical staff can use different methods to achieve the described function each specific application, but this realization is not
It is considered as beyond the scope of this invention.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components
It can be combined or can be integrated into another system, or some features can be ignored or not executed.In addition, shown or beg for
Opinion mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING of device or unit
Or communication connection, it is also possible to electricity, mechanical or other form connections.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.Some or all of unit therein can be selected to realize the embodiment of the present invention according to the actual needs
Purpose.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, is also possible to two or more units and is integrated in one unit.It is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can be personal computer, server or the network equipment etc.) executes the complete of each embodiment the method for the present invention
Portion or part steps.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only
Memory), random access memory (RAM, Random Access Memory), magnetic or disk etc. are various can store journey
The medium of sequence code.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. a kind of roads recognition method based on image, which comprises the following steps:
According to the rgb value of pixel each in images to be recognized, doubtful road area and non-doubtful road area are judged；
Super-resolution rebuilding is carried out using trained SRGAN model to the doubtful road area, obtains higher resolution
Doubtful road area image, and gaussian filtering process is carried out to non-doubtful road area；
Wavelet threshold denoising processing is carried out to the doubtful road area image after super-resolution rebuilding；
The edge line in image exported after wavelet threshold denoising is handled using Canny operator extraction；
The edge line extracted is filled up and organized into groups, realization connects together the ipsilateral penumbra line of same road, and will
The two sides of same road are bound together；
The filling road region between edge line on both sides of the road, obtains closed road.
2. the method according to claim 1, wherein the edge line extracted at described Dui is filled up and is organized into groups
The step of after, further comprise the steps of:
The line segment for extending the road edge line after marshalling, obtains final road edge line.
3. the method according to claim 1, wherein the rgb value according to pixel each in images to be recognized,
The step of judging doubtful road area and non-doubtful road area, comprising:
Judge the rgb value of pixel, if rgb value meet (165-185,170-190,170-190) or (95-115,105-115,
110-125), then judge that the pixel belongs to doubtful road area, be otherwise not belonging to doubtful road area, it is all to belong to doubtful road
The region of the pixel composition in road region is doubtful road area, the area of all pixel compositions for belonging to non-doubtful road area
Domain is non-doubtful road area.
4. the method according to claim 1, wherein the step of described pair of edge line extracted is filled up,
Include:
The pixel that Canny operator extraction goes out the marginal point at edge is labeled as 0, the pixel of non-edge point is labeled as 255；
Each marginal point is traversed, its 8 neighborhood is tracked, finds the point that pixel is labeled as 0, then 8 neighbours in the point found
Domain carries out next step tracking, until it can not find the point that pixel is labeled as 0；
After finding each edge line, its front and back endpoint is taken to find marginal point on its 8 neighborhood boundary the edge line, if there is edge
Point then fills up point all on straight line.
5. the method according to claim 1, wherein the step of described pair of edge line extracted is organized into groups,
Include:
Step a is formed using one of endpoint of line segment L2 as the center of circle respectively using R1 and R2 as the concentric circles of radius, if line
Section L1 just intersects with by the circle of radius of R1 and R2, and all only one intersection points, then compiling line segment L1, L2 for one group simultaneously
By its nearest endpoint connection；If the direction difference and lateral distance of line segment L1 and line segment L2 are both less than threshold value, by line segment
L1, L2 are compiled as one group and are connected its nearest endpoint, angle of the direction difference between two line segments, and the lateral distance is
Vertical range of the midpoint of L2 to L1；
The line segment L1 quartering is obtained three quartering points, including two endpoint number consecutivelies 1,2,3,4,5, with every by step b
A endpoint is the center of circle, and neighborhood search is carried out since 3 pixels are radius, until being gradually expanded to 10 pixels or completely
Stop search after sufficient the following conditions: wherein 1 and No. 5 has an intersection point that counting can be added with line segment L3, remaining 2,3, No. 4 necessary
It is counted with line segment L3 there are two intersection point and in ipsilateral be just added, there are 3 to reach count condition in five points, and intersection point exists
When the same side, that is, it can determine that line segment L3 and line segment L1 is the two sides sideline of same road, being compiled is one group；
Repeat the above steps the operation of a and step b, until all marshalling is completed.
6. according to the method described in claim 2, it is characterized in that, it is described extend marshalling after road edge line line segment,
The step of obtaining final road edge line, comprising:
A) endpoint for enabling edge line is A point, and the range points that distance A point four is had a lot of social connections again on edge line are B point, formed one it is new
Line segment BA obtains new point C according to the length that the position of line segment BA extends half of line segment BA, if C point has exceeded image boundary,
Then extend line segment BA to image boundary backed off after random circulation；If C point carries out next step b) in image capturing range；
B) set line segment AC as line segment L5, using C point as dot, the length of line segment BC is radius, carry out neighborhood search, if find and its
The point of intersection then judges whether line segment belonging to crosspoint and the line segment where AB are conllinear, orders the position of C point if conllinear
It for new A point, repeats the above steps and a) b) is operated with this step, is connect until with another line segment, circulation all reduces adjacent every time
The step-length of domain search；If do not find the point intersected with it or crosspoint it is not conllinear if abandon extending.
7. a kind of road Identification device characterized by comprising
Judgment module judges doubtful road area and non-doubtful road for the rgb value according to pixel each in images to be recognized
Road region；
Module is rebuild, for carrying out super-resolution rebuilding using trained SRGAN model to the doubtful road area, is obtained
The doubtful road area image of higher resolution, and gaussian filtering process is carried out to non-doubtful road area；
Module is denoised, for carrying out wavelet threshold denoising processing to the doubtful road area image after super-resolution rebuilding；
Edge line extraction module, in the image for being exported after wavelet threshold denoising is handled using Canny operator extraction
Edge line；
Module is organized into groups, for the edge line extracted to be filled up and organized into groups, realizes and connects the ipsilateral penumbra line of same road
It is connected to together, and the two sides of same road is bound together；
It fills module and obtains closed road for being filled to the region between the line of road two edges.
8. device according to claim 7, which is characterized in that further include extension of module, for extending the road after organizing into groups
The line segment of Road Edge line obtains final road edge line.
9. a kind of electronic equipment including memory, processor and stores the calculating that can be run on a memory and on a processor
Machine program, which is characterized in that the processor realizes the step of claim 1-6 any the method when executing described program.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The step of any one of claim 1-6 the method is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910614644.2A CN110335322B (en) | 2019-07-09 | 2019-07-09 | Road recognition method and road recognition device based on image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910614644.2A CN110335322B (en) | 2019-07-09 | 2019-07-09 | Road recognition method and road recognition device based on image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110335322A true CN110335322A (en) | 2019-10-15 |
| CN110335322B CN110335322B (en) | 2024-03-01 |
Family
ID=68144473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910614644.2A Active CN110335322B (en) | 2019-07-09 | 2019-07-09 | Road recognition method and road recognition device based on image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110335322B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112825131A (en) * | 2019-11-21 | 2021-05-21 | 通用汽车环球科技运作有限责任公司 | Image-based three-dimensional lane detection |
| CN114937212A (en) * | 2022-07-26 | 2022-08-23 | 南通华锐软件技术有限公司 | Aerial photography road type identification method based on frequency domain space conversion |
| CN115393813A (en) * | 2022-08-18 | 2022-11-25 | 中国人民公安大学 | Road identification method, device and equipment based on remote sensing image and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0944629A (en) * | 1995-07-25 | 1997-02-14 | Nippon Telegr & Teleph Corp <Ntt> | Method and device for generating wide area map |
| CN101334263A (en) * | 2008-07-22 | 2008-12-31 | 东南大学 | Circular target circular center positioning method |
| CN101763512A (en) * | 2009-12-11 | 2010-06-30 | 西安电子科技大学 | Method for semi-automatically detecting road target in high-resolution remote sensing images |
| CN101901343A (en) * | 2010-07-20 | 2010-12-01 | 同济大学 | Remote sensing image road extracting method based on stereo constraint |
| JP2012176641A (en) * | 2011-02-25 | 2012-09-13 | Suzuki Motor Corp | Detection apparatus for parking frame |
| CN106023180A (en) * | 2016-05-17 | 2016-10-12 | 李迎春 | Unstructured road RGB entropy segmentation method |
| CN106874875A (en) * | 2017-02-17 | 2017-06-20 | 武汉理工大学 | A kind of vehicle-mounted lane detection system and method |
| CN107909010A (en) * | 2017-10-27 | 2018-04-13 | 北京中科慧眼科技有限公司 | A kind of road barricade object detecting method and device |
| CN109886200A (en) * | 2019-02-22 | 2019-06-14 | 南京邮电大学 | A kind of unmanned lane line detection method based on production confrontation network |
-
2019
- 2019-07-09 CN CN201910614644.2A patent/CN110335322B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0944629A (en) * | 1995-07-25 | 1997-02-14 | Nippon Telegr & Teleph Corp <Ntt> | Method and device for generating wide area map |
| CN101334263A (en) * | 2008-07-22 | 2008-12-31 | 东南大学 | Circular target circular center positioning method |
| CN101763512A (en) * | 2009-12-11 | 2010-06-30 | 西安电子科技大学 | Method for semi-automatically detecting road target in high-resolution remote sensing images |
| CN101901343A (en) * | 2010-07-20 | 2010-12-01 | 同济大学 | Remote sensing image road extracting method based on stereo constraint |
| JP2012176641A (en) * | 2011-02-25 | 2012-09-13 | Suzuki Motor Corp | Detection apparatus for parking frame |
| CN106023180A (en) * | 2016-05-17 | 2016-10-12 | 李迎春 | Unstructured road RGB entropy segmentation method |
| CN106874875A (en) * | 2017-02-17 | 2017-06-20 | 武汉理工大学 | A kind of vehicle-mounted lane detection system and method |
| CN107909010A (en) * | 2017-10-27 | 2018-04-13 | 北京中科慧眼科技有限公司 | A kind of road barricade object detecting method and device |
| CN109886200A (en) * | 2019-02-22 | 2019-06-14 | 南京邮电大学 | A kind of unmanned lane line detection method based on production confrontation network |
Non-Patent Citations (3)
| Title |
|---|
| SM EASA等: "Use of Statellite Imagery for Establishing Road Horizontal Alignments", 《BING》 * |
| YING-YUE LI 等: "Edge Enhanced Super-Resolution", 《IEEE》 * |
| 尹训怡: "目标地物提取在优化地震测线部署中的应用", 《中国优秀硕士学位论文全文数据库（电子期刊）信息科技辑》, vol. 2012, no. 04, pages 13 - 22 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112825131A (en) * | 2019-11-21 | 2021-05-21 | 通用汽车环球科技运作有限责任公司 | Image-based three-dimensional lane detection |
| CN112825131B (en) * | 2019-11-21 | 2023-09-01 | 通用汽车环球科技运作有限责任公司 | Image-based three-dimensional lane detection |
| CN114937212A (en) * | 2022-07-26 | 2022-08-23 | 南通华锐软件技术有限公司 | Aerial photography road type identification method based on frequency domain space conversion |
| CN114937212B (en) * | 2022-07-26 | 2022-11-11 | 南通华锐软件技术有限公司 | Aerial photography road type identification method based on frequency domain space conversion |
| CN115393813A (en) * | 2022-08-18 | 2022-11-25 | 中国人民公安大学 | Road identification method, device and equipment based on remote sensing image and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110335322B (en) | 2024-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109509192B (en) | Semantic segmentation network integrating multi-scale feature space and semantic space | |
| Zhang et al. | Deep gated attention networks for large-scale street-level scene segmentation | |
| CN107644426A (en) | Image, semantic dividing method based on pyramid pond encoding and decoding structure | |
| CN110335322A (en) | Roads recognition method and road Identification device based on image | |
| DE102019130889A1 (en) | ESTIMATE THE DEPTH OF A VIDEO DATA STREAM TAKEN BY A MONOCULAR RGB CAMERA | |
| CN107480726A (en) | A kind of Scene Semantics dividing method based on full convolution and shot and long term mnemon | |
| GB2580671A (en) | A computer vision system and method | |
| Zhang et al. | ST-unet: Swin transformer boosted U-net with cross-layer feature enhancement for medical image segmentation | |
| CN105261017A (en) | Method for extracting regions of interest of pedestrian by using image segmentation method on the basis of road restriction | |
| CN106651774A (en) | License plate super-resolution model reconstruction method and device | |
| CN110334709B (en) | License plate detection method based on end-to-end multi-task deep learning | |
| CN108197644A (en) | A kind of image-recognizing method and device | |
| Woźniak et al. | Graphic object feature extraction system based on cuckoo search algorithm | |
| KR20200092841A (en) | Method and computing device for generating image data set to be used for hazard detection and learning method and learning device using the same | |
| CN103235929B (en) | Identification method and identification device on basis of hand vein images | |
| CN110349167A (en) | A kind of image instance dividing method and device | |
| CN104573742A (en) | Medical image classification method and system | |
| CN110738663A (en) | Double-domain adaptive module pyramid network and unsupervised domain adaptive image segmentation method | |
| CN108241819A (en) | The recognition methods of pavement markers and device | |
| Deshmukh et al. | Swin transformer based vehicle detection in undisciplined traffic environment | |
| CN104112138A (en) | Object color classification method and device | |
| CN113505670A (en) | Remote sensing image weak supervision building extraction method based on multi-scale CAM and super-pixels | |
| CN112488229A (en) | Domain self-adaptive unsupervised target detection method based on feature separation and alignment | |
| CN108460336A (en) | A kind of pedestrian detection method based on deep learning | |
| CN111611933A (en) | Information extraction method and system for document image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |