CN110335322B - Road recognition method and road recognition device based on image - Google Patents
Road recognition method and road recognition device based on image Download PDFInfo
- Publication number
- CN110335322B CN110335322B CN201910614644.2A CN201910614644A CN110335322B CN 110335322 B CN110335322 B CN 110335322B CN 201910614644 A CN201910614644 A CN 201910614644A CN 110335322 B CN110335322 B CN 110335322B
- Authority
- CN
- China
- Prior art keywords
- road
- line segment
- edge
- suspected
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000001914 filtration Methods 0.000 claims abstract description 20
- 238000000605 extraction Methods 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 14
- 238000004590 computer program Methods 0.000 claims description 3
- 238000009954 braiding Methods 0.000 claims 2
- 239000000284 extract Substances 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 8
- 238000012549 training Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008034 disappearance Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 239000010426 asphalt Substances 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 239000004568 cement Substances 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007621 cluster analysis Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000001454 recorded image Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Abstract
The invention relates to a road identification method and a road identification device based on images, wherein the method comprises the following steps: according to RGB values of each pixel point in the image to be identified, judging a suspected road area and a non-suspected road area; performing super-resolution reconstruction on the suspected road area by using a trained SRGAN model to obtain a suspected road area image with higher resolution, and performing Gaussian filtering treatment on a non-suspected road area; carrying out wavelet threshold denoising treatment on the suspected road area image after super-resolution reconstruction; extracting edge lines in the image output after wavelet threshold denoising processing by using a Canny operator; filling and grouping the extracted edge lines, so as to connect the same side edge lines of the same road together and bind two sides of the same road together; filling the area between the two edge lines of the road to obtain a closed road. The method can improve the resolution of the image, quicken the extraction speed of the edge line and extract the complete road more accurately.
Description
Technical Field
The present disclosure relates to the field of deep learning technologies, and in particular, to a road recognition method and a road recognition device based on images.
Background
The types of conventional road extraction are mainly divided into two types. The first is to find the way directly, namely extract the outline edge or frame from the picture, find a pair of parallel lines as the roadside from the picture, but limited by resolution of the picture and limitation of recognition algorithm, the extraction precision is not high; the second is a method of using image segmentation, which extracts a target region through gray values, threshold values, and image information, but since the image surface is smooth, the results output from different regions may be mixed with other linear ground objects such as mountain furrows, rivers, etc.
Disclosure of Invention
The invention aims to provide a road identification method and a road identification device based on images, so as to improve the accuracy of road identification.
In order to achieve the above object, the embodiment of the present invention provides the following technical solutions:
an image-based road recognition method, comprising the steps of:
according to RGB values of each pixel point in the image to be identified, judging a suspected road area and a non-suspected road area;
performing super-resolution reconstruction on the suspected road area by using a trained SRGAN model to obtain a suspected road area image with higher resolution, and performing Gaussian filtering treatment on a non-suspected road area;
carrying out wavelet threshold denoising treatment on the suspected road area image after super-resolution reconstruction;
extracting edge lines in the image output after wavelet threshold denoising processing by using a Canny operator;
filling and grouping the extracted edge lines, so as to connect the same side edge lines of the same road together and bind two sides of the same road together;
filling the area between the two edge lines of the road to obtain a closed road.
In a further optimized solution, after the step of filling and grouping the extracted edge lines, the method further includes the steps of: and extending the line segments of the road edge lines after grouping to obtain the final road edge lines.
In a further optimized scheme, after judging the suspected road area and the non-suspected road area, the method further comprises the steps of: and taking each pixel point belonging to the suspected road area as an origin, taking a plurality of pixel points as radiuses, and defining the area within the range as the suspected road area.
In a further optimized scheme, the step of performing gaussian filtering processing on the non-suspected road area includes: and filtering by using a first convolution kernel, and then filtering by using a second convolution kernel, wherein the first convolution kernel is smaller than the second convolution kernel.
On the other hand, the embodiment of the invention also provides a road identification device, which comprises: the judging module is used for judging a suspected road area and a non-suspected road area according to the RGB value of each pixel point in the image to be identified; the reconstruction module is used for carrying out super-resolution reconstruction on the suspected road area by using the trained SRGAN model to obtain a suspected road area image with higher resolution, and carrying out Gaussian filtering treatment on a non-suspected road area; the denoising module is used for performing wavelet threshold denoising treatment on the suspected road region image after super-resolution reconstruction; the edge line extraction module is used for extracting edge lines in the image output after wavelet threshold denoising processing by using a Canny operator; the grouping module is used for filling and grouping the extracted edge lines, connecting the same side edge lines of the same road together and binding two sides of the same road together; and the filling module is used for filling the area between the two edge lines of the road to obtain a closed road.
Embodiments of the present invention also provide a computer-readable storage medium comprising computer-readable instructions which, when executed, cause a processor to perform the steps of any road identification method when the program is executed.
The embodiment of the invention also provides electronic equipment, which comprises a processor and a computer program stored on a memory and capable of running on the processor, wherein the processor realizes the steps of any road identification method when executing the program.
Compared with the prior art, the invention has the beneficial effects that: the resolution of the image is improved to a certain limit by the super resolution technology, and the image does not need to rely on high-resolution satellite images which are high in price; in the extraction effect, the edge extraction with higher speed is adopted, most roads in the image can be better extracted (finer roads can be extracted), and meanwhile, the road surface can be extracted through the improvement of the GAN algorithm, and the road surface is not just a line segment.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some examples of the present application and therefore should not be considered as limiting the scope, and that other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of an image-based road recognition method according to the embodiment of the present invention;
FIG. 2 is a diagram of an original remote sensing image according to an embodiment of the present invention;
FIG. 3 is a super-resolution image provided by an embodiment of the present invention;
FIG. 4 is an effect diagram of FIG. 2 after edges are extracted using the Canny operator;
FIG. 5 is an effect diagram of FIG. 3 after edge extraction using the Canny operator;
FIGS. 6a-c are schematic diagrams of different processes for road edge lines;
FIG. 7 is a flowchart showing an edge segment extension step in the road recognition method;
fig. 8 is a graph of the result of road extraction provided in the present embodiment;
fig. 9 is a schematic diagram of a functional module of the road identifying device in the present embodiment;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some of the embodiments of the present application, but not all of the embodiments. All other embodiments, based on the embodiments herein, which would be apparent to one of ordinary skill in the art without undue burden are within the scope of the present application.
Accordingly, the following detailed description of the embodiments of the present application, provided in the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, based on the embodiments herein, which would be apparent to one of ordinary skill in the art without undue burden are within the scope of the present application.
Examples
The types of conventional road extraction can be divided into two main types. The first is to find the way directly, namely extract the outline edge or frame from the picture, find a pair of parallel lines as the roadside from the picture, but limited by resolution of the picture and limitation of recognition algorithm, the extraction precision is not high; the second is to extract the target region by using the gray value, the threshold value and the image information by using the image segmentation method, but since the image surface is smooth, the result output from different regions may be mixed with other linear ground objects such as mountain ditches, rivers, etc., and there is also a problem of low road recognition accuracy.
The inventor finds that the road extraction can also be based on the learning thought, and the machine can extract information layer by layer from the original data of the pixel level to the abstract semantic concept, automatically learn the target characteristics, so as to find the correct road, but the method needs enough training sets and test sets and a good learning algorithm, thus having certain difficulty and challenges.
Referring to fig. 1, the image-based road recognition method provided in the present embodiment includes the following steps:
step S1: and judging a suspected road area and a non-suspected road area according to the RGB value of each pixel point in the image to be identified.
The road recognition described in this embodiment mainly extracts two types of roads in the image, one type is a cement road in town, and the other type is an asphalt road in town and city. By acquiring road RGB information of a large number of different remote sensing images (the photographing time is the same, for example, nine to ten am), the RGB values of the cement road are found to be (165-185, 170-190, 170-190) and the RGB values of the asphalt road are found to be (95-115, 105-115, 110-125), so that the pixel points satisfying the RGB values (165-185, 170-190, 170-190) and the pixel points satisfying the RGB values (95-115, 105-115, 110-125) are judged to belong to the road part, that is, the region composed of all the pixel points satisfying the conditions is the suspected road region when the suspected road region is judged. For example, since the RGB value of one pixel is (169,182,170) and 169 is in the 165-185 range, 182 is in the 170-190 range, and 170 is in the 170-190 range, the pixel is determined to belong to the suspected road region.
In addition, since the pixels are inevitably uneven due to shade, vehicles, or road pollution in the road, etc., there may occur regions that are significantly different from the threshold values (i.e., (165-185, 170-190, 170-190) and (95-115, 105-115, 110-125)), in order to avoid missing pixels belonging to the real road, as an embodiment, each pixel selected as the road may be defined as a suspicious road region with a radius of several (e.g., fifteen) pixels.
Step S2: and performing super-resolution reconstruction on the region identified as the suspected road by using the trained SRGAN model to obtain a region image of the suspected road with higher resolution, and performing Gaussian filtering treatment on the non-suspected road region.
When the SRGAN model is trained, 1200 pairs (only by way of example, the number of training samples can be freely set, and theoretically, the more and better) of remote sensing images shot by different satellites are used as training samples, and the two parts are divided into two parts, wherein one part is used as a training set, the other part is used as a test set, the training process is to input the training set into an initial model, then the effect is judged through the test set, and parameters of the neural network are continuously adjusted until the model is saved when the good effect is achieved. When in use, the image to be reconstructed with super resolution is input into the stored model, and the output result is the image with higher resolution. The pair of remote sensing images includes a high resolution image and a low resolution image, wherein the high resolution image is an original remote sensing image, the low resolution image is an image obtained by processing the original remote sensing image by using a plurality of filtering algorithms at random, please refer to fig. 2 and fig. 3, fig. 2 shows an original remote sensing image of a certain area of a dam in a city of river, and fig. 3 is a super resolution image corresponding to fig. 2.
It should be noted that, in the embodiment, when the SRGAN model is trained, the original GAN algorithm is improved, compared with the original GAN algorithm, the wasperstein distance is introduced, and in particular, compared with the original GAN algorithm, four points are improved: 1. the last layer of the arbiter removes the sigmoid.2. The loss of the generator and arbiter does not take log.3. The absolute values of the parameters of the arbiter are truncated to not more than a fixed constant C after each update. 4. An RMSProp optimization algorithm was used. The function of the Wasserstein distance is to solve the difficulty of gradient disappearance and reducing the degree of training of the balance generator and the discriminant. The principle of solving gradient disappearance is that the Wassentin distance has good smoothness compared with KL divergence and JS divergence, the Wassentin distance is written into a form which can be solved by using mathematical transformation, and the Wassentin distance can be approximated by using a discriminator neural network with limited parameter numerical range to maximize the form. The optimization generator under the approximate optimal discriminant reduces the Wasserstein distance, so that the generated distribution and the real distribution can be effectively pulled up, the problem of gradient disappearance is better solved, the generated distribution and the real distribution are more similar, the road edge of the reconstructed higher-resolution image is also more similar to the edge of the real image, and burrs caused by the traditional super-resolution reconstruction are greatly reduced.
After the SRGAN model is trained, the image to be reconstructed with super resolution (i.e. the suspected road area in this embodiment) is directly input into the trained SRGAN model, and then the image with higher resolution can be output.
The non-road portion (i.e., the road area determined to be non-suspicious in step S1) is subjected to gaussian filtering. In this embodiment, two steps are adopted in the specific operation, the first step uses a convolution kernel such as 3*3 for filtering, and the second step uses a convolution kernel such as 7*7 for filtering, that is, the second step filters on the result after the first step, and the second step uses convolution kernels with different sizes for filtering, so as to improve the filtering efficiency. The reason for performing the filtering operation is to reduce interference generated by the non-road portion to the subsequent road extraction, and to greatly increase the speed of road extraction. The purpose of adopting the convolution kernels of 3×3 and 7×7 is to perform filtering operation through the convolution kernel of 3*3, so that high-frequency information, such as line information on a house, is reduced in a small range; the line information between houses is weakened by the larger convolution kernel of 7*7. The use of two convolution kernels of different magnitudes can greatly reduce the impact of dominant interference information, such as a house, compared to using only one set of convolution kernels for filtering.
It is easy to understand that in step S1, one image is divided into a suspected road area and a non-suspected road area, and in step S2, the suspected road area and the non-suspected road area are respectively processed, that is, the same image is processed twice, one is processed for the suspected road area and the other is processed for the non-suspected road area.
Step S3: and denoising the suspected road area image after super-resolution reconstruction by using a wavelet threshold denoising method.
In the step, the specific operation is as follows:
a) Taking the suspected road area image reconstructed by super resolution as a two-dimensional signal to carry out wavelet transformation to obtain a group of wavelet decomposition system omega j，k ；
b) By decomposing coefficient omega for wavelet j，k Performing hard threshold function processing to obtain estimated wavelet coefficient u j，k So that omega j，k -u j，k Minimum. The expression for the hard threshold function is:where the threshold τ=3α, α is the noise standard deviation. The threshold is a normal distribution variable assuming zero mean falls at [ -3α,3α]The other probabilities are 0, i.e. coefficients with an absolute value generally smaller than 3α are generated by noise.
c) Using estimated wavelet coefficients omega j，k And carrying out wavelet reconstruction to obtain a denoised signal.
Step S4: and (3) extracting edge lines of the image which is output after the processing of the step S3 by using a Canny operator. Referring to fig. 4 and 5, fig. 4 is an effect diagram of fig. 2 after extracting edges using a Canny operator, and fig. 5 is an effect diagram of fig. 3 after extracting edges using a Canny operator.
The Gaussian filtering operation of the original Canny operator is omitted, and the method specifically comprises the following steps:
a) The gradient strength and the gradient direction of each pixel point in the image processed in the step S3 are calculated.
b) Non-maximum suppression is applied to eliminate spurious responses from edge detection.
c) Dual threshold detection is applied to determine true and potential edges.
d) Edge detection is ultimately accomplished by suppressing isolated weak edges.
The step of eliminating the Gaussian filter operation of the original Canny operator is because the step S3 of carrying out wavelet threshold denoising on the image, and experiments prove that the wavelet threshold denoising method for the image after super-resolution reconstruction is superior to Gaussian filter.
Step S5: and (3) filling and grouping the edge lines extracted in the step S4. Before filling, a simple screening operation is performed, and the number of edge points contained in the edge line is used for replacing the screening of the actual length of the edge line, namely, the number of the edge points is discarded after 30% of the whole edge line is contained. The discard operation herein can improve road recognition efficiency because many short edge lines (mostly edges of forests or bushes, etc.) are generated by edge extraction, and these edge lines have no effect on road extraction even if scattered road edges are still unusable.
The filling specific operation steps are as follows: after the Canny operator extracts the edge, the pixels of the edge points are marked as 0, and the pixels of the non-edge points are marked as 255. Traversing each edge point, tracking 8 neighborhoods (namely, a 3*3 range taking one edge point as the center, eight pixel points around the center pixel point are 8 neighborhoods, namely, searching for a point with a pixel value of 0 in the 8 neighborhoods), finding a point with a pixel value of 0, and then carrying out next tracking in the 8 neighborhoods (found points) until the point with the pixel value of 0 cannot be found. And then, each edge line is found, the front end point and the rear end point of the edge line are taken for searching edge points at the 8 neighborhood boundary of the edge line, and if the edge points exist, all the points on the straight line are filled.
The specific operation steps of the marshalling are as follows: the purpose of the grouping is two, a is to connect the same side edge lines of the same road together, and b is to bind the two sides of the same road together. After the filling operation, a plurality of line segments are formed, grouping is carried out according to the relative positions of the line segments, and then the step a is carried out, and the step b is carried out. The following is a description of fig. 6a, 6b, 6c and 7.
The grouping rule of step a is to divide the line segments into the following cases:
a1, two line segments are adjacent (adjacent is a road which turns generally, less frequently): the line segment L2 is centered on the end point P1 (or the end point P2) to form concentric circles with R1 and R2 as radii, respectively, where the lengths of R1 and R2 are related to the density of surrounding line segments and the length of L2, for example, if there are more line segments around the line segment L2, then R1 and R2 are shorter, the length and the density are linear functions, and there is a maximum value (defining R2) that is 1/3 of the line segment L2, and a minimum value (defining R1) that is 1/10 of L2, where R1 is always shorter than R2. The reason for setting two radii with different sizes to find the line segment L1 is that some fine line segments are not screened out in the step S5, but do not belong to the road, and then the circles drawn by the two radii are required to intersect with the target line segment as a judging condition, so that the same side edge of the same road can be found more accurately. If line segment L1 just intersects both circles with radii R1 and R2 and there is only one intersection point, then line segments L1, L2 are grouped and the nearest endpoints are connected. If L1 is not found, the enlarged radius continues to find until R2 reaches a maximum (e.g., 1/3 of line segment L2), as shown in FIG. 6 a.
a2, the two line segments are collinear (collinear is used for a common straight road, and most of the line segments are the same): the difference in direction and the lateral distance between the line segments L1 and L2 are both smaller than a threshold value, the difference in direction being the angle θ between the two line segments, the threshold value being defined as, for example, 3 °, the lateral distance being the vertical distance from the midpoint of L2 to L1, the threshold value being, for example, 10 pixels. After the two line segments are judged to be collinear, the two line segments are compiled into a group, and the nearest endpoints of the two line segments are connected.
The grouping rule in the step b is to divide the L1 into four equal parts to obtain three equal parts, wherein the three equal parts comprise two end points with serial numbers of 1,2,3,4 and 5. Starting neighborhood searching from 3 pixel points as radius by taking each end point as a circle center until the neighborhood searching is gradually expanded to 10 pixel points, or stopping searching after the following conditions are met: wherein, the number 1 and the number 5 and the line segment L3 have an intersection point, namely the count can be added (the number of the intersection points is recorded), the other numbers 2,3 and 4 have two intersection points with the line segment L3 and are positioned on the same side, the count is added, 3 of the five points reach the count condition, and when the intersection points are positioned on the same side, the line segment L3 and the line segment L1 are judged to be two side edges of the same road, and the line segment L3 and the line segment L1 are organized into a group, as shown in fig. 6 b.
It should be noted that, the steps a and b are not sequentially separated.
Repeating the operations of the step a and the step b until all the grouping is completed.
The grouping operation simultaneously groups two edge lines of the road into two large groups, wherein the edge lines on the left side and the lower side are the A groups, and the edge lines on the right side and the upper side are the B groups, so as to prevent interference when the road extends. The two sides of the same road are bound together through the operation b of the grouping, and the left side and the right side (up and down) are the position relationship of the two side edges of the same road. For example, the left-right relationship is determined by x coordinates of two pixel points located on two side lines respectively in the horizontal line direction.
Step S6: and (5) extending the road seed points to obtain the road edge line.
After the processing in step S5, the segments with relatively close distances are already warp-knitted into a group of main bodies for obtaining the road, and if the segments with relatively far distances still exist, accurate extension is required to be performed, so as to obtain the final road edge line. Two edge lines of the road are required to be extended, and the group B does not participate in the extension process when the group A extends, and the extension method is the same. Taking an edge line as an example, the processing procedure of this step is as follows:
a) The end point of the edge line is made to be the point A, the distance point which is four times the road width from the point A on the edge line is made to be the point B, a new line segment L4 (namely BA) is formed, the new point C is obtained by extending half the length of L4 according to the position of L4, and if the point C exceeds the image boundary, the L4 is extended to the image boundary and then the cycle is exited; if the C point is within the image range, the next step is performed.
b) Setting a line segment AC as a line segment L5, taking a C point as a round point, taking the length of the line segment BC as a radius, carrying out neighborhood search, judging whether the line segment which the intersection point belongs to is collinear with the line segment where the AB is located if the intersection point is found, if the intersection point is collinear with the line segment where the AB is located, then, setting the position of the C point as a new A point, repeating the operation of the step a) and the step b) until the intersection point is connected with another line segment, and reducing the step length of neighborhood search every time (except for the first time of BC, for example, the search radius every time is 2/3 of the original search radius, namely BC); if no point is found that intersects it or the points of intersection are not collinear, the extension is abandoned.
After the edge lines of the group a and the group B are extended, judging the line segment relationship of the extension area (namely judging whether the line segments of the extension areas of the group a and the group B are parallel or not, as shown in 6c, and the area shown by the dotted line is the extension area), if the line segments exceed the threshold value of the parallel condition, repeating the step S6 until the condition is met, namely the threshold value of the parallel condition is not exceeded. The threshold value of the parallel condition is determined according to different images, and after the grouping operation is carried out, the angles of the line segments with the lengths of 80% in front are counted and clustering is carried out. The range of each threshold is the difference in segment angles near the center 70% of the cluster. It can be understood that 70% of the normal distribution is only done here using machine-learned cluster analysis, which has the advantage that the threshold setting can be made more flexible and accurate.
Step S7: filling the area between the two side lines of the road to obtain an extraction result, namely, making a perpendicular line of the other side line through the end point of one side line to obtain a closed road, referring to fig. 8, fig. 8 is a graph of the result of road extraction.
Referring to fig. 9, based on the same inventive concept, an embodiment of the present invention also provides a road recognition device, including the following components:
the judging module is used for judging a suspected road area and a non-suspected road area according to the RGB value of each pixel point in the image to be identified;
the reconstruction module is used for carrying out super-resolution reconstruction on the suspected road area by using the trained SRGAN model to obtain a suspected road area image with higher resolution, and carrying out Gaussian filtering treatment on a non-suspected road area;
the denoising module is used for performing wavelet threshold denoising treatment on the suspected road region image after super-resolution reconstruction;
the edge line extraction module is used for extracting edge lines in the image output after wavelet threshold denoising processing by using a Canny operator;
the grouping module is used for filling and grouping the extracted edge lines, connecting the same side edge lines of the same road together and binding two sides of the same road together;
the extension module is used for extending the line segments of the grouped road edge lines to obtain final road edge lines;
and the filling module is used for filling the area between the two edge lines of the road to obtain a closed road.
The specific execution process of each module can be referred to the corresponding description in the road recognition method, and for the sake of space saving, the description is omitted here.
As shown in fig. 10, the present embodiment also provides an electronic device that may include a processor 51 and a memory 52, wherein the memory 52 is coupled to the processor 51. It is noted that the diagram is exemplary, and that other types of structures may be used in addition to or in place of the structures to implement data extraction, chart redrawing, communication, or other functions.
As shown in fig. 10, the electronic device may further include: an input unit 53, a display unit 54, and a power supply 55. It is noted that the electronic device need not necessarily include all of the components shown in fig. 5. In addition, the electronic device may further comprise components not shown in fig. 5, to which reference is made to the prior art.
The processor 51, sometimes also referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, which processor 51 receives inputs and controls the operation of the various components of the electronic device.
The memory 52 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable medium, a volatile memory, a nonvolatile memory, or other suitable devices, and may store information such as configuration information of the processor 51, instructions executed by the processor 51, and recorded image data. The processor 51 may execute programs stored in the memory 52 to realize information storage or processing, and the like. In one embodiment, a buffer memory, i.e., a buffer, is also included in memory 52 to store intermediate information.
The input unit 53 may be, for example, a file reading device for providing the processor 51 with the road image to be identified. The display unit 54 is for displaying the processed image in the road recognition process, and may be, for example, an LCD display, but the present invention is not limited thereto. The power supply 55 is used to provide power to the electronic device.
Embodiments of the present invention also provide a computer readable instruction, wherein the program when executed in an electronic device causes the electronic device to perform the operational steps involved in the image-based road identification method as shown in fig. 1, or a part of the steps in the method as shown in fig. 1.
Embodiments of the present invention also provide a storage medium storing computer-readable instructions that cause an electronic device to perform the operational steps involved in the road identification method shown in fig. 1, or a portion of the steps in the method shown in fig. 1.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices, or elements, or may be an electrical, mechanical, or other form of connection.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present invention.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. An image-based road recognition method, characterized by comprising the steps of:
according to RGB values of each pixel point in the image to be identified, judging a suspected road area and a non-suspected road area;
performing super-resolution reconstruction on the suspected road area by using a trained SRGAN model to obtain a suspected road area image with higher resolution, and performing Gaussian filtering treatment on a non-suspected road area;
carrying out wavelet threshold denoising treatment on the suspected road area image after super-resolution reconstruction;
extracting edge lines in the image output after wavelet threshold denoising processing by using a Canny operator;
filling and grouping the extracted edge lines, so as to connect the same side edge lines of the same road together and bind two sides of the same road together;
filling the area between two edge lines of the road to obtain a closed road;
wherein the step of grouping the extracted edge lines includes:
step a, taking one of the endpoints of the line segment L2 as the circle center to form concentric circles respectively taking R1 and R2 as the radius, and if the line segment L1 just intersects with the circles taking R1 and R2 as the radius and only has one intersection point, braiding the line segments L1 and L2 into a group and connecting the nearest endpoints; if the direction difference and the lateral distance of the line segments L1 and L2 are smaller than the threshold value, the line segments L1 and L2 are compiled into a group and are connected with the nearest end points, the direction difference is the included angle between the two line segments, the lateral distance is the vertical distance from the midpoint of the L2 to the L1, the minimum radius value of R1 is 1/10 of the L2, the maximum radius value of R2 is 1/3 of the line segment L2, R1 is always shorter than R2, the fact that two circles drawn by the two radiuses intersect with the target line segment is required to be a judging condition, the same side edge of the same road can be found more accurately, if the line segment L1 just intersects with the circle taking the R1 and the R2 as the radius and only one intersection point exists, the line segments L1 and L2 are compiled into a group and are connected with the nearest end points, and if the L1 is not found, the radius of the R2 is expanded until the radius of the R2 reaches the maximum value;
step b, dividing the line segment L1 into four equal parts to obtain three equal parts, wherein the three equal parts comprise two endpoints numbered 1,2,3,4 and 5 in sequence, and starting neighborhood search from 3 pixel points serving as radiuses by taking each endpoint as a circle center until the number of the pixel points is gradually increased to 10 or the search is stopped after the following conditions are met: the counting can be added by one intersection point between the numbers 1 and 5 and the line segment L3, the other numbers 2,3 and 4 are added by the same side as the line segment L3, and when 3 of the five points reach the counting condition and the intersection points are on the same side, the line segment L3 and the line segment L1 can be judged to be two side edges of the same road, and the two side edges are woven into a group;
repeating the operations of the step a and the step b until all the grouping is completed.
2. The method of claim 1, further comprising, after the step of filling and grouping the extracted edge lines, the steps of:
and extending the line segments of the road edge lines after grouping to obtain the final road edge lines.
3. The method of claim 1, wherein the step of determining the suspected road area and the non-suspected road area according to RGB values of each pixel in the image to be identified comprises:
and judging the RGB value of the pixel point, if the RGB value meets (165-185, 170-190, 170-190) or (95-115, 105-115, 110-125), judging that the pixel point belongs to a suspected road area, otherwise, not belonging to the suspected road area, wherein the area formed by all the pixel points belonging to the suspected road area is the suspected road area, and the area formed by all the pixel points belonging to the non-suspected road area is the non-suspected road area.
4. The method of claim 1, wherein the step of padding the extracted edge lines comprises:
marking the pixel of the edge point of the edge extracted by the Canny operator as 0 and the pixel of the non-edge point as 255;
traversing each edge point, tracking 8 neighborhoods of the edge points, finding points with the pixel marks of 0, and then carrying out next tracking on the 8 neighborhoods of the found points until the points with the pixel marks of 0 cannot be found;
after each edge line is found, the front and rear end points of the edge line are taken to find edge points at the 8 neighborhood boundary of the edge line, and if the edge points exist, all the points on the straight line are filled.
5. The method of claim 2, wherein the step of extending the segments of the road edge line after the grouping to obtain a final road edge line comprises:
a) The end point of the edge line is made to be the point A, the distance point which is four times the road width of the point A on the edge line is made to be the point B, a new line segment BA is formed, the length of a half line segment BA is extended according to the position of the line segment BA to obtain a new point C, and if the point C exceeds the image boundary, the line segment BA is extended to the image boundary and then the cycle is exited; if the point C is in the image range, carrying out the next step b);
b) Setting a line segment AC as a line segment L5, taking a point C as a round point, taking the length of a line segment BC as a radius, carrying out neighborhood searching, judging whether the line segment which the intersection point belongs to is collinear with the line segment where the AB is located if the intersection point is found, if the intersection point is collinear with the line segment where the AB is located, designating the position of the point C as a new point A, repeating the operation of the step a) and the step b) until the intersection point is connected with another line segment, and reducing the step length of neighborhood searching every time; if no point is found that intersects it or the points of intersection are not collinear, the extension is abandoned.
6. A road identification device, characterized by comprising:
the judging module is used for judging a suspected road area and a non-suspected road area according to the RGB value of each pixel point in the image to be identified;
the reconstruction module is used for carrying out super-resolution reconstruction on the suspected road area by using the trained SRGAN model to obtain a suspected road area image with higher resolution, and carrying out Gaussian filtering treatment on a non-suspected road area;
the denoising module is used for performing wavelet threshold denoising treatment on the suspected road region image after super-resolution reconstruction;
the edge line extraction module is used for extracting edge lines in the image output after wavelet threshold denoising processing by using a Canny operator;
the grouping module is used for filling and grouping the extracted edge lines, connecting the same side edge lines of the same road together and binding two sides of the same road together;
the filling module is used for filling the area between the two edge lines of the road to obtain a closed road;
wherein the step of grouping the extracted edge lines includes:
step a, taking one of the endpoints of the line segment L2 as the circle center to form concentric circles respectively taking R1 and R2 as the radius, and if the line segment L1 just intersects with the circles taking R1 and R2 as the radius and only has one intersection point, braiding the line segments L1 and L2 into a group and connecting the nearest endpoints; if the direction difference and the lateral distance of the line segments L1 and L2 are smaller than the threshold value, the line segments L1 and L2 are compiled into a group and are connected with the nearest end points, the direction difference is the included angle between the two line segments, the lateral distance is the vertical distance from the midpoint of the L2 to the L1, the minimum radius value of R1 is 1/10 of the L2, the maximum radius value of R2 is 1/3 of the line segment L2, R1 is always shorter than R2, the fact that two circles drawn by the two radiuses intersect with the target line segment is required to be a judging condition, the same side edge of the same road can be found more accurately, if the line segment L1 just intersects with the circle taking the R1 and the R2 as the radius and only one intersection point exists, the line segments L1 and L2 are compiled into a group and are connected with the nearest end points, and if the L1 is not found, the radius of the R2 is expanded until the radius of the R2 reaches the maximum value;
step b, dividing the line segment L1 into four equal parts to obtain three equal parts, wherein the three equal parts comprise two endpoints numbered 1,2,3,4 and 5 in sequence, and starting neighborhood search from 3 pixel points serving as radiuses by taking each endpoint as a circle center until the number of the pixel points is gradually increased to 10 or the search is stopped after the following conditions are met: the counting can be added by one intersection point between the numbers 1 and 5 and the line segment L3, the other numbers 2,3 and 4 are added by the same side as the line segment L3, and when 3 of the five points reach the counting condition and the intersection points are on the same side, the line segment L3 and the line segment L1 can be judged to be two side edges of the same road, and the two side edges are woven into a group;
repeating the operations of the step a and the step b until all the grouping is completed.
7. The apparatus of claim 6, further comprising an extension module for extending segments of the road edge line after the grouping to obtain a final road edge line.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of claims 1-5 when the program is executed.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910614644.2A CN110335322B (en) | 2019-07-09 | 2019-07-09 | Road recognition method and road recognition device based on image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910614644.2A CN110335322B (en) | 2019-07-09 | 2019-07-09 | Road recognition method and road recognition device based on image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110335322A CN110335322A (en) | 2019-10-15 |
| CN110335322B true CN110335322B (en) | 2024-03-01 |
Family
ID=68144473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910614644.2A Active CN110335322B (en) | 2019-07-09 | 2019-07-09 | Road recognition method and road recognition device based on image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110335322B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11100344B2 (en) * | 2019-11-21 | 2021-08-24 | GM Global Technology Operations LLC | Image-based three-dimensional lane detection |
| CN114937212B (en) * | 2022-07-26 | 2022-11-11 | 南通华锐软件技术有限公司 | Aerial photography road type identification method based on frequency domain space conversion |
| CN115393813B (en) * | 2022-08-18 | 2023-05-02 | 中国人民公安大学 | Road identification method, device, equipment and storage medium based on remote sensing image |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0944629A (en) * | 1995-07-25 | 1997-02-14 | Nippon Telegr & Teleph Corp <Ntt> | Method and device for generating wide area map |
| CN101334263A (en) * | 2008-07-22 | 2008-12-31 | 东南大学 | Circular target circular center positioning method |
| CN101763512A (en) * | 2009-12-11 | 2010-06-30 | 西安电子科技大学 | Method for semi-automatically detecting road target in high-resolution remote sensing images |
| CN101901343A (en) * | 2010-07-20 | 2010-12-01 | 同济大学 | Remote sensing image road extracting method based on stereo constraint |
| JP2012176641A (en) * | 2011-02-25 | 2012-09-13 | Suzuki Motor Corp | Detection apparatus for parking frame |
| CN106023180A (en) * | 2016-05-17 | 2016-10-12 | 李迎春 | Unstructured road RGB entropy segmentation method |
| CN106874875A (en) * | 2017-02-17 | 2017-06-20 | 武汉理工大学 | A kind of vehicle-mounted lane detection system and method |
| CN107909010A (en) * | 2017-10-27 | 2018-04-13 | 北京中科慧眼科技有限公司 | A kind of road barricade object detecting method and device |
| CN109886200A (en) * | 2019-02-22 | 2019-06-14 | 南京邮电大学 | A kind of unmanned lane line detection method based on production confrontation network |
-
2019
- 2019-07-09 CN CN201910614644.2A patent/CN110335322B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0944629A (en) * | 1995-07-25 | 1997-02-14 | Nippon Telegr & Teleph Corp <Ntt> | Method and device for generating wide area map |
| CN101334263A (en) * | 2008-07-22 | 2008-12-31 | 东南大学 | Circular target circular center positioning method |
| CN101763512A (en) * | 2009-12-11 | 2010-06-30 | 西安电子科技大学 | Method for semi-automatically detecting road target in high-resolution remote sensing images |
| CN101901343A (en) * | 2010-07-20 | 2010-12-01 | 同济大学 | Remote sensing image road extracting method based on stereo constraint |
| JP2012176641A (en) * | 2011-02-25 | 2012-09-13 | Suzuki Motor Corp | Detection apparatus for parking frame |
| CN106023180A (en) * | 2016-05-17 | 2016-10-12 | 李迎春 | Unstructured road RGB entropy segmentation method |
| CN106874875A (en) * | 2017-02-17 | 2017-06-20 | 武汉理工大学 | A kind of vehicle-mounted lane detection system and method |
| CN107909010A (en) * | 2017-10-27 | 2018-04-13 | 北京中科慧眼科技有限公司 | A kind of road barricade object detecting method and device |
| CN109886200A (en) * | 2019-02-22 | 2019-06-14 | 南京邮电大学 | A kind of unmanned lane line detection method based on production confrontation network |
Non-Patent Citations (3)
| Title |
|---|
| SM Easa等.Use of Statellite Imagery for Establishing Road Horizontal Alignments.《Bing》.2007,全文. * |
| Ying-Yue Li 等 .Edge Enhanced Super-Resolution.《IEEE》.2018,全文. * |
| 尹训怡.目标地物提取在优化地震测线部署中的应用.《中国优秀硕士学位论文全文数据库（电子期刊）信息科技辑》.2012,第2012卷(第04期),第13-22页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110335322A (en) | 2019-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110335322B (en) | Road recognition method and road recognition device based on image | |
| CN109685060B (en) | Image processing method and device | |
| CN108416789A (en) | Method for detecting image edge and system | |
| US11049275B2 (en) | Method of predicting depth values of lines, method of outputting three-dimensional (3D) lines, and apparatus thereof | |
| US7848571B2 (en) | Computer-implemented method for efficient image segmentation using automated saddle-point detection | |
| CN114529459B (en) | Method, system and medium for enhancing image edge | |
| CN109300104B (en) | Angular point detection method and device | |
| CN109858438B (en) | Lane line detection method based on model fitting | |
| CN108875504B (en) | Image detection method and image detection device based on neural network | |
| CN105654428A (en) | Method and system for image noise reduction | |
| CN116664559A (en) | Machine vision-based memory bank damage rapid detection method | |
| Raza et al. | Deconvolving convolutional neural network for cell detection | |
| CN114757950B (en) | Ultrasonic image processing method, device and computer readable storage medium | |
| CN109544577B (en) | Improved straight line extraction method based on edge point grouping | |
| CN111523341B (en) | Binarization method and device for two-dimensional code image | |
| CN108256445A (en) | Method for detecting lane lines and system | |
| CN107742302B (en) | Contour detection method based on primary visual cortex multi-scale contour fusion | |
| Khordehchi et al. | Automatic lung nodule detection based on statistical region merging and support vector machines | |
| CN110956078B (en) | Power line detection method and device | |
| CN104268550A (en) | Feature extraction method and device | |
| CN114119437A (en) | GMS-based image stitching method for improving moving object distortion | |
| CN114399480A (en) | Method and device for detecting severity of vegetable leaf disease | |
| CN116363168A (en) | Remote sensing video target tracking method and system based on super-resolution network | |
| CN110008881A (en) | The recognition methods of the milk cow behavior of multiple mobile object and device | |
| CN111753723B (en) | Fingerprint identification method and device based on density calibration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |