CN112818775B - Forest road rapid identification method and system based on regional boundary pixel exchange - Google Patents
Forest road rapid identification method and system based on regional boundary pixel exchange Download PDFInfo
- Publication number
- CN112818775B CN112818775B CN202110075332.6A CN202110075332A CN112818775B CN 112818775 B CN112818775 B CN 112818775B CN 202110075332 A CN202110075332 A CN 202110075332A CN 112818775 B CN112818775 B CN 112818775B
- Authority
- CN
- China
- Prior art keywords
- region
- boundary
- sub
- image
- road
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a forest road real-time identification method and system based on regional boundary pixel exchange. Firstly, video images are collected, extracted and subjected to preliminary segmentation based on space constraint. And secondly, carrying out similarity judgment on the boundary pixels of the region by constructing an energy function, and determining the exchange of the boundary pixels and the brother connected domain according to the energy maximization condition. And (3) performing attribution judgment and exchange of all sub-region boundary pixels through iteration, realizing rapid region updating and rapid image segmentation, and generating a new region boundary. Then, introducing a sub-region feature operator through a support vector machine model, and quickly identifying and classifying each region. And finally, extracting the boundary coordinates of the road area, generating a smooth road boundary through spline curve fitting, and completing the real-time identification of the unstructured road area in the forest region. The method solves the problem of misidentification of the existing algorithm in the forest environment, and highlights the accuracy and instantaneity of identifying the unstructured roads in the forest.
Description
Technical Field
The invention belongs to the technical field of autonomous navigation and unstructured road identification of special vehicles in forest areas, and particularly relates to a forest area road rapid identification method and system based on regional boundary pixel exchange.
Background
Because the forest roads lack effective references and manual identification, the method has the characteristics of strong nonlinearity and uncertainty, and provides serious challenges for forest operation vehicles and vehicle drivers. Therefore, urgent demands are put forward for autonomy and intellectualization in the operation of the vehicles in the forest region. Perception of the forest environment and recognition of the road through visual information are bases for realizing autonomous running of the forest vehicle. The existing road recognition algorithm carried by the autonomous driving platform is mainly used for recognizing the structured road in the urban environment. Although image extraction and semantic recognition of surrounding lane lines, sign marks, signal lights, pedestrians, vehicles, etc. have been basically accomplished, the urban roads differ greatly from the unstructured roads in the forest region. The forest road lacks obvious lane marks and general traffic signal indication marks, the existing road recognition algorithm cannot reasonably divide and understand semantics of the forest road lacking reference objects and manual marks, and the direct transplanting of the forest road into the forest road can cause unpredictable risks. The environment perception based on the visual image is a key link for realizing the identification of the unstructured road of the forest region by the autonomous operation vehicle of the forest region. Therefore, on one hand, based on the unstructured road recognition of the forest region under the condition of visual information, the semantic understanding of the autonomous navigation vehicle of the forest region to the road and the passable region is facilitated, and on the other hand, the rapid processing and real-time calculation of visual images in the vehicle driving process have important practical significance for realizing the autonomous navigation of the vehicle of the forest region.
Patent application number is CN201110341479.1, and the invention is Chinese patent of 'unstructured road detection method based on self-adaptive edge registration'. In unstructured road identification [0003], the invention mainly aims at research objects such as non-urban arterial roads and other road types (such as campuses, residential communities, rural roads and the like) without obvious lane marks. This type of road has uncertain characteristics such as road surface breakage and cracks, although the road surface has irregular shapes relative to the urban structured road. However, such road areas still have significant features in the image that are relatively deterministic in shape and relatively consistent in texture over the forest roads. The forest roads have different forms, and have uncertain interferences such as various weeds, humus, stone rubble, crown coverage, shadows and the like, so that the texture characteristics are very uneven, and the light influence is serious. In this case it is difficult to find color and texture features similar to non-urban arterial roads, resulting in a dilemma in the routing behaviour. In addition, the Canny edge detection algorithm in the algorithm [0040] is realized based on the fact that the rural road boundaries are irregular, but quite obvious boundary distinguishing characteristics still exist, and ideal recognition effects are difficult to achieve on the road surfaces with forest weeds, rubble coverage and fuzzy boundaries.
The patent application number is CN201610812183.6, and the invention is Chinese patent of an unstructured road identification method. The invention adopts Laplaction second order differential operator to carry out sharpening operation in [0018], and only adopts two index components of tone and saturation value of gray level image to carry out OTSU segmentation in [0025 ]. Although the influence of illumination conditions on image segmentation is objectively eliminated, the algorithm is not limited by a color threshold value in the implementation process. And under the condition of complex interference of the forest environment, the regional segmentation after sharpening the image and the operation adopted by [0059] can interfere the judgment of the road region and the stability of the algorithm and cause hidden danger. In order to improve the recognition speed, the algorithm [0049] adopts a strategy of compressing a picture with the size of 1280 pixels by 960 pixels into an image with the size of 320 pixels by 240 pixels, so that the recognition accuracy and recognition effect are greatly influenced on the fuzzy irregular road boundary of the forest region, the image quality is improved, the calculation speed is reduced, the real-time requirement cannot be met, and the algorithm needs to sacrifice and balance between the two.
The patent application number is CN201911145132.2, and the invention is Chinese patent of an unstructured road identification method. According to the invention, the ant colony algorithm is adopted to optimize the parameters of the BP neural network in [0009], and although the understanding effect of the neural network on the image semantics can be objectively enhanced, the ant colony algorithm has higher space complexity, and the optimization process can possibly cause local optimal solution. In addition, in the process of extracting the characteristic values by image blocks [0060] mainly extracts several indexes of gray scale mean value, variance, consistency and smoothness. The basic reason that the algorithm can meet the requirements is that the rural roads are single and obvious in characteristic information relatively, the characteristic is outstanding compared with the surrounding environment, the recognition degree is high, and the recognition effect is poor when the rural roads are directly transplanted to the forest road recognition.
The patent application number is CN201811614332.3, and the invention is Chinese patent of a road recognition model training method, a road recognition method and a road recognition device. The invention adopts a convolution neural network structure to extract the relevant characteristics between the continuous frame images in the multi-frame image processing process [0009], is based on traffic identification and category labels under a large number of structured roads, and takes the traffic identification and category labels as input to obtain a recognition result, but the model mainly aims at target recognition under the condition of the urban structured roads. Relatively, the road environment of the forest region is complex, a reference object and an artificial mark are absent, and the algorithm cannot accurately identify the road condition without obvious artificial mark in the forest region.
In the four methods, the first three unstructured road identifications are based on non-urban arterial roads and rural roads, but the type of unstructured roads still have very obvious structure, color and texture characteristics on images, so that the adopted image characteristics can basically realize the road identification only based on gray images. The fourth method is based on neural network identification of urban structured roads. However, the unstructured roads in the forest region can not reach similar standards, because a large number of uncertain interference factors such as dead objects, weeds, earth stones and the like are widely distributed on the unstructured roads in the forest region, and the identification of the roads and the resolution of the passable areas are greatly affected. Although the above methods propose different algorithms for unstructured road and structured road recognition, real-time detection and image semantic recognition of complex unstructured roads in forest areas cannot be substantially solved.
Disclosure of Invention
Aiming at the technical defects in the prior art, the embodiment of the invention aims to provide a forest road rapid identification method and system based on regional boundary pixel exchange.
In a first aspect, an embodiment of the present invention provides a method for quickly identifying a forest road based on regional boundary pixel exchange, including:
s1, video image acquisition: collecting continuous frame images in real time through image collecting equipment, and extracting the frame images to obtain an image to be processed containing a forest region unstructured road region;
s2, image initial segmentation: dividing the image to be processed into N rectangular subareas with the same size according to the pixel size, numbering each rectangular subarea, and defining adjacent rectangular subareas as brother connected domains;
s3, reconstructing a sub-region based on boundary pixel switching: constructing an energy function based on image characteristics, performing energy maximization detection on each sub-region by adopting the energy function, judging whether region boundary pixels are exchanged with brother connected domains based on an energy maximization principle to optimize segmentation of the sub-regions, traversing all sub-region boundary points in an iterative mode, and rapidly completing reconstruction of the sub-regions and realizing final segmentation of the image to be processed by region boundary pixel exchange;
s4, extracting the characteristics of the subareas: mapping the image of the subarea subjected to the energy maximization detection and final segmentation in the step S3 to different color spaces by adopting an HSV mean value and a texture mean value, extracting the characteristics of each subarea, and generating a corresponding characteristic operator;
s5, road boundary fitting identification: and (3) rapidly resolving a feature operator of the subarea based on boundary pixel exchange through a support vector machine model, judging whether the subarea is classified into two types of subareas, performing binarization processing on the images on the basis of classification, primarily merging the passable subareas into the road area according to binarization results, extracting left and right boundary coordinates of the area, fitting left and right boundary coordinate scattered points through spline curves to obtain a smooth road boundary, and completing the identification of unstructured roads in the forest region.
As a specific embodiment of the present application, in step S3, an energy function based on image features is constructed, specifically:
constructing a color distribution item and a boundary pixel item of the subarea;
and carrying out weighted summation on the color distribution item and the boundary pixel item to construct an energy function based on image characteristics.
As a specific embodiment of the present application, in step S3, the energy function is used to perform energy maximization detection on the region boundary pixels of the image to be processed, specifically:
firstly, counting color histograms in specific color spaces of all subareas, and then constructing an energy function based on probability density distribution of different color channels of the color histograms, wherein the more concentrated the color distribution in the subareas is, the larger the energy function is; extracting region boundary pixels of the sub-region; and through edge pixel exchange and iteration, sub-region energy maximization detection and updating are realized.
As a specific embodiment of the present application, step S4 specifically includes:
carrying out image HSV average processing on a single subarea, and extracting three characteristic values of hue (H), saturation (S) and brightness (V) of the subarea;
calculating and extracting texture features of the subareas, wherein the average value of the extracted texture features comprises four indexes including an LBP texture index, a gray level co-occurrence matrix, a gray level-gradient co-occurrence matrix and a Gabor small ripple;
the seven characteristic values form a regional characteristic operator tau, and the segmented sub-regional characteristics are described by the characteristic operator tau;
and carrying out rapid feature extraction on each sub-region in sequence, and generating a corresponding feature operator.
In a second aspect, an embodiment of the present invention provides a rapid forest road identification system based on regional boundary pixel exchange, including:
the video image acquisition module is used for acquiring continuous frame images in real time through the image acquisition equipment, extracting the frame images and obtaining an image to be processed containing a forest region unstructured road region;
the image initial segmentation module is used for dividing the image to be processed into N rectangular subareas with the same size according to the pixel size, numbering each rectangular subarea, and defining the adjacent rectangular subareas as brother connected domains;
the sub-region reconstruction module is used for constructing an energy function based on image characteristics, carrying out energy maximization detection on each sub-region by adopting the energy function, judging whether the region boundary pixels are exchanged with brother connected regions based on an energy maximization principle so as to optimize the segmentation of the sub-region, traversing all region boundary points in an iterative mode, and rapidly completing the reconstruction of the sub-region and realizing the final segmentation of the image to be processed through region boundary pixel exchange;
the subarea feature extraction module is used for mapping the image into different color spaces by adopting an HSV mean value and a texture mean value to perform feature extraction on each subarea and generate a corresponding feature operator;
the road boundary fitting recognition module is used for rapidly resolving a characteristic operator of a sub-region based on boundary pixel exchange through a support vector machine model, judging whether the sub-region is in a two-class mode or not, carrying out two-value processing on the image on the basis of classification, preliminarily merging the passable region into a road region according to a two-value result, extracting left and right boundary coordinates of the region, carrying out spline curve fitting to obtain a smooth road boundary, and completing the recognition of unstructured roads in a forest region.
Wherein, in a preferred embodiment of the present application, the image acquisition device is a monocular vision camera.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
according to the forest road rapid identification method based on regional boundary pixel exchange, energy function construction and energy function maximization are used as cores, and rapid detection and identification of unstructured roads lacking reference objects and artificial marks in a forest are completed. The image area division based on area boundary pixel exchange does not need to judge all pixels of a frame image, but carries out energy maximization judgment on the attribution of the edge pixels on the basis of area color distribution density, has small operation amount, high calculation efficiency and high recognition speed, can meet the real-time forest road detection under the condition of medium-low speed movement in the safe driving process of a forest special operation vehicle, overcomes the defect that only urban structured roads, rural roads and residential roads can be recognized in the existing non-structural road recognition and detection technology, greatly improves the road detection capability of the forest operation vehicle in complex and changeable forest environments, and provides technical support for improving the autonomous navigation capability of the forest operation vehicle.
In addition, compared with the prior art, the forest road rapid identification method has the advantages of being rapid, high in efficiency and the like.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a main flowchart of a forest road rapid identification method based on regional boundary pixel exchange according to an embodiment of the present invention.
Fig. 2 is an image segmentation effect diagram generated after the region boundary-based fast swap.
Fig. 3 shows a region two classification based on the region-feature operator τ.
Fig. 4 is a graph of the effects of extraction of unstructured road boundaries and left and right boundary segmentation in a forest region.
FIG. 5 is a process of fitting a smooth road boundary during the calculation of the present invention.
Fig. 6 is a graph of a smooth road boundary fit effect.
Fig. 7 is a block diagram of a rapid forest road recognition system based on regional boundary pixel exchange according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the method for quickly identifying a forest road based on regional boundary pixel exchange according to the embodiment of the present invention mainly includes the following steps:
s1, video image acquisition: and acquiring continuous frame images in real time through image acquisition equipment, and extracting the frame images to obtain an image to be processed containing the unstructured road area of the forest region.
Specifically, a monocular vision camera is adopted to collect real-time continuous frame images, then the frame images are extracted to obtain images containing unstructured road areas in forest areas, the collected images are read into a system, and the images are converted into OpenCV Mat multichannel image types.
For example, after parameter presetting and system initialization are performed on the system, five frames of images are read, so that a buffer queue with initial basic length of 5 is formed.
S2, image initial segmentation: dividing the image to be processed into N rectangular subareas with the same size according to the pixel size, numbering each rectangular subarea, and defining adjacent rectangular subareas as brother connected domains.
Specifically, the read current frame image is preprocessed, the image is subjected to region segmentation according to space constraint and segmentation quantity setting, the image is divided into N rectangular subareas with the same size, each subarea is numbered, the initial segmentation of the frame image is only related to the image size and the space constraint, the adjacent subareas are defined as 'brother connected domains', the image size acquired by a camera in the embodiment is 640 pixels by 480 pixels, and the image is divided into 25 (N=5×5) initial areas with the subarea initial aspect ratio of 4:3 in order to ensure the calculation speed and the region segmentation effect at the same time.
S3, reconstructing a sub-region based on boundary pixel switching: constructing an energy function based on image characteristics, performing energy maximization detection on each sub-region by adopting the energy function, judging whether region boundary pixels are exchanged with brother connected domains based on an energy maximization principle to optimize segmentation of the sub-regions, traversing all region boundary points in an iterative mode, and rapidly completing reconstruction of the sub-regions and realizing final segmentation of the image to be processed through region boundary pixel exchange.
Specifically, S3 mainly includes:
in step 301, an energy function of the image subregion is constructed, and an optimal segmentation strategy of the region is realized through the maximization of the energy function. The energy function is composed of two indexes of a color distribution item and a boundary pixel item. Wherein the energy function formula is:
E(s)＝α*D(s)+β*B(s) (1)
in the formula (1), E(s) is a sub-region energy index and is also a judging basis for maximizing the energy of the region, and when the term reaches the maximum value, all boundary pixels of the region complete attribution distribution. D(s) is a color distribution term, B(s) is a boundary pixel term index, alpha and beta are proportionality coefficients, weights are allocated to the two indexes, and the attribution of the boundary pixels is judged through the maximization of an energy function.
Step 302, constructing a sub-region color distribution item. And determining the color clusters of the principal components of the region by voting the color density distribution of the region, and then calculating the quality index of the color distribution of the region, wherein the color quality of the region is contributed by each pixel in the region. The sub-region color distribution term is mainly used for measuring the color quality of the region. The color distribution term formula is:
in the formula (2), i is a regional pixel point or a super pixel block in the subarea,for the color distribution function +.>A color distribution histogram for a set of pixels, wherein +.>The expression of (2) is:
in the formula (3), j is a single color histogram in the color distribution histogram, S is a set of all pixel points in the subarea A, delta is a proportion function for recording the number of the pixel points falling into the distribution area in the color histogram set, I (I) is a pixel color statistical function, if the color is I, 1 is calculated, otherwise 0 is calculated, and A is a subarea pixel point set.
In the formula (4), the amino acid sequence of the compound,as a color distribution function, the configuration thereof is to measure the color distribution according to the degree of dispersion of the pixel distribution.
In step 303, a boundary pixel term index is constructed, and the main purpose of the boundary pixel term index is to control the smoothness of the region boundary. The main behavior is to limit the boundary pixel exchange condition by adjusting the boundary punishment mechanism of the parameter limiting region so as to achieve the purposes of controlling and evaluating the boundary smoothness, wherein the boundary pixel term formula is as follows:
B(s)＝∑ i ∑ k (b i (k)) 2 (5)
in the formula (5), B(s) is a boundary pixel term index, B i (k) And (3) representing a color distribution histogram of the boundary patch, wherein k is a straight square in the color distribution histogram of the patch, and constructing a boundary penalty function through the color distribution histogram of the pixel points in the patch. In the formula (6), S is a set of all pixels in the sub-region a, δ is a scaling function for recording the number of pixels falling into the region where the color histogram is concentrated, i is a pixel in the patch, and a is a set of pixels in the sub-region.
And (3) finishing attribute judgment of the boundary pixels of the subareas through energy function construction, traversing the boundary of the area to realize definition and quick division of the boundary of the area, and finishing the division of the area when the boundary pixels of the brother connected domain are not exchanged any more. The segmentation method and the segmentation effect can be understood in connection with fig. 2.
It should be noted that, according to the foregoing description, the step S3 mainly describes an area edge pixel switching algorithm, which is calculated by: first, the frame image is initially divided. In order to achieve reasonable expression of the computing efficiency and the sub-region semantics at the same time, the frame image is divided into N rectangular sub-initial regions with the same shape, then, sub-region semantic evaluation index functions are constructed through energy functions, region boundary pixels are extracted, sub-region energy maximization is achieved through the influence of the boundary pixels on the energy functions, meanwhile, judgment and exchange are conducted on the attribution of the boundary pixels, finally, all region boundaries are traversed through iteration, and optimization and division of all sub-regions are completed.
Further, the construction process of the energy function mainly comprises the following steps: and constructing two indexes of a color distribution item and a boundary pixel item, and carrying out weighted summation on the two indexes. The color distribution item is based on the sub-region color density description of the statistical histogram, and the color density is summed to obtain the color quality of the region; the boundary pixel item is calculated based on the color quality of the pixel color channel; the nature of the weighting coefficient is a boundary pixel attribution penalty coefficient, and objectively represents an evaluation index of sub-region boundary smoothness.
S4, extracting the characteristics of the subareas: and (3) mapping the sub-regions subjected to energy maximization detection and final segmentation in the step (S3) to different color spaces by adopting an HSV mean value and a texture mean value, extracting the characteristics of each sub-region, and generating corresponding characteristic operators.
Specifically, S4 mainly includes:
in step 401, the feature extraction of the subarea is to perform image HSV average processing on a single subarea, and extract three bases of hue (H), saturation (S) and brightness (V) of the subarea to perform rapid feature value extraction.
Step 402, mapping the sub-region texture mean to different color spaces, and calculating and extracting texture features of the region, wherein the extracted texture mean comprises four indexes of LBP texture indexes, gray level co-occurrence matrix, gray level-gradient co-occurrence matrix and Gabor wavelet.
In step S403, the above seven feature values together form a region feature operator τ, and the segmented sub-region features may be described by the feature operator τ.
Step S404, carrying out rapid feature extraction on each sub-region in sequence through iteration, and generating a corresponding feature operator.
S5, road boundary fitting identification: and (3) rapidly resolving a characteristic operator of the sub-region based on boundary pixel exchange through a support vector machine model, judging whether the sub-region of the frame image is in two categories, performing binarization processing on the image on the basis of the categories, primarily merging the passable region into a road region according to binarization results, extracting left and right boundary coordinates of the region, fitting left and right boundary coordinate scattered points through spline curves to obtain a smooth road boundary, and completing the identification of unstructured roads in a forest region.
Specifically, S5 mainly includes:
step 501, inputting the feature operator describing the sub-region into a trained support vector machine model, rapidly resolving the sub-region feature operator based on boundary pixel exchange through the support vector machine, judging whether the sub-region of the frame image is classified into a road region, and then primarily merging the passable region into the road region. The feature operator based support vector machine classification results can be understood in conjunction with the binary image shown in fig. 3.
Step 502, on the basis of the road area division, obtaining boundary pixels of the road area boundary, and extracting coordinates corresponding to the boundary pixels, wherein the extracted road boundary is a scattered point set. The initial road boundary can be understood by a line graph formed by the road left and right boundary scattered points shown in fig. 4.
In step 503, spline curve fitting is performed on the coordinates of the scattered points corresponding to the boundary pixels, so as to obtain a smooth road boundary, and fig. 5 is a spline curve fitting process. The road boundary spline curve obtained by fitting is presented in fig. 6 in a mode of equally dispersing points on the abscissa, the black dispersion points in the middle are the positions of the central lines of the roads, and the unstructured road identification in the forest region is completed.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
according to the forest road rapid identification method based on regional boundary pixel exchange, energy function construction and energy function maximization are used as cores, and rapid detection and identification of unstructured roads lacking reference objects and artificial marks in a forest are completed. The image area division based on area boundary pixel exchange does not need to judge all pixels of a frame image, but carries out energy maximization judgment on the attribution of the edge pixels on the basis of area color distribution density, has small operation amount, high calculation efficiency and high recognition speed, can meet the real-time forest road detection under the condition of medium-low speed movement in the safe driving process of a forest special operation vehicle, overcomes the defect that only urban structured roads, rural roads and residential roads can be recognized in the existing non-structural road recognition and detection technology, greatly improves the road detection capability of the forest operation vehicle in complex and changeable forest environments, and provides technical support for improving the autonomous navigation capability of the forest operation vehicle.
Based on the same inventive concept, the embodiment of the invention also provides a forest road rapid identification system based on regional boundary pixel exchange, as shown in fig. 7, comprising:
the video image acquisition module 11 is used for acquiring continuous frame images in real time through the image acquisition equipment, extracting the frame images and obtaining an image to be processed containing a forest region unstructured road region;
the image initial segmentation module 12 is configured to divide the image to be processed into N rectangular subregions with the same size according to the pixel size, number each rectangular subregion, and define adjacent rectangular subregions as sibling connected domains;
the sub-region reconstruction module 13 is configured to construct an energy function based on image features, perform energy calculation and evaluation on the sub-region of the image to be processed by adopting the energy function, judge whether the region boundary pixels are exchanged with the sibling connected region based on an energy maximization principle to update and optimize the sub-region, traverse all region boundary points to realize region boundary pixel exchange, and quickly complete sub-region reconstruction and realize final segmentation of the image to be processed by iterative optimization of region segmentation;
the sub-region feature extraction module 14 is configured to map the image to different color spaces by using an HSV mean value and a texture mean value, perform feature extraction on each sub-region, and generate a corresponding feature operator;
the road boundary fitting recognition module 15 is configured to quickly calculate a feature operator of the sub-region based on boundary pixel exchange through a support vector machine model, determine whether the sub-region of the frame image is a classification of the road region, perform binarization processing on the image based on the classification, primarily combine the passable region into the road region according to the binarization result, extract left and right boundary coordinates of the region, and fit left and right boundary coordinate scattering points through spline curves to obtain a smooth road boundary, thereby completing the unstructured road recognition of the forest region.
Wherein, the sub-region reconstruction module 13 is specifically configured to:
constructing a color distribution item and a boundary pixel item of the subarea;
and carrying out weighted summation on the color distribution item and the boundary pixel item to construct an energy function based on image characteristics.
The road boundary fitting recognition module is specifically configured to:
carrying out image HSV average processing on a single subarea, and extracting three characteristic values of hue (H), saturation (S) and brightness (V) of the subarea;
calculating and extracting texture features of the subareas, wherein the average value of the extracted texture features comprises four indexes including an LBP texture index, a gray level co-occurrence matrix, a gray level-gradient co-occurrence matrix and a Gabor small ripple;
the seven characteristic values form a regional characteristic operator tau, and the segmented sub-regional characteristics are described by the characteristic operator tau;
and carrying out rapid feature extraction on each sub-region in sequence, and generating a corresponding feature operator.
It should be noted that, for a more specific description of the system embodiment, please refer to the foregoing method embodiment, and a detailed description is omitted herein.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (9)
1. A forest road rapid identification method based on regional boundary pixel exchange is characterized by comprising the following steps:
s1, video image acquisition: collecting continuous frame images in real time through image collecting equipment, and extracting the frame images to obtain an image to be processed containing a forest region unstructured road region;
s2, image initial segmentation: dividing the image to be processed into N rectangular subareas with the same size according to the pixel size, numbering each rectangular subarea, and defining adjacent rectangular subareas as brother connected domains;
s3, reconstructing a sub-region based on boundary pixel switching: constructing an energy function based on image characteristics, performing energy maximization detection on each sub-region by adopting the energy function, judging whether region boundary pixels are exchanged with brother connected domains based on an energy maximization principle to optimize segmentation of the sub-regions, traversing all sub-region boundary points in an iterative mode, and rapidly completing reconstruction of the sub-regions and realizing final segmentation of the image to be processed by region boundary pixel exchange;
s4, extracting the characteristics of the subareas: mapping the sub-regions subjected to energy maximization detection and final segmentation in the step S3 to different color spaces by adopting HSV mean values and texture mean values, extracting features of each sub-region, and generating corresponding feature operators;
s5, road boundary fitting identification: and (3) rapidly resolving a feature operator of the subarea based on boundary pixel exchange through a support vector machine model, judging whether the subarea is classified into two types of subareas, performing binarization processing on the images on the basis of classification, primarily merging the passable subareas into the road area according to binarization results, extracting left and right boundary coordinates of the area, fitting left and right boundary coordinate scattered points through spline curves to obtain a smooth road boundary, and completing the identification of unstructured roads in the forest region.
2. The method according to claim 1, wherein in step S3, an energy function based on image features is constructed, in particular:
constructing a color distribution item and a boundary pixel item of the subarea;
and carrying out weighted summation on the color distribution item and the boundary pixel item to construct an energy function based on image characteristics.
3. The method according to claim 2, wherein the constructing of the color distribution item is specifically:
determining the color clusters of the main components of the subareas through the color density distribution of the subareas in a voting mode, and calculating the quality index of the color distribution of the subareas.
4. The method according to claim 1, wherein in step S3, the energy function is used to perform energy maximization detection on region boundary pixels of the image to be processed, specifically:
firstly, counting color histograms in specific color spaces of all subareas, and then constructing an energy function based on probability density distribution of different color channels of the color histograms, wherein the more concentrated the color distribution in the subareas is, the larger the energy function is; extracting region boundary pixels of the sub-region; and through edge pixel exchange and iteration, sub-region energy maximization detection and updating are realized.
5. The method according to claim 1, wherein step S4 is specifically:
carrying out image HSV average processing on a single subarea, and extracting three characteristic values of hue, saturation and brightness of the subarea;
calculating and extracting texture features of the subareas, wherein the average value of the extracted texture features comprises four indexes including an LBP texture index, a gray level co-occurrence matrix, a gray level-gradient co-occurrence matrix and a Gabor small ripple;
the seven characteristic values form a regional characteristic operator tau, and the segmented sub-regional characteristics are described by the characteristic operator tau;
and carrying out rapid feature extraction on each sub-region in sequence, and generating a corresponding feature operator.
6. A forest road rapid identification system based on regional boundary pixel exchange, comprising:
the video image acquisition module is used for acquiring continuous frame images in real time through the image acquisition equipment, extracting the frame images and obtaining an image to be processed containing a forest region unstructured road region;
the image initial segmentation module is used for dividing the image to be processed into N rectangular subareas with the same size according to the pixel size, numbering each rectangular subarea, and defining the adjacent rectangular subareas as brother connected domains;
the sub-region reconstruction module is used for constructing an energy function based on image characteristics, carrying out energy maximization detection on each sub-region by adopting the energy function, judging that region boundary pixels are exchanged with brother connected regions based on an energy maximization principle, traversing all region boundary points, optimizing the segmentation of the sub-regions in an iterative mode, and rapidly completing the reconstruction of the sub-regions and realizing the final segmentation of the image to be processed;
the subarea feature extraction module is used for mapping the image to different color spaces by adopting an HSV mean value and a texture mean value, extracting features of each subarea and generating corresponding feature operators;
the road boundary fitting recognition module is used for rapidly resolving a sub-region feature operator based on boundary pixel exchange through a support vector machine model, judging whether the sub-region is in two categories, performing binarization processing on the image on the basis of the categories, primarily merging the passable region into the road region according to the binarization result, extracting left and right boundary coordinates of the region, fitting left and right boundary coordinate scattered points through spline curves to obtain a smooth road boundary, and completing the unstructured road recognition of the forest region.
7. The system of claim 6, wherein the sub-region reconstruction module is specifically configured to:
constructing a color distribution item and a boundary pixel item of the subarea;
and carrying out weighted summation on the color distribution item and the boundary pixel item to construct an energy function based on image characteristics.
8. The system of claim 6, wherein the road boundary fit identification module is specifically configured to:
carrying out image HSV average processing on a single subarea, and extracting three characteristic values of hue (H), saturation (S) and brightness (V) of the subarea;
calculating and extracting texture features of the subareas, wherein the average value of the extracted texture features comprises four indexes including an LBP texture index, a gray level co-occurrence matrix, a gray level-gradient co-occurrence matrix and a Gabor small ripple;
the seven characteristic values form a regional characteristic operator tau, and the segmented sub-regional characteristics are described by the characteristic operator tau;
and carrying out rapid feature extraction on each sub-region in sequence, and generating a corresponding feature operator.
9. The system of claim 6, wherein the image acquisition device is a monocular vision camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110075332.6A CN112818775B (en) | 2021-01-20 | 2021-01-20 | Forest road rapid identification method and system based on regional boundary pixel exchange |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110075332.6A CN112818775B (en) | 2021-01-20 | 2021-01-20 | Forest road rapid identification method and system based on regional boundary pixel exchange |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112818775A CN112818775A (en) | 2021-05-18 |
| CN112818775B true CN112818775B (en) | 2023-07-25 |
Family
ID=75858416
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110075332.6A Active CN112818775B (en) | 2021-01-20 | 2021-01-20 | Forest road rapid identification method and system based on regional boundary pixel exchange |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112818775B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113706561B (en) * | 2021-10-29 | 2022-03-29 | 华南理工大学 | Image semantic segmentation method based on region separation |
| CN114332370B (en) * | 2021-12-28 | 2023-02-21 | 埃洛克航空科技（北京）有限公司 | Road image processing method, device, equipment and storage medium |
| CN114419338B (en) * | 2022-03-28 | 2022-07-01 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN116912820B (en) * | 2023-09-13 | 2023-12-12 | 青岛君盛食品股份有限公司 | Visual inspection method for infant food safety |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015010451A1 (en) * | 2013-07-22 | 2015-01-29 | 浙江大学 | Method for road detection from one image |
| CN104392212A (en) * | 2014-11-14 | 2015-03-04 | 北京工业大学 | Method for detecting road information and identifying forward vehicles based on vision |
| CN104700071A (en) * | 2015-01-16 | 2015-06-10 | 北京工业大学 | Method for extracting panorama road profile |
| CN106485715A (en) * | 2016-09-09 | 2017-03-08 | 电子科技大学成都研究院 | A kind of unstructured road recognition methods |
| CN106651872A (en) * | 2016-11-23 | 2017-05-10 | 北京理工大学 | Prewitt operator-based pavement crack recognition method and system |
| CN107292253A (en) * | 2017-06-09 | 2017-10-24 | 西安交通大学 | A kind of visible detection method in road driving region |
| CN108280450A (en) * | 2017-12-29 | 2018-07-13 | 安徽农业大学 | A kind of express highway pavement detection method based on lane line |
| WO2020107716A1 (en) * | 2018-11-30 | 2020-06-04 | 长沙理工大学 | Target image segmentation method and apparatus, and device |
| CN111611919A (en) * | 2020-05-20 | 2020-09-01 | 西安交通大学苏州研究院 | Road scene layout analysis method based on structured learning |
| CN111798459A (en) * | 2020-06-16 | 2020-10-20 | 北京林业大学 | Unmanned aerial vehicle aerial photography tree self-adaptive segmentation method and system based on switching thought |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7245766B2 (en) * | 2000-05-04 | 2007-07-17 | International Business Machines Corporation | Method and apparatus for determining a region in an image based on a user input |
-
2021
- 2021-01-20 CN CN202110075332.6A patent/CN112818775B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015010451A1 (en) * | 2013-07-22 | 2015-01-29 | 浙江大学 | Method for road detection from one image |
| CN104392212A (en) * | 2014-11-14 | 2015-03-04 | 北京工业大学 | Method for detecting road information and identifying forward vehicles based on vision |
| CN104700071A (en) * | 2015-01-16 | 2015-06-10 | 北京工业大学 | Method for extracting panorama road profile |
| CN106485715A (en) * | 2016-09-09 | 2017-03-08 | 电子科技大学成都研究院 | A kind of unstructured road recognition methods |
| CN106651872A (en) * | 2016-11-23 | 2017-05-10 | 北京理工大学 | Prewitt operator-based pavement crack recognition method and system |
| CN107292253A (en) * | 2017-06-09 | 2017-10-24 | 西安交通大学 | A kind of visible detection method in road driving region |
| CN108280450A (en) * | 2017-12-29 | 2018-07-13 | 安徽农业大学 | A kind of express highway pavement detection method based on lane line |
| WO2020107716A1 (en) * | 2018-11-30 | 2020-06-04 | 长沙理工大学 | Target image segmentation method and apparatus, and device |
| CN111611919A (en) * | 2020-05-20 | 2020-09-01 | 西安交通大学苏州研究院 | Road scene layout analysis method based on structured learning |
| CN111798459A (en) * | 2020-06-16 | 2020-10-20 | 北京林业大学 | Unmanned aerial vehicle aerial photography tree self-adaptive segmentation method and system based on switching thought |
Non-Patent Citations (4)
| Title |
|---|
| 基于北斗卫星导航系统的林区智能巡检测绘系统研究;赵燕东;涂佳炎;;农业机械学报(第07期);全文 * |
| 基于区域边界最优映射的图像分割算法;赵婕;张春美;张小勇;姚峰林;计算机应用研究;第33卷(第1期);全文 * |
| 基于单目视觉的机动车道路检测和跟踪研究;李向阳;CNKI;全文 * |
| 集成学习框架下的道路图像分割算法研究;刘步实;CNKI;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112818775A (en) | 2021-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112818775B (en) | Forest road rapid identification method and system based on regional boundary pixel exchange | |
| CN109977812B (en) | Vehicle-mounted video target detection method based on deep learning | |
| CN108052880B (en) | Virtual and real lane line detection method for traffic monitoring scene | |
| CN109657632B (en) | Lane line detection and identification method | |
| CN110263717B (en) | Method for determining land utilization category of street view image | |
| CN106651872A (en) | Prewitt operator-based pavement crack recognition method and system | |
| CN103824081B (en) | Method for detecting rapid robustness traffic signs on outdoor bad illumination condition | |
| CN107273896A (en) | A kind of car plate detection recognition methods based on image recognition | |
| CN111738064B (en) | Haze concentration identification method for haze image | |
| CN108549864A (en) | Area-of-interest filter method towards vehicle-mounted thermal imaging pedestrian detection and device | |
| CN111580131B (en) | Method for identifying vehicles on expressway by three-dimensional laser radar intelligent vehicle | |
| CN109840483B (en) | Landslide crack detection and identification method and device | |
| CN103136537B (en) | Vehicle type identification method based on support vector machine | |
| CN110992381A (en) | Moving target background segmentation method based on improved Vibe + algorithm | |
| CN108171695A (en) | A kind of express highway pavement detection method based on image procossing | |
| CN108509950B (en) | Railway contact net support number plate detection and identification method based on probability feature weighted fusion | |
| CN110473199A (en) | A kind of detection of color spot acne and health assessment method based on the segmentation of deep learning example | |
| CN110889840A (en) | Effectiveness detection method of high-resolution 6 # remote sensing satellite data for ground object target | |
| CN113409267A (en) | Pavement crack detection and segmentation method based on deep learning | |
| CN113837094A (en) | Road condition rapid analysis method based on full-color high-resolution remote sensing image | |
| CN106845458A (en) | A kind of rapid transit label detection method of the learning machine that transfinited based on core | |
| CN113628164A (en) | Pavement crack detection method based on deep learning and web end positioning | |
| CN112906616A (en) | Lane line extraction and generation method | |
| Yang et al. | PDNet: Improved YOLOv5 nondeformable disease detection network for asphalt pavement | |
| CN115170479A (en) | Automatic extraction method for asphalt pavement repairing diseases |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |