CN112036466A - Mixed terrain classification method - Google Patents
Mixed terrain classification method Download PDFInfo
- Publication number
- CN112036466A CN112036466A CN202010873934.1A CN202010873934A CN112036466A CN 112036466 A CN112036466 A CN 112036466A CN 202010873934 A CN202010873934 A CN 202010873934A CN 112036466 A CN112036466 A CN 112036466A
- Authority
- CN
- China
- Prior art keywords
- pixel
- terrain classification
- segnet
- result
- terrain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 230000011218 segmentation Effects 0.000 claims abstract description 28
- 238000012545 processing Methods 0.000 claims abstract description 6
- 230000008569 process Effects 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 5
- 238000005192 partition Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 3
- 238000011524 similarity measure Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 4
- 230000005021 gait Effects 0.000 description 4
- 239000000654 additive Substances 0.000 description 3
- 230000000996 additive effect Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a mixed terrain classification method, which comprises the following steps that 1, a camera is used for obtaining a color image and a depth image of the same geographic environment; 2. performing SLIC superpixel segmentation on the color image to obtain a plurality of terrain boundary segmentation areas; 3. obtaining a SegNet terrain classification result and a semantic filling result, wherein the semantic filling result is marked by RGB color identification; 4. combining the SLIC boundary segmentation result with the SegNet terrain classification result, wherein the pixels correspond to each other, finding out the mode of RGB components of the SegNet terrain classification area containing all the pixels, using the mode of three components as the RGB value of the SegNet terrain classification area, and assigning the RGB value to all the pixels in the SegNet terrain classification area; 5. taking the mode of the RGB components as a semantic filling value of the classification area, and assigning the semantic filling value to pixel points corresponding to the segmentation area in the SLIC super-pixel segmentation processing result; and obtaining a terrain classification result.
Description
Technical Field
The invention belongs to the field of machine vision, and relates to a mixed terrain classification method.
Background
The premise that the outdoor mobile robot keeps normal work is that the outdoor mobile robot has autonomous navigation capability and terrain recognition capability. The perception and detection of the environment by the robot mainly depend on the obtained visual information, and the vision-based terrain recognition helps the robot to quickly acquire the upcoming terrain environment and make a reasonable prediction. The terrain is classified by an appropriate terrain classification method, so that the relevant environment information is fed back to the robot to select the most appropriate gait. Currently, the most widely used classification methods include SVMs, neural networks, bias classifiers and gaussian mixtures, which mostly have good adaptability and high recognition accuracy. However, since the multi-legged robot is mainly used in a complex and unstructured severe environment, these terrain classifiers have certain limitations and cannot meet the classification requirements of mobile robots in mixed terrain. In terrain classification, most classification methods have limited recognizable terrain samples and cannot be applied to complex and variable unstructured outdoor environments, so that terrain recognition is wrong or unrecognized terrain types occur. In addition, the classification result of the mixed terrain and the determination of the terrain boundary information determine the gait adopted by the robot, and provide important basis for the multi-legged robot to timely make gait transformation suitable for walking on the terrain. The terrain classification process can be quickly realized by a terrain classification method, but clear boundary information cannot be obtained all the time, so that the gait of the robot is changed at an improper moment, and the balance of the robot is influenced.
Disclosure of Invention
The present invention is directed to overcome the above-mentioned shortcomings of the prior art, and to provide a method for classifying mixed terrains, which improves the accuracy of terrains classification and the definition of boundaries.
In order to achieve the purpose, the invention adopts the following technical scheme to realize the purpose:
a method of mixed terrain classification comprising the steps of;
firstly, a camera is used for acquiring a color image and a depth image of the same geographic environment;
step two, SLIC superpixel segmentation is carried out on the color image to obtain a plurality of terrain boundary segmentation areas;
thirdly, utilizing the color image and the depth image to obtain a SegNet terrain classification result and a semantic filling result, marking the semantic filling result with RGB color identification marks, and marking the SegNet terrain classification result and the RGB color identification marks on the depth image;
step four, overlapping the color image subjected to SLIC superpixel segmentation processing in the step two with the depth image in the step three, combining the SLIC boundary segmentation result with the segNet terrain classification result, corresponding the segNet terrain classification result with the pixel point of the SLIC boundary segmentation result, finding out the mode of the segNet terrain classification area containing RGB components of all pixels, taking the mode of the three components as the RGB value of the segNet terrain classification area, and assigning the RGB value to all pixels in the segNet terrain classification area;
step five, taking the mode of RGB components in the corresponding pixel points in the SegNet terrain classification result as the semantic filling value of the classification area, and assigning the mode to the pixel points corresponding to the segmentation areas in the SLIC superpixel segmentation processing result; and obtaining a terrain classification result.
Further, in the second step, the SLIC superpixel segmentation of the color image comprises the following steps;
a, initializing a clustering center: generating the set number of super pixels, and uniformly distributing the seed points in the color image;
b, correcting the clustering center: calculating gradient values of all pixel points in an n-x-n neighborhood of the seed points, and replacing original seed points at the places with minimum gradients in the neighborhood;
c, label distribution of pixel points: distributing class labels for all pixel points in the neighborhood of the seed points, and judging the cluster centers to which the pixel points belong;
d, similarity measure: respectively calculating the distance between each seed point and each searched pixel point, and the color similarity and the image coordinate distance between the target pixel point and the seed point;
e, iterative optimization: and (d) repeating the steps a to d for continuous iteration until the error is converged to the clustering center of each pixel point and no change occurs.
Further, in the step a, a seed pixel is randomly found for each region to be segmented as a starting point for growth, then pixels in the neighborhood around the seed pixel, which have the same or similar properties as the seed pixel, are merged into the region where the seed pixel is located, and the new pixels are taken as new seed pixels to continue the above process until no pixel meeting the condition is included, and finally a region is formed.
Further, after the iterative optimization is completed, the super pixels with undersize and discontinuous sizes are redistributed and grouped into the nearest super pixel block, and the pixel points are distributed to the corresponding labels until all the points are traversed.
Further, in step e, the number of iterations is 10.
Preferably, in the second step, each partition area is marked as: a1, a2, A3, …, An, where An represents a set of pixels in each partition area, and is marked as IAn (x, y), and is assigned and marked according to the size of the set of pixels, so that An is equal to n.
Compared with the prior art, the invention has the following beneficial effects:
according to the invention, each pixel point in the SegNet terrain classification result corresponds to the pixel point of the SLIC classification result, and then the mode of the color value of the corresponding pixel point in SegNet is given to the SLIC segmentation result, so that the terrain classification result with clear boundary and semantic filling is obtained. And the classification process is simplified, the accuracy and the efficiency of terrain classification are improved, and the segmentation result is optimized.
Drawings
FIG. 1 is a SLIC-SegNet terrain classification flow diagram of the present invention;
FIG. 2 is a topographic environment map of the camera acquisition of the present invention;
FIG. 3 is a diagram of SLIC superpixel segmentation results according to the present invention;
FIG. 4 is a diagram of the SegNet terrain classification results of the present invention;
fig. 5 is a diagram of SLIC-SegNet terrain classification results of the present invention.
Detailed Description
The invention is described in further detail below with reference to the accompanying drawing 1:
step one, collecting an image k by using a Kinect v2.0 camera, and acquiring a color image and a depth image, wherein pixel points of the color image and the depth image correspond to each other one by one as shown in FIG. 2. The image size is 480 × 360, and the coordinates of each pixel point are recorded as: (x)i,yj)。
And step two, performing SLIC superpixel segmentation and clustering processing on the color image to obtain a terrain boundary segmentation result, as shown in FIG. 3. And marking each partition area as: a. the1,A2,A3,...AnWherein A isnThe set representing the pixel points of each partition region is marked as IAn(x, y), and carrying out assignment marking according to the size of the pixel point set, and enabling An to be equal to n. The SLIC superpixel segmentation algorithm comprises the following steps:
a. initializing a clustering center: the method comprises the steps of generating a set number of super pixels, uniformly distributing a plurality of seed points in a color image, specifically, randomly finding a seed pixel as a growth starting point for each region needing to be segmented, and then combining pixels which have the same or similar properties with the seed pixels in the neighborhood around the seed pixels into the region where the seed pixels are located. The above process continues with these new pixels as new seed pixels until no more pixels that satisfy the condition can be included, eventually forming a region.
Assuming that the color image has P pixel points and the number of the pre-partitioned superpixel blocks is N, the size of each superpixel is P/N, and the step length between the superpixels is P/N
So as to ensure that the superpixel blocks with regular shapes and uniform and compact distribution are generated.
b. Correction of the cluster center: and calculating gradient values of all pixel points in an n-x-n neighborhood of the seed points, and replacing the original seed points at the positions with the minimum gradient in the neighborhood, thereby avoiding the influence of the center on the contour boundary with larger gradient on the clustering effect.
c. And (3) label distribution of pixel points: and (4) distributing a class label for each pixel point in each seed point neighborhood, and judging which clustering center the pixel point belongs to.
d. Similarity measurement: and respectively calculating the distance between the seed point and each searched pixel point, wherein the distance comprises the color similarity between the target pixel point and the seed point and the distance of the image coordinate.
Wherein l, a, b are characteristic parameters of the pixel in Lab color space, dcIs the color difference, d, of the pixel point i, jsIs the Euclidean distance, N, of the pixel point i, jsThe maximum space distance in the class is equal to the step value. N is a radical ofcFor maximum color distance, we usually take a fixed constant m, with a value in the range of [1, 40 ]]And 10 is usually taken.
e. Iterative optimization: and repeating the process for continuous iteration until the error converges to the clustering center of each pixel point and no change occurs. Experiments prove that when the iteration frequency reaches 10 times, most images can achieve the ideal effect, so that the iteration frequency is generally 10.
f. And (3) enhancing connectivity: and for the super pixels with undersize and discontinuous sizes, redistributing the super pixels into the nearest super pixel block, and distributing the pixel points to the labels where the corresponding adjacent super pixel blocks are located until all the points are traversed.
And step three, obtaining a SegNet terrain classification and semantic filling result by utilizing the color image and the depth image, wherein each classified area in the color image and the depth image is marked as: b is1,B2,B3,...BmIn which B ismRepresenting a set of pixels in one of the classification regions, semanticsThe filling result is marked by RGB color identification, and the corresponding RGB color of each pixel point is (R (x)i,yj),G(xi,yj),B(xi,yj)). In SegNet, the parameter solving process mainly includes: and calculating residual errors of the convolutional layer and the sub-sampling layer and derivatives of corresponding weight parameters and bias parameters. In the convolution layer, the feature maps of the previous layer and a learnable convolution kernel are subjected to convolution operation, and an output feature map is obtained through an activation function. Each output map can be convolved by combining a plurality of input maps to obtain a SegNet terrain classification result, as shown in fig. 4:
wherein
For the output value of pixel point j at the ith layer in the x direction of coordinate axis, Mj represents the set of selected input maps, and the convolution is an "effective" boundary process. Each output map gives an additive deviation b,
representing a convolution kernel. Then, the network learning speed is accelerated through batch normalization, and the problems of gradient disappearance and gradient explosion are solved. For a given map, its sensitivity map can be calculated. The gradient of the bias base is quickly calculated by summing all nodes in the sensitivity map in the layer l, and the gradient of the convolution kernel weight value can be calculated and obtained by using a BP algorithm. In the sub-sampling layer, there are N input maps and N output maps, except that each output map becomes smaller. Where down (.) is the down-sampling function. Each output map has a multiplicative bias β and an additive bias b corresponding thereto. The gradients of the additive bias b and the multiplicative bias β are then calculated.
Step four, dividing the result A according to the SLIC boundary1,A2,A3,...ANCorresponding each pixel point in the SegNet terrain classification result with the pixel point of the SLIC segmentation result, and finding outAnd (4) the mode of RGB components of all pixel points in the terrain classification area is obtained, the mode of three components is used as the RGB value of the area, and the values are assigned to all the pixel points in the area.
Step five, taking the mode of RGB components in the corresponding pixel points in SegNet as the semantic filling value of the classification area, and assigning the mode to A in SLICnThe corresponding pixel point.
And step six, as shown in fig. 5, obtaining a terrain classification result with clear boundary and semantic filling.
The above-mentioned contents are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modification made on the basis of the technical idea of the present invention falls within the protection scope of the claims of the present invention.
Claims (6)
1. A mixed terrain classification method is characterized by comprising the following steps;
firstly, a camera is used for acquiring a color image and a depth image of the same geographic environment;
step two, SLIC superpixel segmentation is carried out on the color image to obtain a plurality of terrain boundary segmentation areas;
thirdly, utilizing the color image and the depth image to obtain a SegNet terrain classification result and a semantic filling result, marking the semantic filling result with RGB color identification marks, and marking the SegNet terrain classification result and the RGB color identification marks on the depth image;
step four, overlapping the color image subjected to SLIC superpixel segmentation processing in the step two with the depth image in the step three, combining the SLIC boundary segmentation result with the segNet terrain classification result, corresponding the segNet terrain classification result with the pixel point of the SLIC boundary segmentation result, finding out the mode of the segNet terrain classification area containing RGB components of all pixels, taking the mode of the three components as the RGB value of the segNet terrain classification area, and assigning the RGB value to all pixels in the segNet terrain classification area;
step five, taking the mode of RGB components in the corresponding pixel points in the SegNet terrain classification result as the semantic filling value of the classification area, and assigning the mode to the pixel points corresponding to the segmentation areas in the SLIC superpixel segmentation processing result; and obtaining a terrain classification result.
2. The mixed terrain classification method of claim 1, wherein in step two, the color image undergoing SLIC superpixel segmentation comprises the steps of;
a, initializing a clustering center: generating the set number of super pixels, and uniformly distributing the seed points in the color image;
b, correcting the clustering center: calculating gradient values of all pixel points in an n-x-n neighborhood of the seed points, and replacing original seed points at the places with minimum gradients in the neighborhood;
c, label distribution of pixel points: distributing class labels for all pixel points in the neighborhood of the seed points, and judging the cluster centers to which the pixel points belong;
d, similarity measure: respectively calculating the distance between each seed point and each searched pixel point, and the color similarity and the image coordinate distance between the target pixel point and the seed point;
e, iterative optimization: and (d) repeating the steps a to d for continuous iteration until the error is converged to the clustering center of each pixel point and no change occurs.
3. The mixed terrain classification method according to claim 2, characterized in that in step a, a seed pixel is randomly found for each region to be segmented as a starting point for growth, then pixels in the neighborhood around the seed pixel, which have the same or similar properties as the seed pixel, are merged into the region where the seed pixel is located, and the above process is continued with these new pixels as new seed pixels until no more pixels meeting the conditions are included, and finally a region is formed.
4. The method of claim 2, wherein after the iterative optimization is completed, for super pixels with undersize and discontinuity, the super pixels are reassigned and grouped into the nearest super pixel block, and the pixel points are assigned to the corresponding labels until all the points are traversed.
5. A method of mixed terrain classification as claimed in claim 2, characterized in that in step e, the number of iterations is 10.
6. A method for classifying mixed terrain according to claim 1, wherein in step two, each of the divided areas is marked as: a. the1，A2，A3，…，AnWherein A isnThe set representing the pixel points of each partition region is marked as IAn(x, y) and carrying out assignment marking according to the size of the pixel point set to order An＝n。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010873934.1A CN112036466A (en) | 2020-08-26 | 2020-08-26 | Mixed terrain classification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010873934.1A CN112036466A (en) | 2020-08-26 | 2020-08-26 | Mixed terrain classification method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112036466A true CN112036466A (en) | 2020-12-04 |
Family
ID=73581602
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010873934.1A Pending CN112036466A (en) | 2020-08-26 | 2020-08-26 | Mixed terrain classification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112036466A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112861669A (en) * | 2021-01-26 | 2021-05-28 | 中国科学院沈阳应用生态研究所 | High-resolution DEM topographic feature enhancement extraction method based on earth surface slope constraint |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109741341A (en) * | 2018-12-20 | 2019-05-10 | 华东师范大学 | A kind of image partition method based on super-pixel and long memory network in short-term |
| CN110096961A (en) * | 2019-04-04 | 2019-08-06 | 北京工业大学 | A kind of indoor scene semanteme marking method of super-pixel rank |
-
2020
- 2020-08-26 CN CN202010873934.1A patent/CN112036466A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109741341A (en) * | 2018-12-20 | 2019-05-10 | 华东师范大学 | A kind of image partition method based on super-pixel and long memory network in short-term |
| CN110096961A (en) * | 2019-04-04 | 2019-08-06 | 北京工业大学 | A kind of indoor scene semanteme marking method of super-pixel rank |
Non-Patent Citations (2)
| Title |
|---|
| YAGUANG ZHU 等: ""Superpixel Segmentation Based SyntheticClassifications with Clear Boundary Information for a Legged Robot"", 《SENSORS》 * |
| 张弘 等, 机械工业出版社 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112861669A (en) * | 2021-01-26 | 2021-05-28 | 中国科学院沈阳应用生态研究所 | High-resolution DEM topographic feature enhancement extraction method based on earth surface slope constraint |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109685067B (en) | Image semantic segmentation method based on region and depth residual error network | |
| CN109410321B (en) | Three-dimensional reconstruction method based on convolutional neural network | |
| Song et al. | Road extraction using SVM and image segmentation | |
| CN111259936B (en) | Image semantic segmentation method and system based on single pixel annotation | |
| CN109977997B (en) | Image target detection and segmentation method based on convolutional neural network rapid robustness | |
| CN111161317A (en) | Single-target tracking method based on multiple networks | |
| CN111652892A (en) | Remote sensing image building vector extraction and optimization method based on deep learning | |
| CN107784657A (en) | A kind of unmanned aerial vehicle remote sensing image partition method based on color space classification | |
| CN109101981B (en) | Loop detection method based on global image stripe code in streetscape scene | |
| CN109543634B (en) | Data processing method and device in positioning process, electronic equipment and storage medium | |
| CN110634142B (en) | Complex vehicle road image boundary optimization method | |
| CN112036466A (en) | Mixed terrain classification method | |
| CN114820668A (en) | End-to-end building regular outline automatic extraction method based on concentric ring convolution | |
| Liu et al. | Comparison of 2D image models in segmentation performance for 3D laser point clouds | |
| CN114140485A (en) | Method and system for generating cutting track of main root of panax notoginseng | |
| CN117496401A (en) | Full-automatic identification and tracking method for oval target points of video measurement image sequences | |
| CN112767429A (en) | Ground-snow surface point cloud rapid segmentation method | |
| CN111104942A (en) | Template matching network training method, template matching network recognition method and template matching network recognition device | |
| CN113096048B (en) | Method and system for radiation correction of generalized cloud-driven and geometric collaborative remote sensing image | |
| CN114463534A (en) | Target key point detection method, device, equipment and storage medium | |
| Guo et al. | 3D Lidar SLAM Based on Ground Segmentation and Scan Context Loop Detection | |
| CN108921194B (en) | Terrain slope classification self-adaptive clustering method | |
| Gonzalez-Huitron et al. | Jaccard distance as similarity measure for disparity map estimation | |
| Kurbatova | Road detection based on color and geometry characteristics | |
| CN116503743B (en) | Optimal matching method for geographic vector data and high-resolution remote sensing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201204 |
|
| RJ01 | Rejection of invention patent application after publication |