CN110147780B

CN110147780B - Real-time field robot terrain identification method and system based on hierarchical terrain

Info

Publication number: CN110147780B
Application number: CN201910451893.4A
Authority: CN
Inventors: 马昕; 邹家祥; 李贻斌; 宋锐; 荣学文; 田国会
Original assignee: Shandong University
Current assignee: Shandong University
Priority date: 2019-05-28
Filing date: 2019-05-28
Publication date: 2021-01-01
Anticipated expiration: 2039-05-28
Also published as: CN110147780A

Abstract

The invention discloses a real-time terrain identification method and a real-time terrain identification system for a field robot based on a hierarchical terrain, wherein the method comprises the following steps: obtaining a topographic sample to be tested; respectively extracting RGB color histogram characteristics, variance curve characteristics and rotation and symmetry invariant LBP characteristics of the topographic sample to be detected; and inputting the RGB color histogram characteristic and the variance curve characteristic as color characteristics and the rotation and symmetrical invariant LBP characteristic as texture characteristics into a terrain recognition classifier, and obtaining a final classification result through the terrain recognition classifier to finish terrain recognition of the to-be-detected terrain sample. The invention has the beneficial effects that: the use of the variance curve characteristic and the LBP characteristic with unchanged rotation and symmetry greatly reduces the characteristic dimension and obtains good real-time performance.

Description

Real-time field robot terrain identification method and system based on hierarchical terrain

Technical Field

The disclosure relates to the technical field of terrain identification, in particular to a real-time terrain identification method and system for a field robot based on a level terrain.

Background

The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.

In recent years, the robot vision technology has been paid attention to, wherein, for field or other intelligent devices with driving force, the detection and identification capability for terrain scenes fully reflects the intelligence degree thereof. Researchers have done many studies on terrain classification so far, and the problems and the method ideas around them change from the heavy large-scale sensors in the vehicle-mounted system which was originally concerned to the fact that the massive data classification calculation is used nowadays, even the cloud online learning process is covered by the earth. The research on the terrain identification angle gradually develops towards the characteristics of convenience in carrying and installation, small volume, large amount of characteristic data and the like.

The inventor finds that in current algorithm research on terrain recognition, image recognition based on a deep neural network is rapidly developed due to the development of deep learning algorithms in recent years. The deep neural network algorithm based on image convolution is the method which can reach the highest recognition rate so far, and mainly depends on the neural network with nearly infinite training parameters and a mathematical model which is difficult to be interpreted by human visual angles. But the defects are also obvious, and most of deep learning cannot be completely unsupervised, so that an accurately calibrated training sample is needed, a large amount of training time is needed, the error detection caused by a complex model is difficult, and the recognition speed is reduced caused by the fact that the number of network layers is too deep. In terrain recognition, neural network models also have extremely wide diversity, and bring difficulty to comparative evaluation between methods.

Disclosure of Invention

In order to solve the problems, the invention provides a real-time terrain identification method and a real-time terrain identification system for a field robot based on a hierarchical terrain, wherein RGB (red, green and blue) color histogram features, variance curve features and rotation and symmetry invariant LBP (local binary pattern) features are used as the color and texture features of an image, and the features after fusion and terrain samples in a database are used for classification in a subsequent hierarchical classifier.

In some embodiments, the following technical scheme is adopted in the invention:

a real-time field robot terrain identification method based on hierarchical terrain comprises the following steps:

obtaining a topographic sample to be tested;

respectively extracting RGB color histogram characteristics, variance curve characteristics and rotation and symmetry invariant LBP characteristics of the topographic sample to be detected;

and inputting the RGB color histogram characteristic and the variance curve characteristic as color characteristics and the rotation and symmetrical invariant LBP characteristic as texture characteristics into a terrain recognition classifier, and obtaining a final classification result through the terrain recognition classifier to finish terrain recognition of the to-be-detected terrain sample.

In other embodiments, the invention adopts the following technical scheme:

a computer readable storage medium having stored therein a plurality of instructions adapted to be loaded by a processor of a terminal device and to execute the above-mentioned real-time field robot terrain identification method based on hierarchical terrain.

In other embodiments, the invention adopts the following technical scheme:

a terminal device comprising a processor and a computer-readable storage medium, the processor being configured to implement instructions; the computer readable storage medium is used for storing a plurality of instructions which are suitable for being loaded by a processor and executing the real-time field robot terrain identification method based on the hierarchical terrain.

Compared with the prior art, the invention has the beneficial effects that:

(1) the use of the variance curve characteristic and the LBP characteristic with unchanged rotation and symmetry greatly reduces the characteristic dimension and obtains good real-time performance.

(2) The hierarchical terrain recognition classifier accurately classifies single terrain samples to be detected by using the difference of related classification coefficients among different hierarchies, so that the operation speed is greatly improved; the method is very instructive to subsequent terrain type extension and fuzzy terrain processing.

(3) Compared with the traditional SVM classifier or the classifier based on the neural network, the classification accuracy and the classification instantaneity can be well balanced.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application.

FIG. 1(a) is a sample of a topographic data set of a concrete slab according to one embodiment;

FIG. 1(b) is a schematic representation of a herbal terrain dataset of an embodiment;

FIG. 2 is a flowchart of a terrain recognition method for a real-time field robot based on a hierarchical terrain according to an embodiment;

FIGS. 3(a) - (b) are a chart of a grass 1 swatch and an RGB histogram in one embodiment;

FIGS. 4(a) - (b) are a grass 2 swatch and an RGB histogram of one embodiment;

FIG. 5 is a diagram illustrating RGB variance curves in accordance with an embodiment;

FIG. 6 is a flow chart of the RGB variance curve calculation in the first embodiment;

FIG. 7 is a table of the rotational and symmetric invariant LBP neighborhood in one embodiment;

FIGS. 8(a) - (b) are a scene graph and a rotation and symmetry invariant LBP feature graph in the first embodiment;

fig. 9 is a schematic diagram of a double-layer terrain partition type in the first embodiment.

Detailed Description

It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.

Example one

In one or more embodiments, disclosed is a real-time field robot terrain recognition method based on hierarchical terrain, comprising: obtaining a topographic sample to be tested; respectively extracting RGB color histogram characteristics, variance curve characteristics and rotation and symmetry invariant LBP characteristics of the topographic sample to be detected; and inputting the RGB color histogram characteristic and the variance curve characteristic as color characteristics and the rotation and symmetrical invariant LBP characteristic as texture characteristics into a terrain recognition classifier, and obtaining a final classification result through the terrain recognition classifier to finish terrain recognition of the to-be-detected terrain sample.

The method of this embodiment will be described in detail below.

1 topographic dataset

The SDU-Terrain version1.0 Terrain data set is a Terrain database which is built by a robot research center of Shandong university by self-acquiring live-action pictures, and the first generation SDUterrain database adopts classical Terrain classification through the summary processing of former Terrain researchers, as shown in figures 1(a) - (b); the method comprises the following steps: cement (Asphalt), Mulch (Mulch), Grass (Grass), Mud (Mud), Sand (Sand), Gravel (Gravel) and the like.

The embodiment performs the following supplementary and standardization work on the basis of continuing to use the first generation SDU-Terrain version1.0 database:

1) the variety of data sets is expanded. Sky, shrub and tree data sets are added, and interference of natural targets on terrain identification during scene segmentation is avoided through detection and segmentation of the natural targets. Each sample data retains 350 sample slices

2) And a sample format aiming at a PC terminal product platform program is standardized. The training samples are composed of samples with 60 × 60 resolution, and the picture format is Joint Photographic Experts Group (JPEG), and the picture format has the characteristic that the compression ratio is high and is usually between 10:1 and 40:1, so that the storage pressure of the robot is greatly relieved.

3) The original picture is a natural scene under different illumination. In order to enhance robustness to different light interferences in data processing during acquisition of training samples, three groups of shooting of the same terrain are respectively performed at 8 am, 2 pm and 4 pm of the season when data images are acquired.

The SDU-Terrain Version2.0 is expanded and standardized into 7 natural scene types, namely Concrete ground (Concrete), grassland (Grass), Mud land (Mud), Sand land (Sand), Bush (Bush), Sky (Sky) and Trees (Trees). Wherein, the terrain sample types are 5, each has 350 samples, and the resolution of each sample is 60 multiplied by 60. Each type of sample wafer has sample wafers with different illumination conditions in three time periods. After the data set is expanded and standardized, the requirements of subsequent data training and image recognition segmentation work can be met.

2 terrain recognition method

The whole algorithm flow of the identification method used in this embodiment is shown in fig. 2, the analysis of the input to-be-detected terrain sample is mainly based on two types, namely color features and texture features, the input to a hierarchical terrain classifier after feature extraction, feature fusion in different proportions is performed in the classifier according to different branch structures, a discrimination result is finally obtained, and a final classification result is obtained through a discrimination sorting algorithm.

2.1 terrain recognition method

2.1.1RGB color histogram feature

The RGB color histogram feature is a typical statistical feature based on color elements. The color histogram feature is widely applied to retrieval and information extraction of two-dimensional images, the feature distribution is based on regional statistics, information of RGB three channels of all points in the two-dimensional images is arranged according to gray intensity, and pixel colors of original images are recoded. The coding restores the color information of the original image, simply strips the associativity and the arrangement sequence of RGB channel colors, and only counts the RGB color distribution condition, so the coding has obvious advantages in the aspect of image statistics of repeated arrangement of similar elements, and has no strong reference significance to images without repeated distribution characteristics. FIGS. 3(a) - (b) and FIGS. 4(a) - (b) show two grass swatches and their RGB histograms, respectively.

The RGB color histogram extraction mainly comprises the following steps:

1) inputting a topographic sample to be tested

2) Analyzing the color intensity of three channels point by point to obtain the number of gray distribution pixels.

3) The ratio of each gray scale is calculated according to the number of pixels of the sample.

The expression formula of the RGB color histogram feature is as follows:

in the formula (1), H represents the distribution of the number of gray levels under different channels, H_GRepresenting the statistical quantity under the gray level G (Grey level) of different channels, Col representing the width (Column) of the topographic sample to be tested, Row representing the height (Row) of the topographic sample to be tested,_Gand (I, j) represents whether the gray level of the pixel at the jth row of the ith column is equal to G, and I (I, j) represents the gray level at the jth row of the ith column in the three image channels.

2.1.2RGB variance Curve characteristics

Based on the color histogram feature described above, a curve feature based on RGB variance is proposed herein to eliminate the influence of the expansion of the test set data on the data processing efficiency. The RGB variance curve characteristic is a discrete RGB characteristic range, the expected value is the mean value of RGB characteristics of 3 x 256 dimensions, and each dimension characteristic can obtain an accurate variance value according to the distribution condition of an actual training sample. Each type of terrain stores a set of color grayscale means and a set of color grayscale variances, each set of data having dimensions of 3 x 256. This feature data is performed during the training of the data set, so that the scale of the features does not change regardless of the number of training coupons expanded.

And (3) performing statistical calculation on the training set in the local form, wherein the calculation process is shown as formula (2).

Wherein R represents the statistical score of the gray mean of the to-be-measured terrain sample with the variance curve characteristicCloth, r_GExpressing the gray average value of the terrain sample to be tested under the gray level G, Size expressing the number of the sample under the local type, R' expressing the statistical distribution of the gray average occupancy rate of the terrain sample to be tested under the variance curve characteristic, B expressing the statistical distribution of the gray variance of the variance curve characteristic, B_GAnd B' represents the gray variance of the normalized variance curve characteristic.

The RGB variance curve is characterized by the average number of pixels at different gray levels for different channels under the native shape type, and the variance deviation range centered on this value. The column in fig. 5 represents the statistical mean of the topographic sample to be measured at different gray levels, and the top represents the variance of the sample under the statistical mean. The variance curve characteristic is mainly a process of obtaining a mean value and a variance from a terrain training set, the flow is shown in fig. 6, the process is independent of the number of samples in the training set, and the characteristic condition is independent of the number of samples, so that the real-time identification efficiency caused by the number of the training samples can be completely eliminated in a computer terrain identification system.

2.1.1 rotationally and symmetrically invariant LBP feature with unidirectional dead band

The Local Binary Pattern (Local Binary Pattern) is a gray level intensity expressed by the sum of color channels, is a texture feature description, and has the advantages of high calculation speed, high feature point searching efficiency and obvious boundary line extraction effect. The disadvantage is that the method is easily interfered by noise, and the non-unity, namely zero, judgment condition is low in robustness to the noise.

The present embodiment provides an LBP algorithm based on the LBP features and based on the rotation and symmetry invariance of the region statistics. Based on the traditional LBP algorithm, aiming at the characteristics of no rotation and symmetry characteristics of similar element arrangement in the terrain texture, a 256-dimensional LBP operator is compressed into 30 dimensions, and the rotation and symmetry in the original texture image are completely eliminated on the basis of reducing the data volume. The improved algorithm can extract more effective LBP texture features, and the accuracy of subsequent classification is also more advantageous due to the characteristics of rotation and symmetry invariance while the feature dimension is reduced.

The rotation and symmetrical invariant LBP characteristics are composed of two parts, namely the mapping calculation of the LBP characteristics and the establishment of an adjacency list. The LBP mapping calculation mainly uses an algorithm to map the traditional LBP characteristics to 30-dimensional rotation and symmetrical invariant LBP characteristics, and is beneficial to subsequent terrain type classification. The purpose of the adjacent table is to use in a watershed water filling method used in subsequent image segmentation and filling operation. Compared with the RGB color gray level of the image, the LBP texture has no corresponding size relation, so that the LBP texture is used in a watershed algorithm in a fusion mode with color features, and a rotation and symmetrical invariant LBP adjacency list is introduced in order to solve the correspondence of the size relation. The idea of the mapping algorithm of the mapping calculation of the rotation and symmetry LBP characteristics is that the LBP characteristic binary representation value Q of the point (i, j) is calculated according to the idea of the LBP algorithm_LBP(i, j) and calculating the LBP reverse order characteristic binary representation value Q according to bit reverse order_L′_BP(i, j) when cyclically shifted by Q_LBP(i, j) and Q_L′_BPAnd (i, j) obtaining the numerical value minimum mode code, namely obtaining the mode class to which the code belongs. In this way, a set of mappings from 256 dimensions to 30 dimensional schema classes can be obtained. The mapping relation can be calculated and stored by a computer at the starting time or calculated at the running time, and the result is called when the field robot extracts the texture features.

The rotation and symmetry invariant LBP algorithm firstly uses a computer to exhaustively calculate the mode codes without rotation and symmetry characteristics, and uses a method of solving the minimum binary number to sort the mode codes in order to prevent the binary codes from being repeated after the binary codes are rotationally symmetric. As shown in equation (3):

wherein Q is_LBP(i, j) is the right-rotated binary code for point (i, j), Q_L′_BP(i, j) is the left-hand-rotated binary code of point (i, j), g_cIs the LBP feature center point gray value, g_bIs the gray value of the b-th comparison point of the LBP characteristic, d represents the forward gray dead zone threshold valueShift (Q, p) is a cyclic right Shift operation of the binary code, shifting the binary Q to the right by p bits, complementing the bits beyond the right from the left in a cyclic manner, min (x) is the minimum of the value.

(2) Neighborhood table of rotation and symmetric invariant LBP features

In the embodiment, the LBP feature with the unchanged rotation sequence is used as a terrain texture feature, and the neighborhood table is established by changing the 30-dimensional mode of the feature by one bit according to two adjacent modes. The 30 abstract LBP pattern classes in fig. 7 correspond to the adjacency relationships in table 1.

The purpose of establishing and initializing the adjacency list by the terrain system is to enable LBP texture feature vectors with unchanged rotation to have a coupling relation, improve robustness of regional statistical classification, and enable patterns to be lack of a comparison method and have no ordering relation, so that no reference system exists when filling is performed by a water injection method during image segmentation. Therefore, the terrain system needs to establish the LBP vector neighborhood table in the system initialization phase. As shown in fig. 7, the neighborhood feature numbers corresponding to each dimension of the 30-dimensional features are labeled. The sufficient requirement that the two-dimensional characteristic is a neighborhood characteristic is that the two-dimensional 8-bit characteristic value has difference of only one bit and binary number after rotation sequence change. The neighborhood table can be stored in a field robot memory or temporarily calculated by using an operating system during starting, and the speed of temporarily calculating the rotation-invariant LBP neighborhood table can meet the delay requirement during starting the system because the rotation sequence invariant feature is subjected to dimension compression and only has 30-dimensional feature vectors.

TABLE 1 rotationally and symmetrically invariant LBP feature neighbor pattern table

After the rotation and the sequence invariance are eliminated, 30 LBP models exist, and a mapping table of 256-dimensional features to the 30-dimensional features is stored in the computer so as to classify the LBP features of the pixel points. Compared with the dimension reduction idea of LBP equivalent mode class proposed by Ojala et al, the dimension reduction idea has the advantages that the storage data volume is improved to about 11.7% of the total amount of original data, the compression ratio is about 15:2, the description of detail problems is more accurate, almost only rotation and symmetry characteristics are stripped, other data are reserved, and the dimension reduction idea is suitable for identification of terrain texture.

Meanwhile, in order to reduce the gray sensitivity of the to-be-detected terrain sample image, a one-way dead zone is introduced into the original LBP characteristic. The unidirectional dead zone means that an offset is introduced for judging the LBP classification in the positive direction or the negative direction, for the terrain sample to be detected, the dead zone value in the positive direction means that the bit in the characteristic byte is a bright point, and the dead zone value in the negative direction means that the bit in the characteristic byte is a dark point. The introduction of the dead zone can avoid the influence of most noises, and because the change of the to-be-measured terrain sample in a small-range gray scale area does not have reference value for many times, the introduction of the dead zone can eliminate the ubiquitous phenomenon. However, the introduction of the dead zone also means that only the abrupt change pixels of the bright spots and the abrupt change pixels of the dark spots are searched, and certain detail characteristics are lost, but in general, the method greatly improves the accuracy of final identification and the anti-interference performance of the system. Fig. 8(a) - (b) give scene graphs and rotation and symmetry invariant LBP feature graphs.

2.2 terrain-level methods and classifier design

2.2.1 multilayer topographic structure and parallel fusion method

The feature fusion method used in the single terrain recognition of the embodiment mainly includes serial and parallel fusion of the RGB color histogram feature and the conventional LBP feature, and serial and parallel fusion of the RGB color histogram feature and the LBP feature with unchanged rotation order. The RGB three-channel color histogram feature I has 256 multiplied by 3 dimensional features, the traditional LBP feature has 256 dimensional features, and the LBP feature with unchanged rotation order has 30 dimensional features.

The embodiment provides a parallel feature fusion method based on terrain hierarchy based on the characteristics of a terrain recognition system. Corresponding to the classifier to be mentioned below, the terrain classifier provides branch data of the terrain structure, which can be used as an index coefficient to be provided when feature fusion. The algorithm thinking is that a texture characteristic factor and two corresponding base type base numbers are set and stored on corresponding hierarchical branches in a classifier, hierarchical classification is carried out from top to bottom in terrain identification calculation to obtain a discrimination result, the type of a to-be-detected terrain sample is judged, the operation is carried out once in a certain node of each layer, and iteration is carried out from a root node. Wherein D represents the standard of the distinguishing classifier of the terrain category of the current layer, the classification standard corresponding to each layer is obtained through the classification result of the previous layer, each layer corresponds to one classifier, and the parameter standard D can be static and obtained from the result of a training set or dynamic and obtained through online learning of feedback data. Static training derived data methods are used herein.

The parallel fusion based on the terrain levels is to investigate the lower subdivision types on the basis of investigating different upper terrain types. The branches provided by the terrain recognition classifier all contain a decision coefficient related to the color texture of the corresponding terrain, so as to adjust the terrain recognition model.

The classifier design of the layered terrain can allow the design of completely independent classifiers in each layered classification system without being limited to each other. The classifier in the terrain classification is based on the discrimination of all terrain data in each layer, namely, the result of the terrain classification is influenced by other terrains in the same layer, compared with the traditional SVM classifier which directly classifies all terrains in one layer, the SVM classifier is not easy to identify the terrains with fuzzy terrain concepts, the fuzzy terrains are easy to be directly filtered by the classifier, and in addition, the layered terrain design is very easy to expand and correct on line. The terrain layer can obscure the boundaries of the terrain classification of the previous layer, so that different terrain categories can be distinguished more easily on a more abstract classification layer, and accurate determination of terrain concrete categories can be made on the lower iteration of concrete subdivision. The method conforms to the characteristic of clear terrain identification level, in addition, in a relatively abstract base level, the judgment limit is more clear, and the model is easy to build.

2.2.2 terrain recognition classifier

The histogram cross comparison method is a fast and efficient discrete data identification method. And performing weighted difference calculation on the range of the area blocks by using two groups of data with the same dimensionality and arrangement relation so as to calculate the difference between the two groups of overall data. The larger the obtained difference degree is, the more obvious the type difference between the two groups of multi-dimensional discrete data is, and conversely, the smaller the difference degree is, the smaller the type difference between the two groups of multi-dimensional discrete data is.

The algorithm idea is to use two groups of data vectors, arrange the P-dimension data vectors in respective histograms in sequence, and form two groups of histogram data. And setting a current comparison vector i in the first histogram, sequentially selecting 1-P-dimensional data vectors from the i, weighting and carrying out difference modulo with data which takes the i as the center and has the neighborhood width of k in the second histogram, and accumulating the obtained modulo to determine the difference degree d of two groups of discrete data. The algorithm is as the formula (4):

wherein, ω (x) is a weighting term for increasing the weight of the central contrast point, and k is a weighted neighborhood H₁(i) And H₂(j) Are respectively a comparison feature group H₁And H₂And the ith and j vector eigenvalues of (1).

In the embodiment, when the histogram feature cross comparison method is used, the process of judging the terrain category according to human vision is simulated. The method comprises the following steps of dividing the terrain in a terrain library into a two-layer structure, wherein the first layer is composed of three scene elements of Roads (Roads), Plants (Plants) and Sky (Sky), the second layer structure of the road elements is composed of Concrete Roads (Concret), Sand lands (Sand) and muddy lands (Mud), and the second layer structure of the plant elements is composed of Trees (Trees), shrubs (Bush) and grasslands (Grass). The main hues of the three road elements are mainly gray, dark yellow and brick red, and the main hues of the plant elements are mainly emerald green, dark green and dark green. Among the road elements, the concrete road surface is more smooth than the sand, and the line feel of the muddy ground is much stronger than that of the concrete road surface and the sand road surface, as shown in fig. 9.

The cross comparison algorithm of the double-layer classification structure is designed according to the bionic logic of judging the terrain by human vision. The scaling factor (x) is used to scale the two types of features in different layer classifications. The algorithm thinking is that the RGB color histogram features input the multidimensional features into the terrain classifier 1, the LBP or other texture features are input into the terrain classifier 2, the texture features are not added as the consideration objects when the judgment is carried out for the first time, after the fact that the terrain belongs to which type in the first layer of scene is determined, the judgment can be carried out only in the corresponding type of terrain, and at the moment, the color judgment domain can be expanded into a full-value domain. When the first type of terrain elements are determined, the number of the second type of terrain elements to be classified is also greatly reduced, and the real-time performance of terrain recognition of the robot can be well improved by applying the method to a field robot platform.

When the second-layer terrain elements are judged, the similarity of corresponding LBP or other texture features is calculated, when the second-layer terrain elements are classified, in order to dynamically combine color and texture features together, a dynamic variable (x) is set, wherein x is the reliability of the RGB color histogram features for judging the sample to be detected to be the closest terrain of a certain class through the classifier 1, and the value of (x) is between 0 and 1 and is a weighted item of the color features. When the terrain category can be accurately identified through the RGB color histogram feature, (x) is closer to 1, otherwise, (x) is closer to 0, and finally the judgment result of the terrain category is mainly determined by the classifier 2 according to the texture feature.

In this embodiment, the RGB color histogram feature and the variance curve feature mentioned above are used for the color feature, and a cross-comparison method is used, and the algorithm idea is shown in formula (5):

wherein d is_R，d_G，d_BThe histogram feature discrimination of RGB three channels, k is the width of cross-comparison neighborhood, P is the number of feature vectors in a certain channel feature,

is the characteristic value of x color intensity in the color histogram characteristic channel c of the image img, and D is the total discrimination of the RGB color histogram characteristic. Generally, in the experiment, the img1 takes the image of the region to be measured, and the img2 takes the data of the training set.

And inputting the terrain texture features into a terrain texture feature classifier, and fusing output results of the terrain texture feature classifier with output result features of a color feature classifier after normalization processing to finally finish classification of second-layer terrain elements of terrain classification.

For the texture features, a rotation and symmetrical invariant LBP neighborhood table is used, the distribution conditions of all dimensions of the texture features can be expanded in a one-dimensional coordinate axis and convoluted with the texture features of the sample plates in the sample plate set, and the texture difference obtained by taking a neighborhood mode as a scale can be obtained. Similar to the color features, the greater the degree of difference, the greater the texture difference of the swatches, and the lower the likelihood of uniformity.

When the double-layer terrain structure is used for cross alignment, the extracted texture features are mapped by the method, normalized into the feature mode corresponding to the color features, fused with the double-layer classification structure adjusting coefficient to form the final classification result and output.

3 experiments and analyses

3.1 Classification experiment of topographic sample to be tested

In this embodiment, the Terrain sample set in SDU-Terrain version2.0 is used as a training set and a test set. All the terrain samples are subjected to standardization work of resolution and illumination condition diversity, so that the influence of more objective differences can be avoided. For comparison, LIBSVM is used as an SVM source code library, and the LIBSVM is a simple and easy-to-use C + + library and has a good reference value in the industry.

The method comprises the steps of setting 8 groups of test methods for test experiments of to-be-tested terrain samples, and testing test sets of cement land, sand land, muddy land, grassland, forest and shrub by respectively using RGB color histograms and variance curve characteristics as color characteristics, LBP with unchanged rotation and symmetry as texture characteristics and using a double-layer terrain structure and an SVM as classifiers, wherein each group of 240 test samples are provided. The classification accuracy as shown in table 2 was obtained.

TABLE 2 dataset accuracy for several algorithms

In table 2, RGB-H refers to RGB color histogram features, LBP refers to normal grayscale LBP, SVM refers to support vector machine classifier, DTH refers to terrain level classifier, ILBP refers to rotational and symmetric invariant LBP, RGB-SDC refers to color variance curve features, and the other data in the table are. It can be seen that in most cases, the classification accuracy using RGB-H and ILBP is better than using RGB-SDC and LBP features.

Because the training set has limited types and may not completely cover the types of the terrain samples of the test set, the recognition rate is obviously reduced under some terrain types (such as Bush). The terrain recognition system of the embodiment is based on the analysis of the difference degree, and in a real field robot system, more fuzzy samples can be classified into an unfolded type, so that the fuzzy samples can be abandoned in a judgment system. If the terrain sample is computationally marked as of an outstanding type, a higher relative accuracy can be achieved, which can be increased by approximately 5% and a relative error rate below 3%.

The embodiment provides a field scene terrain classifier based on hierarchical terrain, and the accuracy of identification can be guaranteed while the data volume is greatly compressed by using the fusion of the color and texture characteristics mentioned in the text. Compared with the traditional SVM classifier or the classifier based on the neural network, the classification accuracy and the classification instantaneity can be well balanced.

Example two

In one or more embodiments, a terminal device is disclosed that includes a processor and a computer-readable storage medium, the processor to implement instructions; the computer readable storage medium is used for storing a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the real-time field robot terrain identification method based on the hierarchy terrain in the first embodiment. For brevity, no further description is provided herein.

It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and so on. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The computer readable storage medium may include a read-only memory and a random access memory and provide instructions and data to the processor, and a portion of the memory may also include a non-volatile random access memory. For example, the memory may also store device type information.

In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software.

The steps of a method in connection with one embodiment may be embodied directly in a hardware processor, or in a combination of the hardware and software modules in the processor. The software modules may be located in ram, flash, rom, prom, or eprom, registers, among other storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor. To avoid repetition, it is not described in detail here.

Those of ordinary skill in the art will appreciate that the various illustrative elements, i.e., algorithm steps, described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.

Claims

1. A real-time field robot terrain recognition method based on hierarchical terrain is characterized by comprising the following steps:

obtaining a topographic sample to be tested;

the method for extracting the variance curve characteristics of the to-be-detected terrain sample comprises the following steps:

wherein R represents the statistical distribution of the gray mean of the terrain sample to be measured with variance curve characteristics, R_GExpressing the gray average value of the terrain sample to be tested under the gray level G, Size expressing the number of the sample under the local type, R' expressing the statistical distribution of the gray average occupancy rate of the terrain sample to be tested under the variance curve characteristic, B expressing the statistical distribution of the gray variance of the variance curve characteristic, B_GRepresenting the gray level of the topographic sample to be measured under the gray level GVariance, B' represents the gray-scale variance of the normalized variance curve characteristic, h_GExpressing the statistical quantity under the gray level G under different channels, Col expressing the width of the to-be-detected terrain sample, Row expressing the height of the to-be-detected terrain sample;

2. The terrain recognition method of the real-time field robot based on the hierarchical terrain as claimed in claim 1, wherein the extraction of the RGB color histogram features of the terrain sample to be tested specifically comprises:

analyzing the three-channel color intensity point by point for the topographic sample to be detected to obtain the number of gray-scale distribution pixels;

and calculating the occupation ratio of each gray level according to the gray level distribution pixel number of the to-be-detected terrain sample.

3. The terrain recognition method for a real-time field robot based on hierarchical terrain as claimed in claim 1, wherein the rotation and symmetry invariant LBP features comprise mapping calculation of LBP features and creation of adjacency list;

the method for calculating the mapping of the LBP characteristics specifically comprises the following steps:

calculating the LBP characteristic binary representation Q of the point (i, j)_LBP(i, j), calculating LBP reverse order characteristic binary representation value Q 'in bit reverse order'_LBP(i,j)；

Respectively cyclic shift Q_LBP(i, j) and Q'_LBP(ii) the binary code of (i, j);

obtaining a binary code with the minimum numerical value as a mode class to which the binary code belongs;

finally, a set of mapping relations from 256 dimensions to 30-dimensional schema classes is obtained.

4. The terrain recognition method of a real-time field robot based on hierarchical terrain as claimed in claim 3, wherein the rotation and symmetry invariant LBP features are used to build a neighborhood table by changing 30-dimensional pattern classes of the features into two adjacent classes with only one bit according to the change of eight LBP sampling points, and the neighborhood table is used in calculating the texture discrimination in the texture classifier.

5. The terrain real-time field robot recognition method based on the hierarchical terrain as claimed in claim 3, wherein in order to reduce the gray sensitivity of the terrain sample image to be detected, a unidirectional dead zone is introduced into the LBP feature with unchanged rotation and symmetry, and the unidirectional dead zone introduces an offset to the judgment of LBP classification in a positive direction or a negative direction.

6. The terrain recognition method for a real-time field robot based on hierarchical terrain as claimed in claim 1, wherein the terrain recognition classifier divides the terrain in the terrain library into two layers, the first layer is composed of three scene elements of road, plant and sky; the concrete pavement, the sand and the muddy ground form a second layer structure of road elements, and the trees, shrubs and the grassland form a second layer structure of plant elements;

setting a dynamic variable (x), wherein x is the reliability of the RGB color histogram characteristic for judging that the terrain sample to be detected is the closest terrain of a certain class through a first-layer structure classifier, and the value of (x) is between 0 and 1 and is a weighted item of the color characteristic;

when the terrain category can be accurately identified by the RGB color histogram feature, (x) is closer to 1; otherwise, the closer (x) is to 0, the judgment result of the terrain category is determined by the second-layer structure classifier according to the texture features.

7. The terrain recognition method of the real-time field robot based on the hierarchical terrain as claimed in claim 1, wherein the terrain recognition classifier calculates the difference between the terrain sample to be tested and the picture in the training data set by using a cross comparison method for the RGB color histogram feature and the variance curve feature of the input image and the data set sample; the specific method comprises the following steps:

using two groups of data vectors, and sequentially arranging P-dimension data vectors in respective histograms to form two groups of histogram data;

the first group of histogram data is the color characteristics of the terrain sample to be detected; the second group of histogram data is the color features of the pictures in the training dataset;

setting a current comparison vector i in the first group of histogram data, sequentially selecting 1-P dimension data vectors from the i, weighting and performing difference modulo with data which takes the i as a center and has a neighborhood width of k in the second histogram, and accumulating the obtained modulo to determine the difference degree d of two groups of discrete data;

the larger the obtained difference degree is, the more obvious the type difference between the two sets of multi-dimensional discrete data is, and the smaller the difference degree is, the smaller the type difference between the two sets of multi-dimensional discrete data is.

8. A computer-readable storage medium having stored thereon a plurality of instructions adapted to be loaded by a processor of a terminal device and to perform the method for terrain-based real-time field robotic terrain recognition of any of claims 1-7.

9. A terminal device comprising a processor and a computer-readable storage medium, the processor being configured to implement instructions; a computer readable storage medium for storing a plurality of instructions adapted to be loaded by a processor and to perform the method for terrain-based real-time field robotic terrain recognition of any of claims 1-7.