CN110414561A - A Construction Method of Natural Scene Dataset Suitable for Machine Vision - Google Patents

Abstract

The invention discloses a method for constructing a natural scene data set suitable for machine vision, comprising the following steps: determining the classification category of the data set based on the ecosystem type; downloading the original image from the Internet by using keywords; performing preliminary Matching judgment, classify the fuzzy category pictures again to form an image data set; use the convolutional neural network to verify the data set, and then merge the ambiguous categories according to the confusion matrix. It solves the problem that natural scene classification is difficult to define accurately and objectively like object classification, and provides reliable data support for training deep convolutional neural networks, which can be used in natural scene recognition systems.

Description

A Construction Method of Natural Scene Dataset Suitable for Machine Vision

technical field

The invention relates to the technical field of image processing, in particular to a method for constructing a natural scene data set suitable for machine vision.

Background technique

In the field of image processing technology, image recognition has received more and more attention in recent years. Image recognition technology is the basis of practical technologies such as motion analysis, stereo vision, and data fusion. Among them, target recognition is due to convolutional neural networks and large-scale data. The application of the set has greatly improved the recognition accuracy, but it is difficult to obtain the same level of accuracy by using the convolutional neural network to recognize the scene where the image occurs. A large part of the reason is that the scene data set caused by inaccurate self-categorization.

Using machine vision to identify objects in an image can provide an overview of what is happening in the image, and identifying the scene where the image occurs allows us to obtain more information related to the image, and at the same time guess what may happen in the past and the future through the scene what event. With the development of autonomous driving, unmanned vehicles, and unmanned aerial vehicles, accurate detection of the surrounding environment provides a guarantee for safety, so a standard scene database is particularly important.

For most scene databases, there are often the following two problems: 1) The hierarchy between categories is confusing, and there is confusion between the categories of "farmland" and "plain" in some datasets, where plain is a landform type , and farmland can also be established on plains, and the two are not independent; 2) The meaning is repeated and confused, for example, "forest" and "forest" are only distinguished under special circumstances.

Different from the objective evaluation of category labels in the construction process of object data sets, scene data sets often have different judgment standards for the same image due to different subjective evaluations, which also leads to great differences between images in the same category. This limits the improvement of algorithm accuracy, so a unified classification criterion needs to be established in the scene dataset.

For a natural scene, organisms and the environment help us determine the specific type of the scene, and the unified whole composed of organisms and the environment is called an ecosystem in ecological research, and these ecosystems have the most diverse types and sizes. The classification of the natural scene data set corresponds to the classification of the natural scene data set, so that the classification of the data set meets a unified judgment scale.

To determine the natural scene to which an image belongs, it is necessary to combine the objects and background in the image; to determine the ecosystem to which a certain space belongs, it is necessary to combine the organisms and the environment in the space; the two are very similar in determination, so The invention proposes to use the ecosystem as the judgment scale of the data set category, that is, the category in the ecosystem corresponds to the category in the natural scene data set, so that the operation is basically the same in terms of the subjective feeling of classification and the objective requirements.

Contents of the invention

In order to solve the above problems, the present invention provides a method for constructing a natural scene database, which aims to improve the defects in the existing data set classification, so that there is a unified classification criterion when constructing the natural scene data set classification, and the final data The set can be better applied to the field of machine vision.

To achieve the above object, the technical solution of the present invention specifically includes the following steps:

1. A method for building a natural scene dataset suitable for machine vision, comprising the following steps:

Step 1: Preliminarily determine the categories and corresponding subcategories contained in the data set according to the ecosystem type; among them, the ecosystem types are divided into:

Terrestrial ecosystems: including natural terrestrial ecosystems and artificial terrestrial ecosystems;

Aquatic ecosystems: including inland aquatic ecosystems, marine ecosystems and artificial aquatic ecosystems;

Underground ecosystems: including cave ecosystems;

Step 2: Use the subcategory name as a keyword to search on the Internet, search at least 1000 pictures corresponding to the subcategory, and complete the collection of pictures;

Step 3: Preliminary manual screening of the collected pictures, and further screening of pictures that do not meet the corresponding category, to obtain a data set containing scene category labels;

Step 4: Divide the data set into a training set and a validation set, use a deep convolutional neural network to classify images, and use the confusion matrix to combine the prediction results with the real category labels to generate a similarity matrix, and combine the similarity degrees according to the similarity matrix High category, get the final data set, including:

Step 4.1, randomly divide the original data set according to the ratio of 80%/20% to obtain a training set and a verification set;

Step 4.2, convert the pictures in the training set into 224×224 resolution pictures, and select the 13-layer bn-inception network structure as the scene prediction classification network; the original bn- The inception network converts a 224×224 resolution image into a 28×28 feature map, which enables faster processing in the subsequent 10 inception layers, of which the step size of two layers is 2, and the remaining step size is 1, so 28 The feature map of ×28 is finally changed to a feature map of 7×7; finally, the global mean pooling layer is used to aggregate the activations across spatial dimensions;

In step 4.3, use the model obtained in step 4.2 to predict the classification of the verification set to obtain the classification result, and combine the prediction result with the real classification to obtain the confusion matrix C, which shows the crossover error between categories, implicitly Indicates the degree of similarity between them, formally we define this similarity in the following way:

S=C+C ^T

where C∈R ^N×N is the confusion matrix, N is the number of natural scene categories, C _ij represents the probability of classifying the i-th class into the j-th class, and a larger value indicates a higher ambiguity between the two classes; According to this formula, the similarity matrix S is generated. This equation ensures that the similarity is a symmetric measure and provides clues for merging two similar categories;

Step 4.4, set an appropriate threshold τ, where τ=0.5, merge the two categories whose similarity exceeds the threshold τ; analyze the rationality of the merger, if the i-th row and the j-th row are merged into the same type, update The corresponding similarity matrix, delete i and j rows and columns, will be Add it as a new row and column, and repeat this step until the final data set does not exceed the threshold or meet other requirements, so that the obtained data set is more suitable for machine vision processing.

2. a kind of construction method that is applicable to the natural scene data set of machine vision according to claim 1, is characterized in that, described ecosystem subcategory classification will satisfy:

Condition 1. The subcategory belongs to an ecosystem;

The second condition is that the subcategories are independent of each other.

3. a kind of construction method applicable to the natural scene dataset of machine vision according to claim 1, is characterized in that, subcategory is divided into:

Natural terrestrial ecosystems: polar and alpine tundra, semi-desert, coniferous forest, deciduous forest, natural grassland, wasteland, evergreen broad-leaved forest, Mediterranean shrub, desert, mountain, tropical rainforest;

Artificial terrestrial ecosystem: artificial grassland, farmland, greenhouse, city;

Inland aquatic ecosystems: swamps, wetlands, rivers, lakes;

Marine ecosystems: coastal, coral reef, deep sea;

Artificial aquatic ecosystems: ponds;

Contains cave ecosystems: Caves.

4. A method for constructing a natural scene data set suitable for machine vision according to claim 1, wherein the category name in step 2 is added with weather adjectives, and the weather adjectives include: sunny, cloudy, under rainy, snowy.

5. A method for constructing a natural scene data set suitable for machine vision according to claim 1, characterized in that, the pictures that do not meet the corresponding categories in step 3 are picked out and summarized into unclassified pictures, if the total collection of pictures There are M pictures, and the total category is N categories. When the unclassified pictures exceed N/M, a new category is constructed, and the new category meets the subcategory classification conditions described in step 1.

Therefore, the present invention has the following advantages: (1) the picture samples in the same subcategory in the data set constructed according to this method are diversified, and the overall quality of the constructed data set is high; (2) different subcategories are not easily confused, which can improve the test precision.

Description of drawings

Fig. 1 is the overall block diagram of the present invention.

Figure 2 is a 13-layer bn-inception network structure diagram used in the example.

Detailed ways

In order to more clearly illustrate the purpose, technical solutions and beneficial effects of the present invention, further description will be given below in conjunction with the accompanying drawings and embodiments. It should be understood that the present invention should not be limited to the content disclosed in the embodiments, and the protection scope of the present invention is subject to the scope defined in the claims.

As shown in Figure 1, the present invention discloses a method for constructing a natural scene data set suitable for machine vision, and the specific implementation steps are as follows:

Step 1: Initially determine the categories contained in the dataset according to the ecosystem type:

In step 1.1, ecosystem types can be divided into three basic categories: terrestrial ecosystems, aquatic ecosystems, and underground ecosystems;

Step 1.2, each category can be further classified, terrestrial ecosystems can be further divided into natural terrestrial ecosystems and artificial terrestrial ecosystems, aquatic ecosystems can be divided into inland aquatic ecosystems, marine ecosystems and artificial aquatic ecosystems, Underground ecosystems include cave ecosystems;

In step 1.3, the above categories can be further divided into subcategories, wherein the classification of subcategories can be formulated according to the number of required categories, and the classification meets two conditions: (1) the subcategory belongs to a certain ecosystem; (2) the subcategory belongs to independent of each other;

Preferably, said subcategories are divided into: coastal, polar and alpine tundra, swamp, semi-desert, coniferous forest, deciduous forest, natural grassland, wasteland, evergreen broad-leaved forest, Mediterranean-type scrub, desert, mountain, Tropical rainforest, wetland, artificial grass, farmland, greenhouse, river, lake, pond, coral reef, deep sea, city, cave.

Step 2: Collect corresponding pictures and complete the collection of pictures:

For one of the categories C, enter C as a keyword into the search engine, and use the online image search engine to download pictures. In order to increase the variety of pictures, add weather-related words (such as sunny, rainy, cloudy) to the keywords. Exclude pictures that are too small in size and complete the download of all categories of pictures.

Step 3: Filter the collected pictures:

Step 3.1, manually screen the downloaded pictures, keep the pictures that match the categories, and pick out the pictures that do not match the categories to uncategorized pictures;

Step 3.2: Classify the remaining unclassified pictures. If a large number of pictures cannot find a suitable classification, consider building a new category. The new category meets the subcategory classification conditions described in step 1;

In step 3.3, a small number of images that cannot judge the scene are eliminated, and the category confirmation of all images is completed.

Step 4: Use the similarity matrix to merge similar types:

Step 4.1, randomly divide the original data set according to the ratio of 80%/20% to obtain a training set and a verification set;

In step 4.2, the pictures in the training set are converted into pictures with a resolution of 224×224. As shown in Figure 2, a 13-layer bn-inception network structure is selected as the scene prediction and classification network. The original bn-inception network with two convolutional layers and a 3×3 pooling layer converts a 224×224 resolution image into a 28×28 feature map, and the small size of the feature map enables the following 10 inception layers to be able to Faster processing, where the two layers have a step size of 2 and the rest of the step size is 1, so the 28×28 feature map is finally changed to a 7×7 feature map. Finally, the global mean pooling layer is used to aggregate the activations across spatial dimensions;

In step 4.3, use the model obtained in step 4.2 to predict the classification of the verification set to obtain the classification result, and combine the prediction result with the real classification to obtain the confusion matrix C, which shows the crossover error between categories, implicitly Indicates the degree of similarity between them, formally we define this similarity in the following way:

S=C+C ^T

Where C∈R ^N×N is the confusion matrix, N is the number of natural scene categories, C _ij represents the probability of classifying the i-th class into the j-th class, and a larger value indicates a higher degree of ambiguity between the two classes. According to this formula, the similarity matrix S is generated. This equation ensures that the similarity is a symmetric measure and provides clues for merging two similar categories;

In step 4.4, set an appropriate threshold τ, where τ=0.5, and merge the two categories whose similarity exceeds the threshold τ. Analyze the rationality of merging, if the i-th row and the j-th row are merged into the same type, update the corresponding similarity matrix, delete the i-th and j-th row and column, and Add it as a new row and column, and repeat this step until the final data set does not exceed the threshold or meet other requirements, so that the obtained data set is more suitable for machine vision processing.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which the present invention belongs can make various modifications or supplements to the described specific embodiments or adopt similar methods to replace them, but they will not deviate from the spirit of the present invention or go beyond the definition of the appended claims range.

Claims (5)

1. A method for building a natural scene dataset suitable for machine vision, comprising the following steps: Step 1: Preliminarily determine the categories and corresponding subcategories contained in the data set according to the ecosystem type; among them, the ecosystem types are divided into: Terrestrial ecosystems: including natural terrestrial ecosystems and artificial terrestrial ecosystems; Aquatic ecosystems: including inland aquatic ecosystems, marine ecosystems and artificial aquatic ecosystems; Underground ecosystems: including cave ecosystems; Step 2: Use the subcategory name as a keyword to search on the Internet, search at least 1000 pictures corresponding to the subcategory, and complete the collection of pictures; Step 3: Preliminary manual screening of the collected pictures, and further screening of pictures that do not meet the corresponding category, to obtain a data set containing scene category labels; Step 4: Divide the data set into a training set and a validation set, use a deep convolutional neural network to classify images, and use the confusion matrix to combine the prediction results with the real category labels to generate a similarity matrix, and combine the similarity degrees according to the similarity matrix High category, get the final data set, including: Step 4.1, randomly divide the original data set according to the ratio of 80%/20% to obtain a training set and a verification set; Step 4.2, convert the pictures in the training set into 224×224 resolution pictures, and select the 13-layer bn-inception network structure as the scene prediction classification network; the original bn- The inception network converts a 224×224 resolution image into a 28×28 feature map, which enables faster processing in the subsequent 10 inception layers, of which the step size of two layers is 2, and the remaining step size is 1, so 28 The feature map of ×28 is finally changed to a feature map of 7×7; finally, the global mean pooling layer is used to aggregate the activations across spatial dimensions; In step 4.3, use the model obtained in step 4.2 to predict the classification of the verification set to obtain the classification result, and combine the prediction result with the real classification to obtain the confusion matrix C, which shows the crossover error between categories, implicitly Indicates the degree of similarity between them, formally we define this similarity in the following way: S=C+C ^T where C∈R ^N×N is the confusion matrix, N is the number of natural scene categories, C _ij represents the probability of classifying the i-th class into the j-th class, and a larger value indicates a higher ambiguity between the two classes; According to this formula, the similarity matrix S is generated. This equation ensures that the similarity is a symmetric measure and provides clues for merging two similar categories; Step 4.4, set an appropriate threshold τ, where τ=0.5, merge the two categories whose similarity exceeds the threshold τ; analyze the rationality of the merger, if the i-th row and the j-th row are merged into the same type, update The corresponding similarity matrix, delete i and j rows and columns, will be Add it as a new row and column, and repeat this step until the final data set does not exceed the threshold or meet other requirements, so that the obtained data set is more suitable for machine vision processing. 2. a kind of construction method that is applicable to the natural scene data set of machine vision according to claim 1, is characterized in that, described ecosystem subcategory classification will satisfy: Condition 1. The subcategory belongs to an ecosystem; The second condition is that the subcategories are independent of each other. 3. a kind of construction method applicable to the natural scene dataset of machine vision according to claim 1, is characterized in that, subcategory is divided into: Natural terrestrial ecosystems: polar and alpine tundra, semi-desert, coniferous forest, deciduous forest, natural grassland, wasteland, evergreen broad-leaved forest, Mediterranean shrub, desert, mountain, tropical rainforest; Artificial terrestrial ecosystem: artificial grassland, farmland, greenhouse, city; Inland aquatic ecosystems: swamps, wetlands, rivers, lakes; Marine ecosystems: coastal, coral reef, deep sea; Artificial aquatic ecosystems: ponds; Contains cave ecosystems: Caves. 4. A method for constructing a natural scene data set suitable for machine vision according to claim 1, wherein the category name in step 2 is added with weather adjectives, and the weather adjectives include: sunny, cloudy, under rainy, snowy. 5. A method for constructing a natural scene data set suitable for machine vision according to claim 1, characterized in that, the pictures that do not meet the corresponding categories in step 3 are picked out and summarized into unclassified pictures, if the total collection of pictures There are M pictures, and the total category is N categories. When the unclassified pictures exceed N/M, a new category is constructed, and the new category meets the subcategory classification conditions described in step 1.