CN107527031A - A kind of indoor objects detection method based on SSD - Google Patents

Abstract

The invention discloses an SSD-based indoor target detection method, belongs to the field of target recognition, and is a novel application method of a convolutional neural network for indoor target detection. The present invention solves indoor target detection. However, there is currently no marked database that meets the requirements of this experiment. For this reason, a database related to indoor targets is constructed. All images of the database are sampled at an angle with a wider field of view. It conforms to the normal viewing angle of intelligent robots, and has differences in background, lighting, and image size. Manually annotate common indoor targets such as refrigerators, TVs, beds, dining tables, chairs, sofas, coffee tables, toilets, washstands, bathtubs, cups, etc.; use the obtained images to train feature extraction networks and detectors, and finally use the trained features The extraction network and the detector detect the target to be recognized.

Description

An SSD-based indoor target detection method

technical field

The object recognition field of the present invention is a novel application method of a convolutional neural network for indoor object detection.

Background technique

Indoor object recognition is the process of separating and identifying one object from other objects in an indoor environment. It is of great significance for the research and development of household service robots, the design of home monitoring systems, and the retrieval of interior design schemes. However, the detection accuracy and speed of traditional methods cannot meet the requirements, so a more efficient and accurate detection method is urgently needed.

In the field of computer vision, there are many methods that can be used for indoor target detection. Traditional target detection methods are based on features such as SIFT and HOG, and classification methods such as SVM and Adaboot are used. However, these manually designed features often have limited ability to represent images, resulting in low target recognition accuracy and positioning accuracy, which is difficult to meet the requirements of practical applications. As a new branch of machine learning, deep learning has improved in real-time and accuracy. In recent years, deep learning has developed rapidly and has shown good performance in the fields of automatic speech recognition, natural language processing, and image processing. In the field of image processing, deep learning has been widely used in image classification, object detection and semantic segmentation. Convolutional neural network is the core of deep learning, and convolutional neural network is indispensable for the good performance of deep learning. In the field of target detection, models such as Faster R-CNN, YOLO, and SSD have good detection results, and convolutional neural networks play an irreplaceable role in them. At the same time, recurrent neural networks have also developed well in recent years. Recurrent convolutional networks have unique advantages over convolutional neural networks for processing time series. And models such as LSTM have greatly improved the gradient disappearance problem of circular convolutional networks.

For indoor target detection, there are some relatively good solutions, such as: target detection based on Faster R-CNN, using a shared convolutional network to form an RPN (Region Proposal Network), using RPN to directly predict the suggestion box, and the RPN Most of the predictions are completed in the GPU, and the convolutional network and Fast R-CNN are partially shared, so the speed of target detection is greatly improved, but it still cannot meet the real-time requirements. Based on the target detection of YOLO (You Only Look Once), the position of the target can be directly returned. Since the preset frame mechanism is not used, the detection accuracy of YOLO is not very high. In addition, there is a detection method based on SSD (Single Shot Multibox Detector), which also directly regresses the position and category of the target. The detection accuracy is improved compared to YOLO, but it still cannot meet the task requirements.

Neural networks have millions of parameters and are prone to overfitting problems. In order to solve the problem of overfitting, it is generally used to pre-train a model on a large-scale data set, and then fine-tune the model with a specific small-scale data set. At the same time, the dropout method can be used to prevent some hidden layer nodes from working during training to prevent overfitting.

Contents of the invention

The present invention proposes an improved SSD-based indoor target detection, aiming at the application scenarios of home monitoring and service robots. On the established database, extensive experiments are carried out to detect common indoor furniture.

The present invention solves the problem of indoor target detection. However, there is currently no marked database that meets the requirements of this experiment. For this reason, a database related to indoor targets is constructed. All images of the database are sampled at an angle with a wider field of view. It conforms to the normal viewing angle of intelligent robots, and has differences in background, lighting, and image size. Manually mark the common indoor targets refrigerator, TV, bed, dining table, chair, sofa, coffee table, toilet, washstand, bathtub, cup, etc.; use the obtained image to train the feature extraction network and detector, and finally use the trained feature The extraction network and the detector detect the target to be recognized. Therefore, the technical solution of the present invention is an SSD-based indoor target detection method, the method comprising:

Step 1: obtain the indoor target image to be detected;

Step 2: Establish a feature extraction network, and use the feature extraction network to extract the global features of the target image;

Step 3: Input the global features obtained in step 2 into the SSD detector to obtain the corresponding detection results;

It is characterized in that the feature extraction network in step 2 includes: three input modules, the first to eleventh convolution modules, the first to fifth pooling modules, two context information extraction modules, and one normalization module; The three input modules are respectively an image to be detected and the first and second flag information input modules, and the image to be detected is used as the input of the first convolution module; the first convolution module, the first pooling module, The second convolution module, the second pooling module, the third convolution module, the third pooling module, the fourth convolution module, the fourth pooling module, the fifth convolution module, the fifth pooling module, the sixth The convolution module, the seventh convolution module, the eighth convolution module, the ninth convolution module, the tenth convolution module, and the eleventh convolution module are cascaded in sequence; additionally, the output of the fourth convolution module also needs Together with the output of the first flag bit information input module, it is input to the normalization module, and then the output of the normalization module is input to the first context information extraction module; additionally, the output of the seventh convolution module is also combined with the second flag The output of the bit information input module is input to the second context information extraction module together; finally, the outputs of the first and second context information extraction modules, and the eighth to eleventh convolution modules are used as the extracted global features.

Further, the context information extraction module includes two convolution modules, a cascade module, a horizontal feature extraction branch and a vertical feature extraction branch; the output of the first convolution module is respectively input to the horizontal feature extraction A branch and a vertical feature extraction branch, the output of the horizontal feature extraction branch and the vertical feature extraction branch are jointly input to the cascade module, and then the output of the cascade module is input to the second convolution module; wherein the cascade module as a cascade layer. The scheme extracts context information from both horizontal and vertical directions, effectively extracting the overall information of the image.

Further, the two convolution modules in the context information extraction module are both convolution layers with a convolution kernel size of 1*1, a step size of 1, and a padding of 0. This scheme can effectively extract image features and help prevent network divergence.

Further, both the horizontal feature branch and the vertical feature branch are composed of a dimension conversion module, a dimension merging module, a circular convolution module, and a dimension restoration module in sequence; the dimension conversion module The function is to exchange the position of each dimension information of the input image, the function of the dimension merging module is to reduce the dimension quantity of the image, and the function of the dimension restoration module is to restore the reduced image dimension number to its original dimension number and order; The circular convolution module is an LSTM layer (long short-term memory network), and its processing direction is controlled by the dimension conversion module of the corresponding branch. This scheme makes full use of the ability of the circular convolutional network to process time series, flexible application and feature extraction of a single image, and LSTM has a good suppression effect on gradient dispersion.

Further, the dimension of the input image is n*c*w*h, where n represents the number of images, c represents the number of channels, w represents the width of the image, and h represents the height of the image; the dimension conversion in the horizontal feature branch The module changes the dimension order into h*n*w*c, and the dimension conversion module in the longitudinal feature branch changes the dimension order into w*n*h*c; the dimension merging modules of the two branches are both Merges the second and third dimensions into one dimension. The main function of this scheme is to control the direction of image feature extraction by LSTM in the circular convolution module. The horizontal feature extraction branch is to input each line of the image as a time series into the circular convolution module, and the vertical feature extraction branch is to input each line of the image Input as a time series to the recurrent convolution module.

Further, the first and second convolutional modules are composed of two convolutional layers with a kernel size of 3*3, a step size of 1, and a padding of 1; the third to fifth convolutional modules are composed of three The first convolution kernel size is 3*3, the step size is 1, and the convolution layer is filled with 1; the sixth convolution module is composed of a convolution kernel size of 3*3, the step size is 1, and the padding is 6. Convolution layer; the seventh convolution module is composed of a convolution kernel with a size of 1*1, a step size of 1, and a padding of 1; the eighth and ninth convolution modules are composed of a convolution kernel A convolutional layer with a size of 1*1, a stride of 1, and a padding of 0 and a convolutional kernel with a size of 3*3, a stride of 2, and a padding of 1; the eighth and ninth volumes The product module consists of a convolutional layer with a kernel size of 1*1, a stride of 1, and a padding of 0, and a convolutional layer with a kernel size of 3*3, a stride of 1, and a padding of 0. of. This scheme fully extracts the features of the image, which is convenient for the subsequent detector to detect the target effectively.

Based on the existing detection method, the present invention improves the extracted feature part, and flexibly applies the circular convolution network to add context information to the feature spectrum. In the existing methods, considering both accuracy and speed, SSD is selected as the detection scheme. SSD considers multiple signatures, selects the modules added by appropriate signatures, improves the detection accuracy while ensuring the speed, and better meets the actual needs.

Description of drawings

Fig. 1 is the overall network structure of the present invention;

Fig. 2 is the context information extraction module of the present invention;

Figure 3 is a part of the test results.

detailed description

The main work of the present invention is divided into two parts of training and testing, and all work is divided into six steps:

Step 1. Build the database: build an indoor database for the problem to be studied. The image is selected from the interior design website, with a wide viewing angle, and the relative relationship between common indoor objects can be clearly reflected. After excluding some inappropriate images, a total of more than 6,000 images are collected, two thirds of which are used as training samples and one third are used as testing samples. Due to the small number of samples, in order to avoid overfitting, operations such as random cutting and mirroring were performed on the image during training to increase the number of samples.

Step 2. Detection of sample targets: manually label all the images in the database, mark the targets in the images with its ground truth, that is, mark the position and category of the target, and mark the categories as 0, 1, 2.. ..9, a total of ten categories.

Step 3. Improve the SSD model: For indoor objects, there is a close relationship between objects and objects, and between objects and the environment. They are placed according to the common habits of human beings. Therefore, for one of the characteristics of indoor objects, the SSD model is improved. Global information is added to the detection of specific objects. Use the feature extraction network to extract image features, use LSTM's ability to process time series, build a model, divide the feature spectrum of image information into rows and columns, and then input it into the circular convolution network for processing, and then process the information in parallel And convolution operation, the feature spectrum of row processing and column processing is processed into a feature spectrum, and then input into the detection network of SSD for detection. Because the global information of high-level features is seriously lost, and the amount of feature data of too low-level is too large, and there is a lot of redundant information, so the present invention adds the context information extraction module to the feature spectrum output by the fourth and seventh convolution modules.

Step 4. Pre-training model: Because the network model is large, with many parameters and few samples, in order to prevent over-fitting, the model is first trained on ImageNet, a large database, to obtain a pre-training model.

Step 5. Train the improved model: On the basis of step 4, use your own database to continue training the model. Preprocess the data before training to expand the data samples. Train to get the final network model.

Step 6. Test model: Test the correct training model on PASCAL VOC207 and its own database to obtain the position and category of the detected image. Test results Compared with the SSD, the accuracy of the present invention for indoor target detection has been significantly improved, and the detection can still be completed at a faster speed, which satisfies the needs of users.

Through the above six steps, the SSD-based indoor target detection method is improved, which makes full use of the image context information, effectively improves the network's target detection ability for indoor scenes, and ensures the detection speed of the network. Some test results are shown in Figure 3.

Claims (6)

1. An SSD-based indoor target detection method, the method comprising: Step 1: obtain the indoor target image to be detected; Step 2: Establish a feature extraction network, and use the feature extraction network to extract the global features of the target image; Step 3: Input the global features obtained in step 2 into the SSD detector to obtain the corresponding detection results; It is characterized in that the feature extraction network in step 2 includes: three input modules, the first to eleventh convolution modules, the first to fifth pooling modules, two context information extraction modules, and one normalization module; The three input modules are respectively an image to be detected and the first and second flag information input modules, and the image to be detected is used as the input of the first convolution module; the first convolution module, the first pooling module, The second convolution module, the second pooling module, the third convolution module, the third pooling module, the fourth convolution module, the fourth pooling module, the fifth convolution module, the fifth pooling module, the sixth The convolution module, the seventh convolution module, the eighth convolution module, the ninth convolution module, the tenth convolution module, and the eleventh convolution module are cascaded in sequence; additionally, the output of the fourth convolution module also needs Together with the output of the first flag bit information input module, it is input to the normalization module, and then the output of the normalization module is input to the first context information extraction module; additionally, the output of the seventh convolution module is also combined with the second flag The output of the bit information input module is input to the second context information extraction module together; finally, the outputs of the first and second context information extraction modules, and the eighth to eleventh convolution modules are used as the extracted global features. 2. The SSD-based indoor object detection method according to claim 1, wherein the context information extraction module includes two convolution modules, a cascade module, a horizontal feature extraction branch and a vertical feature extraction branch. Extraction branch; the output of the first convolution module is input to the horizontal feature extraction branch and the vertical feature extraction branch respectively, and the output of the horizontal feature extraction branch and the vertical feature extraction branch are jointly input to the cascade module, and then The output of the cascade module is input to the second convolution module; where the cascade module is a cascade layer. 3. A kind of SSD-based indoor object detection method as claimed in claim 1 or 2, it is characterized in that the first and second convolution modules are composed of two convolution kernels with a size of 3*3 and a step size of is 1, the convolution layer is composed of 1; the third to fifth convolution modules are composed of three convolution kernels with a size of 3*3, a step size of 1, and a convolution layer of 1; the sixth volume The product module is composed of a convolution kernel with a size of 3*3, a step size of 1, and a convolution layer with a padding of 6; the seventh convolution module is composed of a convolution kernel with a size of 1*1 and a step size of 1. The convolution layer is filled with 1; the eighth and ninth convolution modules are composed of a convolution kernel with a size of 1*1, a step size of 1, a convolution layer with a padding of 0, and a convolution kernel with a size of 3* 3. The step size is 2, and the convolution layer is filled with 1; the eighth and ninth convolution modules are composed of a convolution kernel with a size of 1*1, a step size of 1, and a convolution layer with a padding of 0. A convolution kernel size is 3*3, the step size is 1, and the convolution layer is filled with 0. 4. A kind of SSD-based indoor object detection method as claimed in claim 2, it is characterized in that the two convolution modules in the context information extraction module all have a convolution kernel size of 1*1 and a step size of 1 , a convolutional layer filled with 0. 5. The SSD-based indoor object detection method according to claim 2, characterized in that the horizontal feature branch and the vertical feature branch are composed of a dimension transformation module, a dimension merging module, and a loop volume A product module and a dimension reduction module are sequentially cascaded; the function of the dimension conversion module is to exchange the position of each dimension information of the input image, the function of the dimension merging module is to reduce the dimension quantity of the image, and the function of the dimension reduction module In order to restore the reduced image dimensionality to its original dimensionality and order; the circular convolution module is an LSTM layer, and its processing direction is controlled by the dimension conversion module of the corresponding branch. 6. The SSD-based indoor target detection method according to claim 5, wherein the dimension of the input image is n*c*w*h, wherein n represents the number of images, c represents the number of channels, and w Indicates the width of the picture, and h indicates the height of the image; the dimension conversion module in the horizontal feature branch changes the dimension order to h*n*w*c, and the dimension conversion module in the vertical feature branch changes the dimension order to w *n*h*c; the dimension merging modules of the two branches both merge the second and third dimensions into one dimension.