CN111611906A - Obstacle detection method, system and medium for automatic parking - Google Patents

Obstacle detection method, system and medium for automatic parking Download PDF

Info

Publication number
CN111611906A
CN111611906A CN202010419317.4A CN202010419317A CN111611906A CN 111611906 A CN111611906 A CN 111611906A CN 202010419317 A CN202010419317 A CN 202010419317A CN 111611906 A CN111611906 A CN 111611906A
Authority
CN
China
Prior art keywords
layer
convolutional layer
output
convolutional
pooling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010419317.4A
Other languages
Chinese (zh)
Inventor
路二伟
杨波
钱磊
张会朋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Yinwo Automotive Technology Co ltd
Original Assignee
Beijing Yinwo Automotive Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Yinwo Automotive Technology Co ltd filed Critical Beijing Yinwo Automotive Technology Co ltd
Priority to CN202010419317.4A priority Critical patent/CN111611906A/en
Publication of CN111611906A publication Critical patent/CN111611906A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/86—Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/88—Sonar systems specially adapted for specific applications
    • G01S15/93—Sonar systems specially adapted for specific applications for anti-collision purposes
    • G01S15/931—Sonar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/88—Sonar systems specially adapted for specific applications
    • G01S15/93—Sonar systems specially adapted for specific applications for anti-collision purposes
    • G01S15/931—Sonar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2015/932—Sonar systems specially adapted for specific applications for anti-collision purposes of land vehicles for parking operations

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a method, a system and a medium for detecting obstacles for automatic parking, which can give consideration to the detection precision of large and small obstacles and have high detection speed, and the method comprises the following steps: step 1: acquiring images through a camera, detecting the obstacles based on visual detection, and obtaining obstacle information; step 2: reading the obstacle information measured by the ultrasonic radar; and step 3: and finally determining the position of the obstacle by combining the obstacle information acquired by the camera vision detection and the ultrasonic radar measurement.

Description

Obstacle detection method, system and medium for automatic parking
Technical Field
The invention relates to the field of obstacle detection, in particular to an obstacle detection method, system and medium for automatic parking.
Background
With the increasing economic prosperity of China, the living standard of people is continuously improved, and vehicles are more and more. The urban space seems to be narrow, parking spaces and parking spaces are common problems in life of people, and the automatic parking technology is suitable for transportation. In the automatic parking technology, the identification of obstacles is a key technical link. The system can be helped to judge the effectiveness of the parking space, and the collision between the vehicle and the barrier is prevented. The existing obstacle detection method has insufficient real-time performance or cannot give consideration to the precision of large and small obstacles.
Disclosure of Invention
In view of the above problems, the present invention provides an obstacle detection method, system and medium for automatic parking, which can achieve both detection accuracy of large and small obstacles and high detection speed.
The technical scheme is as follows: an obstacle detection method for automatic parking, characterized by comprising the steps of:
step 1: acquiring images through a camera, detecting the obstacles based on visual detection, and obtaining obstacle information;
step 2: reading the obstacle information measured by the ultrasonic radar;
and step 3: and finally determining the position of the obstacle by combining the obstacle information acquired by the camera vision detection and the ultrasonic radar measurement.
Further, step 1 specifically includes the following steps:
step 101: constructing a data set, wherein the data set comprises picture data acquired according to a required scene and a label file of an obstacle;
step 102; establishing an SSD network model, and training the SSD network model by using a data set to obtain a trained SSD network model;
step 103: pruning and quantifying the trained SSD network model;
step 104: calibrating the camera to obtain the relation between an image coordinate system and a world coordinate system;
step 105: inputting the pictures acquired by the camera into a trained SSD network model, outputting the detected obstacles in the pictures, and marking the obstacles by a minimum circumscribed rectangle;
step 106: and converting the obtained minimum circumscribed rectangle of the obstacle into a world coordinate system according to the relation between the image coordinate system and the world coordinate system, and obtaining the obstacle information in the world coordinate system.
Further, in step 3, the information of the obstacle obtained by the visual detection of the camera is matched with the information of the obstacle obtained by the measurement of the ultrasonic radar, and the information of the obstacle with the same information after matching is output as a final detection result by using the information of the obstacle obtained by the measurement of the ultrasonic radar; and for the obstacle information with inconsistent information after matching, outputting the information of the obstacle acquired through the visual detection of the camera as a final detection result.
Further, in step 101, the label file includes the category and position information of the car, the pedestrian and the ice cream barrel in the picture.
Further, in step 101, the data set is divided into a test set, a training set and a verification set according to a certain proportion, the test set is used for training, the test set is used for testing whether the detection of the SSD network model is valid, and the verification set is used for monitoring the training effect of the SSD network model.
Further, in step 102, the SSD network model is built by:
an input layer for inputting an image;
the convolutional layer conv1a is characterized in that an input image outputs a feature map through convolution operation, the number of channels is 16, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv1a is connected with the convolutional layer conv1 b;
the convolutional layer conv1b is used for carrying out convolution operation on the output of the convolutional layer conv1a, and outputting a characteristic diagram, wherein the number of channels is 16, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv1b is connected with a Pooling layer Pooling 1;
the Pooling layer Pooling1 is used for Pooling the output of the convolutional layer conv1b, and the Pooling layer Pooling1 is connected with the convolutional layer conv2 a;
the convolutional layer conv2a is used for performing convolution operation on the output of the Pooling1, the number of channels is 32, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv2a is connected with the convolutional layer conv2 b;
the convolutional layer conv2b is used for carrying out convolution operation on the output of the convolutional layer conv2a, outputting a characteristic diagram, wherein the number of channels is 32, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv2b is connected with the Pooling layer Pooling 2;
the Pooling layer Pooling2 is used for Pooling the output of the convolutional layer conv2b, and the Pooling layer Pooling2 is connected with the convolutional layer conv3 a;
the convolutional layer conv3a is used for carrying out convolution operation on the output of the Pooling2, the number of channels is 64, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv3a is connected with the convolutional layer conv3 b;
the convolutional layer conv3b is used for carrying out convolution operation on the output of the convolutional layer conv3a, outputting a characteristic diagram, wherein the number of channels is 64, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv3b is connected with the Pooling layer Pooling 3;
the Pooling layer Pooling3 is used for Pooling the output of the convolutional layer conv3b, and the Pooling layer Pooling3 is connected with the convolutional layer conv4 a;
the convolutional layer conv4a is used for carrying out convolution operation on the output of the Pooling3, the number of channels is 128, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv4a is connected with the convolutional layer conv4 b;
the convolutional layer conv4b is used for carrying out convolution operation on the output of the convolutional layer conv4a, outputting a characteristic diagram, wherein the number of channels is 256, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv4b is connected with a Pooling layer Pooling 4;
the Pooling layer Pooling4 is used for Pooling the output of the convolutional layer conv34, and the Pooling layer Pooling4 is connected with the convolutional layer conv5 a;
the convolutional layer conv5a is used for performing convolution operation on the output of the Pooling4, the number of channels is 512, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv5a is connected with the convolutional layer conv5 b;
the convolutional layer conv5b is used for carrying out convolution operation on the output of the convolutional layer conv5a, outputting a characteristic diagram, wherein the number of channels is 512, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv5b is connected with a Pooling layer Pooling 5;
the Pooling layer Pooling5 is used for Pooling the output of the convolutional layer conv5b, and the Pooling layer Pooling5 is connected with the Pooling layer Pooling 6;
the Pooling layer Pooling6 is used for Pooling the output of the Pooling layer Pooling5, and the Pooling layer Pooling6 is connected with the Pooling layer Pooling 67;
the convolution layer Output1 is used for performing convolution operation on the Output of the convolution layer conv4b, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolution layer Output1 is connected with the PriorBox layer PriorBox1, the convolution layer Loc1 and the convolution layer Conf 1;
the PriorBox layer PriorBox1 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output1, and the PriorBox layer PriorBox1 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc1 is used for performing convolution operation on the Output of the convolutional layer Output1, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc1 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf1 is used for performing convolution operation on the Output of the convolutional layer Output1, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf1 is connected with the fully-connected layer Mbox _ loc;
the convolution layer Output2 is used for performing convolution operation on the Output of the convolution layer conv5b, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolution layer Output2 is connected with the PriorBox layer PriorBox2, the convolution layer Loc2 and the convolution layer Conf 2;
the PriorBox layer PriorBox2 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output2, and the PriorBox layer PriorBox2 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc2 is used for performing convolution operation on the Output of the convolutional layer Output2, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc2 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf2 is used for performing convolution operation on the Output of the convolutional layer Output2, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf2 is connected with the fully-connected layer Mbox _ loc;
the convolutional layer Output3 is used for carrying out convolution operation on the Output of the Pooling layer 5, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Output3 is connected with the PriorBox layer PriorBox3, the convolutional layer Loc3 and the convolutional layer Conf 3;
the PriorBox layer PriorBox3 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output3, and the PriorBox layer PriorBox3 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc3 is used for performing convolution operation on the Output of the convolutional layer Output3, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc3 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf3 is used for performing convolution operation on the Output of the convolutional layer Output3, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf3 is connected with the fully-connected layer Mbox _ loc;
the convolutional layer Output4 is used for carrying out convolution operation on the Output of the Pooling layer 6, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Output4 is connected with the PriorBox layer PriorBox4, the convolutional layer Loc4 and the convolutional layer Conf 4;
the PriorBox layer PriorBox4 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output4, and the PriorBox layer PriorBox4 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc4 is used for performing convolution operation on the Output of the convolutional layer Output4, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc4 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf4 is used for performing convolution operation on the Output of the convolutional layer Output4, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf4 is connected with the fully-connected layer Mbox _ loc;
the convolutional layer Output5 is used for carrying out convolution operation on the Output of the Pooling layer 7, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Output5 is connected with the PriorBox layer PriorBox5, the convolutional layer Loc5 and the convolutional layer Conf 5;
the PriorBox layer PriorBox5 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output5, and the PriorBox layer PriorBox5 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc5 is used for performing convolution operation on the Output of the convolutional layer Output5, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc5 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf5 is used for performing convolution operation on the Output of the convolutional layer Output5, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf5 is connected with the fully-connected layer Mbox _ loc;
the fully-connected layer Mbox _ conf is respectively connected with the outputs of the convolutional layer Loc1, the convolutional layer Loc2, the convolutional layer Loc3, the convolutional layer Loc4 and the convolutional layer Loc5, and the output of the fully-connected layer Mbox _ conf is processed by a reshape layer, a softmax layer and a Flatten layer and is connected to an output layer DeprotectionOutput;
a full link layer Mbox _ loc connected to outputs of the convolutional layers Conf1, convolutional layers Conf2, convolutional layers Conf3, convolutional layers Conf4, and convolutional layers Conf5, respectively, and connected to the output layer detective output;
a full connection layer Mbox _ prior connected to the outputs of the PriorBox layer PriorBox1, the PriorBox layer PriorBox2, the PriorBox layer PriorBox3, the PriorBox layer PriorBox4, the PriorBox layer PriorBox5, respectively, to the output layer DetectionOutput;
and the output layer DetectionOutput is used for outputting the detection result of the SSD network model.
Further, in step 103, pruning the trained SSD network model includes the following steps:
step 103-1: adding a loss function of the trained SSD network model into L1 regularization constraint, and performing network training by taking the weight of the trained SSD network model as an initial value;
step 103-2: adding the loss function of the SSD network model trained in the previous step into L1 regularization constraint, and performing network training by taking the weight of the SSD network model trained in the previous step as an initial value;
step 103-3: the loop goes to step 103-2 for resetting the weight with the smaller weight absolute value to 0.
Further, in step 103, the trained weights are expressed as 32-bit floating point numbers, and the trained SSD network model is quantized to map the weights of the 32-bit floating point numbers to 8-bit integers in binary, i.e. 0 to 255, which are expressed by the following formula:
xquantized＝xfloat÷xscale+xzero_point
in the formula, xquantizedFor quantized values, xfloatThe arbitrary weight in the weights of one convolutional layer in the SSD network model is expressed as:
Figure BDA0002496334420000051
wherein the content of the first and second substances,
Figure BDA0002496334420000052
and
Figure BDA0002496334420000053
the minimum and maximum values of the weight of one convolution layer in the SSD network model are respectively shown, wherein x isscaleTo scale, it is expressed as:
Figure BDA0002496334420000054
in the formula, xzero_pointFor the number after quantization and before quantization with weight of 0, expressed as
Figure BDA0002496334420000061
Further, in step 104, when the collected image is calibrated, a homography matrix of the image coordinate system and the world coordinate system is obtained, and the image coordinate system and the world coordinate system are associated through the homography matrix, so that mutual conversion between coordinates of the associated image coordinate system and coordinates of the world coordinate system is realized.
Further, in step 3, regarding the obstacle obtained by the camera visual detection, the midpoint of the minimum bounding rectangle is taken as the position of the obstacle.
An obstacle detection system for automatic parking, characterized by comprising a memory, a processor, and a program stored on the memory and executable on the processor, the processor implementing the obstacle detection method for automatic parking as described above when executing the program.
A computer-readable storage medium on which a program is stored, characterized in that: the program implements the obstacle detection method for automatic parking as described above when executed by a processor.
The obstacle detection method for automatic parking, provided by the invention, is characterized in that the position of the obstacle is finally determined by combining the obstacle information obtained by the camera vision detection and the ultrasonic radar measurement, the target easy to detect by the radar is a large target, such as a vehicle, the radar is difficult to detect a small target, such as a person, a popsicle barrel or a smaller target, and the camera vision recognition result is used as the standard for the small target difficult to detect by the radar; for detected pedestrians and ice cream barrels, the middle point of the minimum circumscribed rectangle is used as the position of an obstacle, and the position of the obstacle detected by an ultrasonic radar is used as the standard for large targets such as vehicles and the like; in the camera visual detection method, in order to further improve the running speed of the network, the trained SSD network model is pruned and quantized, unimportant weights are removed through pruning, quantized integer weights are calculated through quantization operation, the occupied bandwidth is smaller, the calculation speed is higher, the running speed is improved by 1 time through pruning and quantization operation, the precision is reduced by about 1%, the calculation amount can be reduced while the precision is ensured, and thus the visual detection result can be quickly obtained;
the method can be used for avoiding obstacles, judging the effectiveness of the parking space and the like, and in the automatic parking process, if the parking space is detected, whether the inside of the parking space contains the obstacles can be judged by using the method, so that whether the parking space is available can be judged.
Drawings
Fig. 1 is a main flowchart of an obstacle detection method for automatic parking according to the present invention;
FIG. 2 is a schematic flow chart of step 1;
fig. 3 is a schematic diagram of SSD network model detection of obstacles.
Detailed Description
Referring to fig. 1, an obstacle detection method for automatic parking according to the present invention includes at least the steps of:
step 1: acquiring images through a camera, detecting the obstacles based on visual detection, and obtaining obstacle information;
step 2: reading the obstacle information measured by the ultrasonic radar;
and step 3: and finally determining the position of the obstacle by combining the obstacle information acquired by the camera vision detection and the ultrasonic radar measurement.
Referring to fig. 2, in this embodiment, step 1 includes the following steps:
step 101: and constructing a data set, wherein the data set comprises picture data acquired according to a required scene and a label file of the barrier, the label file comprises the category and position information of a vehicle, a pedestrian and an ice cream barrel in the picture, in addition, the data set is divided into a test set, a training set and a verification set according to a certain proportion, the test set is used for training, the test set is used for testing whether the detection of the SSD network model is effective, and the verification set is used for monitoring the training effect of the SSD network model.
Step 102; establishing an SSD network model, and training the SSD network model by using a data set to obtain the trained SSD network model, wherein the established SSD network model specifically comprises the following steps:
an input layer for inputting an image;
the convolutional layer conv1a is characterized in that an input image outputs a feature map through convolution operation, the number of channels is 16, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv1a is connected with the convolutional layer conv1 b;
the convolutional layer conv1b is used for carrying out convolution operation on the output of the convolutional layer conv1a, and outputting a characteristic diagram, wherein the number of channels is 16, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv1b is connected with a Pooling layer Pooling 1;
the Pooling layer Pooling1 is used for Pooling the output of the convolutional layer conv1b, and the Pooling layer Pooling1 is connected with the convolutional layer conv2 a;
the convolutional layer conv2a is used for performing convolution operation on the output of the Pooling1, the number of channels is 32, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv2a is connected with the convolutional layer conv2 b;
the convolutional layer conv2b is used for carrying out convolution operation on the output of the convolutional layer conv2a, outputting a characteristic diagram, wherein the number of channels is 32, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv2b is connected with the Pooling layer Pooling 2;
the Pooling layer Pooling2 is used for Pooling the output of the convolutional layer conv2b, and the Pooling layer Pooling2 is connected with the convolutional layer conv3 a;
the convolutional layer conv3a is used for carrying out convolution operation on the output of the Pooling2, the number of channels is 64, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv3a is connected with the convolutional layer conv3 b;
the convolutional layer conv3b is used for carrying out convolution operation on the output of the convolutional layer conv3a, outputting a characteristic diagram, wherein the number of channels is 64, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv3b is connected with the Pooling layer Pooling 3;
the Pooling layer Pooling3 is used for Pooling the output of the convolutional layer conv3b, and the Pooling layer Pooling3 is connected with the convolutional layer conv4 a;
the convolutional layer conv4a is used for carrying out convolution operation on the output of the Pooling3, the number of channels is 128, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv4a is connected with the convolutional layer conv4 b;
the convolutional layer conv4b is used for carrying out convolution operation on the output of the convolutional layer conv4a, outputting a characteristic diagram, wherein the number of channels is 256, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv4b is connected with a Pooling layer Pooling 4;
the Pooling layer Pooling4 is used for Pooling the output of the convolutional layer conv34, and the Pooling layer Pooling4 is connected with the convolutional layer conv5 a;
the convolutional layer conv5a is used for performing convolution operation on the output of the Pooling4, the number of channels is 512, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv5a is connected with the convolutional layer conv5 b;
the convolutional layer conv5b is used for carrying out convolution operation on the output of the convolutional layer conv5a, outputting a characteristic diagram, wherein the number of channels is 512, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv5b is connected with a Pooling layer Pooling 5;
the Pooling layer Pooling5 is used for Pooling the output of the convolutional layer conv5b, and the Pooling layer Pooling5 is connected with the Pooling layer Pooling 6;
the Pooling layer Pooling6 is used for Pooling the output of the Pooling layer Pooling5, and the Pooling layer Pooling6 is connected with the Pooling layer Pooling 67;
the convolution layer Output1 is used for performing convolution operation on the Output of the convolution layer conv4b, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolution layer Output1 is connected with the PriorBox layer PriorBox1, the convolution layer Loc1 and the convolution layer Conf 1;
the PriorBox layer PriorBox1 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output1, and the PriorBox layer PriorBox1 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc1 is used for performing convolution operation on the Output of the convolutional layer Output1, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc1 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf1 is used for performing convolution operation on the Output of the convolutional layer Output1, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf1 is connected with the fully-connected layer Mbox _ loc;
the convolution layer Output2 is used for performing convolution operation on the Output of the convolution layer conv5b, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolution layer Output2 is connected with the PriorBox layer PriorBox2, the convolution layer Loc2 and the convolution layer Conf 2;
the PriorBox layer PriorBox2 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output2, and the PriorBox layer PriorBox2 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc2 is used for performing convolution operation on the Output of the convolutional layer Output2, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc2 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf2 is used for performing convolution operation on the Output of the convolutional layer Output2, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf2 is connected with the fully-connected layer Mbox _ loc;
the convolutional layer Output3 is used for carrying out convolution operation on the Output of the Pooling layer 5, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Output3 is connected with the PriorBox layer PriorBox3, the convolutional layer Loc3 and the convolutional layer Conf 3;
the PriorBox layer PriorBox3 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output3, and the PriorBox layer PriorBox3 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc3 is used for performing convolution operation on the Output of the convolutional layer Output3, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc3 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf3 is used for performing convolution operation on the Output of the convolutional layer Output3, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf3 is connected with the fully-connected layer Mbox _ loc;
the convolutional layer Output4 is used for carrying out convolution operation on the Output of the Pooling layer 6, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Output4 is connected with the PriorBox layer PriorBox4, the convolutional layer Loc4 and the convolutional layer Conf 4;
the PriorBox layer PriorBox4 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output4, and the PriorBox layer PriorBox4 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc4 is used for performing convolution operation on the Output of the convolutional layer Output4, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc4 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf4 is used for performing convolution operation on the Output of the convolutional layer Output4, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf4 is connected with the fully-connected layer Mbox _ loc;
the convolutional layer Output5 is used for carrying out convolution operation on the Output of the Pooling layer 7, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Output5 is connected with the PriorBox layer PriorBox5, the convolutional layer Loc5 and the convolutional layer Conf 5;
the PriorBox layer PriorBox5 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output5, and the PriorBox layer PriorBox5 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc5 is used for performing convolution operation on the Output of the convolutional layer Output5, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc5 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf5 is used for performing convolution operation on the Output of the convolutional layer Output5, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf5 is connected with the fully-connected layer Mbox _ loc;
the fully-connected layer Mbox _ conf is respectively connected with the outputs of the convolutional layer Loc1, the convolutional layer Loc2, the convolutional layer Loc3, the convolutional layer Loc4 and the convolutional layer Loc5, and the output of the fully-connected layer Mbox _ conf is processed by a reshape layer, a softmax layer and a Flatten layer and is connected to an output layer DeprotectionOutput;
a full link layer Mbox _ loc connected to outputs of the convolutional layers Conf1, convolutional layers Conf2, convolutional layers Conf3, convolutional layers Conf4, and convolutional layers Conf5, respectively, and connected to the output layer detective output;
a full connection layer Mbox _ prior connected to the outputs of the PriorBox layer PriorBox1, the PriorBox layer PriorBox2, the PriorBox layer PriorBox3, the PriorBox layer PriorBox4, the PriorBox layer PriorBox5, respectively, to the output layer DetectionOutput;
and the output layer DetectionOutput is used for outputting the detection result of the SSD network model.
In the present embodiment, the convolution layer of the SSD network model includes convolution operation, Relu operation, and BatchNorm operation.
The following table 1 shows a structure diagram of the SSD network model in this embodiment, where the feature extraction layer is 1 to 14 layers in the table, the prediction layer is 15 to 40 layers in the table, and the output layer is 41 layers in the table, and in the SSD network model in this embodiment, the following methods are used to ensure accuracy and reduce the calculation amount at the same time: (1) in the feature extraction part, the number of channels of the front layers is set to be smaller, and the number of channels of the rear layers is larger; (2) in the feature extraction section, group of the conv1b, conv2b, conv3b, conv4b, conv5b layers is set to 4; (3) the output part uses less layers, and the model only has 1 layer; (4) setting the convolution kernel size to 1 at the output section; (5) and modifying the parameters of the Priorbox layer according to the size of the obstacle in the picture, specifically modifying the size of a preselected frame of the Priorbox layer, wherein the preselected frame is used for matching with the object in the picture, if the intersection of the preselected frame and the object is large, the matching is successful, and the category of the object is determined through a classification part.
Figure BDA0002496334420000101
Figure BDA0002496334420000111
Figure BDA0002496334420000121
TABLE 1
Step 103: and pruning and quantifying the trained SSD network model.
Specifically, pruning of the trained SSD network model includes the following steps.
Step 103-1: adding a loss function of the trained SSD network model into L1 regularization constraint, and performing network training by taking the weight of the trained SSD network model as an initial value;
step 103-2: adding the loss function of the SSD network model trained in the previous step into L1 regularization constraint, and performing network training by taking the weight of the SSD network model trained in the previous step as an initial value;
step 103-3: the loop goes to step 103-2 for resetting the weight with the smaller weight absolute value to 0.
Adding L1 regularization constraint to the loss function of the trained SSD network model network, training by taking the weight stored in the trained SSD network model as an initial value, adding L1 regularization during training to obtain sparser weight values, namely more values are 0, and then taking the result of the step as the initial value, gradually resetting the weight with a small absolute value to 0, namely cutting the weight, wherein the weight with 0 set in the network is 56.2 percent, the main calculation amount of the SSD algorithm is the dense matrix multiplication operation of the convolutional layer, and after pruning, the weight of the network has a large amount of 0, so that the dense matrix multiplication can be converted into the sparse matrix multiplication, and the speed can be doubled.
The trained weight is expressed as a 32-bit floating point number, the trained SSD network model is quantized, and the trained SSD network model is used for mapping the weight of the 32-bit floating point number into an 8-bit integer under a binary system, namely 0-255, the integer occupies smaller bandwidth, and the operation speed is higher, and the weight is expressed by the following formula:
xquantized＝xfloat÷xscale+xzero_point
in the formula, xquantizedFor quantized values, xfloatThe arbitrary weight in the weights of one convolutional layer in the SSD network model is expressed as:
Figure BDA0002496334420000122
wherein the content of the first and second substances,
Figure BDA0002496334420000123
and
Figure BDA0002496334420000124
the minimum and maximum values of the weights of one convolutional layer in the SSD network model,
in the formula, xscaleTo scale, it is expressed as:
Figure BDA0002496334420000131
in the formula, xzero_pointFor the number after quantization and before quantization with weight of 0, expressed as
Figure BDA0002496334420000132
Step 104: and when the acquired image is calibrated, the homography matrix of the image coordinate system and the world coordinate system is obtained, and the image coordinate system and the world coordinate system are associated through the homography matrix, so that the mutual conversion between the coordinates of the associated image coordinate system and the coordinates of the world coordinate system is realized.
Step 105: inputting the pictures acquired by the camera into a trained SSD network model, outputting the detected obstacles in the pictures, and marking the obstacles by a minimum circumscribed rectangle;
step 106: and converting the obtained minimum circumscribed rectangle of the obstacle into a world coordinate system according to the relation between the image coordinate system and the world coordinate system, and obtaining the obstacle information in the world coordinate system.
As shown in fig. 3, a schematic diagram of detecting obstacles by the SSD network model is shown, and it can be seen that the obstacles of the car, the person, and the ice cream barrel are respectively detected and marked by the minimum circumscribed rectangle.
Specifically, in step 3, the information of the obstacle obtained through the visual detection of the camera is matched with the information of the obstacle obtained through the measurement of the ultrasonic radar, and the information of the obstacle with the same matched information is output as a final detection result by taking the information of the obstacle obtained through the measurement of the ultrasonic radar; and for the obstacle information with inconsistent information after matching, outputting the information of the obstacle acquired through the visual detection of the camera as a final detection result, wherein for the obstacle acquired through the visual detection of the camera, the middle point of the minimum circumscribed rectangle is taken as the position of the obstacle
The obstacle detection method for automatic parking, provided by the invention, is characterized in that the position of the obstacle is finally determined by combining the obstacle information obtained by the camera vision detection and the ultrasonic radar measurement, the target easy to detect by the radar is a large target, such as a vehicle, the radar is difficult to detect a small target, such as a person, a popsicle barrel or a smaller target, and the camera vision recognition result is used as the standard for the small target difficult to detect by the radar; for detected pedestrians and ice cream barrels, the middle point of the minimum circumscribed rectangle is used as the position of an obstacle, and the position of the obstacle detected by an ultrasonic radar is used as the standard for large targets such as vehicles and the like; in the camera visual detection method, in order to further improve the running speed of the network, the trained SSD network model is pruned and quantized, unimportant weights are removed through pruning, quantized integer weights are calculated through quantization operation, the occupied bandwidth is smaller, the calculation speed is higher, the running speed is improved by 1 time through pruning and quantization operation, the precision is reduced by about 1%, the calculation amount can be reduced while the precision is ensured, and thus the visual detection result can be quickly obtained;
the method can be used for avoiding obstacles, judging the effectiveness of the parking space and the like, and in the automatic parking process, if the parking space is detected, whether the inside of the parking space contains the obstacles can be judged by using the method, so that whether the parking space is available can be judged.
In an embodiment of the present invention, there is also provided an obstacle detection system for automatic parking, including a memory, a processor, and a program stored on the memory and executable on the processor, wherein the processor implements the obstacle detection method for automatic parking as described above when executing the program.
In the above implementation of the obstacle detection system for automatic parking, the memory and the processor are electrically connected directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines, such as a bus. The memory stores computer-executable instructions for implementing the data access control method, and includes at least one software functional module which can be stored in the memory in the form of software or firmware, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory.
The Memory may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory is used for storing programs, and the processor executes the programs after receiving the execution instructions.
The processor may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In an embodiment of the present invention, there is also provided a computer-readable storage medium having a program stored thereon, the program, when executed by a processor, implementing the obstacle detection method for automatic parking as described above.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, systems, apparatuses and computer program products according to embodiments of the invention. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart.
The method for detecting an obstacle for automatic parking, the system for detecting an obstacle for automatic parking, and the use of a computer-readable storage medium in the context of the movement provided by the present invention have been described in detail above, and specific examples have been applied herein to illustrate the principles and embodiments of the present invention, and the above description of the embodiments is only intended to help understand the method of the present invention and its core ideas; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (10)

1. An obstacle detection method for automatic parking, characterized by comprising the steps of:
step 1: acquiring images through a camera, detecting the obstacles based on visual detection, and obtaining obstacle information;
step 2: reading the obstacle information measured by the ultrasonic radar;
and step 3: and finally determining the position of the obstacle by combining the obstacle information acquired by the camera vision detection and the ultrasonic radar measurement.
2. The obstacle detection method for automatic parking according to claim 1, wherein step 1 specifically includes the steps of:
step 101: constructing a data set, wherein the data set comprises picture data acquired according to a required scene and a label file of an obstacle;
step 102; establishing an SSD network model, and training the SSD network model by using a data set to obtain a trained SSD network model;
step 103: pruning and quantifying the trained SSD network model;
step 104: calibrating the camera to obtain the relation between an image coordinate system and a world coordinate system;
step 105: inputting the pictures acquired by the camera into a trained SSD network model, outputting the detected obstacles in the pictures, and marking the obstacles by a minimum circumscribed rectangle;
step 106: and converting the obtained minimum circumscribed rectangle of the obstacle into a world coordinate system according to the relation between the image coordinate system and the world coordinate system, and obtaining the obstacle information in the world coordinate system.
3. The obstacle detection method for automatic parking according to claim 1, characterized in that: in step 3, the information of the obstacle obtained by the visual detection of the camera is matched with the information of the obstacle obtained by the measurement of the ultrasonic radar, and the information of the obstacle with the same matched information is output as a final detection result by taking the information of the obstacle obtained by the measurement of the ultrasonic radar; and for the obstacle information with inconsistent information after matching, outputting the information of the obstacle acquired through the visual detection of the camera as a final detection result.
4. The obstacle detection method for automatic parking according to claim 2, characterized in that: in step 101, the label file comprises the category and position information of the vehicle, the pedestrian and the ice cream barrel in the picture; the data set is divided into a test set, a training set and a verification set according to a certain proportion, wherein the test set is used for training, the test set is used for testing whether the detection of the SSD network model is effective, and the verification set is used for monitoring the training effect of the SSD network model; in step 104, when the collected image is calibrated, a homography matrix of the image coordinate system and the world coordinate system is obtained, and the image coordinate system and the world coordinate system are associated through the homography matrix, so that mutual conversion between coordinates of the associated image coordinate system and coordinates of the world coordinate system is realized.
5. The obstacle detection method for automatic parking according to claim 2, characterized in that: in step 102, the SSD network model built comprises:
an input layer for inputting an image;
the convolutional layer conv1a is characterized in that an input image outputs a feature map through convolution operation, the number of channels is 16, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv1a is connected with the convolutional layer conv1 b;
the convolutional layer conv1b is used for carrying out convolution operation on the output of the convolutional layer conv1a, and outputting a characteristic diagram, wherein the number of channels is 16, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv1b is connected with a Pooling layer Pooling 1;
the Pooling layer Pooling1 is used for Pooling the output of the convolutional layer conv1b, and the Pooling layer Pooling1 is connected with the convolutional layer conv2 a;
the convolutional layer conv2a is used for performing convolution operation on the output of the Pooling1, the number of channels is 32, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv2a is connected with the convolutional layer conv2 b;
the convolutional layer conv2b is used for carrying out convolution operation on the output of the convolutional layer conv2a, outputting a characteristic diagram, wherein the number of channels is 32, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv2b is connected with the Pooling layer Pooling 2;
the Pooling layer Pooling2 is used for Pooling the output of the convolutional layer conv2b, and the Pooling layer Pooling2 is connected with the convolutional layer conv3 a;
the convolutional layer conv3a is used for carrying out convolution operation on the output of the Pooling2, the number of channels is 64, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv3a is connected with the convolutional layer conv3 b;
the convolutional layer conv3b is used for carrying out convolution operation on the output of the convolutional layer conv3a, outputting a characteristic diagram, wherein the number of channels is 64, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv3b is connected with the Pooling layer Pooling 3;
the Pooling layer Pooling3 is used for Pooling the output of the convolutional layer conv3b, and the Pooling layer Pooling3 is connected with the convolutional layer conv4 a;
the convolutional layer conv4a is used for carrying out convolution operation on the output of the Pooling3, the number of channels is 128, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv4a is connected with the convolutional layer conv4 b;
the convolutional layer conv4b is used for carrying out convolution operation on the output of the convolutional layer conv4a, outputting a characteristic diagram, wherein the number of channels is 256, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv4b is connected with a Pooling layer Pooling 4;
the Pooling layer Pooling4 is used for Pooling the output of the convolutional layer conv34, and the Pooling layer Pooling4 is connected with the convolutional layer conv5 a;
the convolutional layer conv5a is used for performing convolution operation on the output of the Pooling4, the number of channels is 512, the group is 1, the size of a convolution kernel is 3, and the convolutional layer conv5a is connected with the convolutional layer conv5 b;
the convolutional layer conv5b is used for carrying out convolution operation on the output of the convolutional layer conv5a, outputting a characteristic diagram, wherein the number of channels is 512, the group is 4, the size of a convolution kernel is 3, and the convolutional layer conv5b is connected with a Pooling layer Pooling 5;
the Pooling layer Pooling5 is used for Pooling the output of the convolutional layer conv5b, and the Pooling layer Pooling5 is connected with the Pooling layer Pooling 6;
the Pooling layer Pooling6 is used for Pooling the output of the Pooling layer Pooling5, and the Pooling layer Pooling6 is connected with the Pooling layer Pooling 67;
the convolution layer Output1 is used for performing convolution operation on the Output of the convolution layer conv4b, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolution layer Output1 is connected with the PriorBox layer PriorBox1, the convolution layer Loc1 and the convolution layer Conf 1;
the PriorBox layer PriorBox1 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output1, and the PriorBox layer PriorBox1 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc1 is used for performing convolution operation on the Output of the convolutional layer Output1, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc1 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf1 is used for performing convolution operation on the Output of the convolutional layer Output1, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf1 is connected with the fully-connected layer Mbox _ loc;
the convolution layer Output2 is used for performing convolution operation on the Output of the convolution layer conv5b, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolution layer Output2 is connected with the PriorBox layer PriorBox2, the convolution layer Loc2 and the convolution layer Conf 2;
the PriorBox layer PriorBox2 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output2, and the PriorBox layer PriorBox2 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc2 is used for performing convolution operation on the Output of the convolutional layer Output2, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc2 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf2 is used for performing convolution operation on the Output of the convolutional layer Output2, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf2 is connected with the fully-connected layer Mbox _ loc;
the convolutional layer Output3 is used for carrying out convolution operation on the Output of the Pooling layer 5, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Output3 is connected with the PriorBox layer PriorBox3, the convolutional layer Loc3 and the convolutional layer Conf 3;
the PriorBox layer PriorBox3 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output3, and the PriorBox layer PriorBox3 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc3 is used for performing convolution operation on the Output of the convolutional layer Output3, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc3 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf3 is used for performing convolution operation on the Output of the convolutional layer Output3, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf3 is connected with the fully-connected layer Mbox _ loc;
the convolutional layer Output4 is used for carrying out convolution operation on the Output of the Pooling layer 6, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Output4 is connected with the PriorBox layer PriorBox4, the convolutional layer Loc4 and the convolutional layer Conf 4;
the PriorBox layer PriorBox4 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output4, and the PriorBox layer PriorBox4 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc4 is used for performing convolution operation on the Output of the convolutional layer Output4, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc4 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf4 is used for performing convolution operation on the Output of the convolutional layer Output4, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf4 is connected with the fully-connected layer Mbox _ loc;
the convolutional layer Output5 is used for carrying out convolution operation on the Output of the Pooling layer 7, the number of channels is 256, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Output5 is connected with the PriorBox layer PriorBox5, the convolutional layer Loc5 and the convolutional layer Conf 5;
the PriorBox layer PriorBox5 is used for deploying a preselected box in a characteristic diagram Output by the convolutional layer Output5, and the PriorBox layer PriorBox5 is connected with the fully-connected layer Mbox _ prior;
the convolutional layer Loc5 is used for performing convolution operation on the Output of the convolutional layer Output5, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Loc5 is connected with a full connection layer Mbox _ conf;
the convolutional layer Conf5 is used for performing convolution operation on the Output of the convolutional layer Output5, the number of channels is 24, the group is 1, the size of a convolution kernel is 1, and the convolutional layer Conf5 is connected with the fully-connected layer Mbox _ loc;
the fully-connected layer Mbox _ conf is respectively connected with the outputs of the convolutional layer Loc1, the convolutional layer Loc2, the convolutional layer Loc3, the convolutional layer Loc4 and the convolutional layer Loc5, and the output of the fully-connected layer Mbox _ conf is processed by a reshape layer, a softmax layer and a Flatten layer and is connected to an output layer DeprotectionOutput;
a full link layer Mbox _ loc connected to outputs of the convolutional layers Conf1, convolutional layers Conf2, convolutional layers Conf3, convolutional layers Conf4, and convolutional layers Conf5, respectively, and connected to the output layer detective output;
a full connection layer Mbox _ prior connected to the outputs of the PriorBox layer PriorBox1, the PriorBox layer PriorBox2, the PriorBox layer PriorBox3, the PriorBox layer PriorBox4, the PriorBox layer PriorBox5, respectively, to the output layer DetectionOutput;
and the output layer DetectionOutput is used for outputting the detection result of the SSD network model.
6. The obstacle detection method for automatic parking according to claim 2, characterized in that: in step 103, pruning the trained SSD network model comprises the following steps:
step 103-1: adding a loss function of the trained SSD network model into L1 regularization constraint, and performing network training by taking the weight of the trained SSD network model as an initial value;
step 103-2: adding the loss function of the SSD network model trained in the previous step into L1 regularization constraint, and performing network training by taking the weight of the SSD network model trained in the previous step as an initial value;
step 103-3: the loop goes to step 103-2 for resetting the weight with the smaller weight absolute value to 0.
7. The obstacle detection method for automatic parking according to claim 2, characterized in that: in step 103, the trained weights are expressed as 32-bit floating point numbers, and the trained SSD network model is quantized to map the weights of the 32-bit floating point numbers to 8-bit integers in binary, i.e. 0 to 255, which are expressed by the following formula:
xquantized＝xfloat÷xscale+xzero_point
in the formula, xquantize8For quantized values, xfloatThe arbitrary weight in the weights of one convolutional layer in the SSD network model is expressed as:
Figure FDA0002496334410000051
wherein the content of the first and second substances,
Figure FDA0002496334410000052
and
Figure FDA0002496334410000053
the minimum and maximum values of the weight of one convolution layer in the SSD network model are respectively shown, wherein x isscaleTo scale, it is expressed as:
Figure FDA0002496334410000054
in the formula, xzero-pointFor the number after quantization and before quantization with weight of 0, expressed as
Figure FDA0002496334410000055
8. The obstacle detection method for automatic parking according to claim 3, characterized in that: in step 3, regarding the obstacle obtained by the camera visual detection, the midpoint of the minimum bounding rectangle is taken as the position of the obstacle.
9. An obstacle detection system for automatic parking, characterized by comprising a memory, a processor, and a program stored on the memory and executable on the processor, the processor implementing the obstacle detection method for automatic parking according to claim 1 when executing the program.
10. A computer-readable storage medium on which a program is stored, characterized in that: the program, when executed by a processor, implements the obstacle detection method for automatic parking according to claim 1.
CN202010419317.4A 2020-05-18 2020-05-18 Obstacle detection method, system and medium for automatic parking Pending CN111611906A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010419317.4A CN111611906A (en) 2020-05-18 2020-05-18 Obstacle detection method, system and medium for automatic parking

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010419317.4A CN111611906A (en) 2020-05-18 2020-05-18 Obstacle detection method, system and medium for automatic parking

Publications (1)

Publication Number Publication Date
CN111611906A true CN111611906A (en) 2020-09-01

Family

ID=72196808

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010419317.4A Pending CN111611906A (en) 2020-05-18 2020-05-18 Obstacle detection method, system and medium for automatic parking

Country Status (1)

Country Link
CN (1) CN111611906A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112277935A (en) * 2020-10-30 2021-01-29 广州小鹏自动驾驶科技有限公司 Automatic parking method and device
CN112572430A (en) * 2020-12-14 2021-03-30 深兰人工智能(深圳)有限公司 Collision risk determination method and device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106908783A (en) * 2017-02-23 2017-06-30 苏州大学 Obstacle detection method based on multi-sensor information fusion
US9760806B1 (en) * 2016-05-11 2017-09-12 TCL Research America Inc. Method and system for vision-centric deep-learning-based road situation analysis
CN109544990A (en) * 2018-12-12 2019-03-29 惠州市德赛西威汽车电子股份有限公司 A kind of method and system that parking position can be used based on real-time electronic map identification
CN109740463A (en) * 2018-12-21 2019-05-10 沈阳建筑大学 A kind of object detection method under vehicle environment
CN110775052A (en) * 2019-08-29 2020-02-11 浙江零跑科技有限公司 Automatic parking method based on fusion of vision and ultrasonic perception
CN111062382A (en) * 2019-10-30 2020-04-24 北京交通大学 Channel pruning method for target detection network

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9760806B1 (en) * 2016-05-11 2017-09-12 TCL Research America Inc. Method and system for vision-centric deep-learning-based road situation analysis
CN107368890A (en) * 2016-05-11 2017-11-21 Tcl集团股份有限公司 A kind of road condition analyzing method and system based on deep learning centered on vision
CN106908783A (en) * 2017-02-23 2017-06-30 苏州大学 Obstacle detection method based on multi-sensor information fusion
CN109544990A (en) * 2018-12-12 2019-03-29 惠州市德赛西威汽车电子股份有限公司 A kind of method and system that parking position can be used based on real-time electronic map identification
CN109740463A (en) * 2018-12-21 2019-05-10 沈阳建筑大学 A kind of object detection method under vehicle environment
CN110775052A (en) * 2019-08-29 2020-02-11 浙江零跑科技有限公司 Automatic parking method based on fusion of vision and ultrasonic perception
CN111062382A (en) * 2019-10-30 2020-04-24 北京交通大学 Channel pruning method for target detection network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
薛宪堂等: ""基于卷积神经网络的接触网支柱号识别方法研究"", 铁路计算机应用 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112277935A (en) * 2020-10-30 2021-01-29 广州小鹏自动驾驶科技有限公司 Automatic parking method and device
CN112277935B (en) * 2020-10-30 2022-03-11 广州小鹏自动驾驶科技有限公司 Automatic parking method and device
CN112572430A (en) * 2020-12-14 2021-03-30 深兰人工智能(深圳)有限公司 Collision risk determination method and device

Similar Documents

Publication Publication Date Title
CN111160379B (en) Training method and device of image detection model, and target detection method and device
CN109087510B (en) Traffic monitoring method and device
WO2023193401A1 (en) Point cloud detection model training method and apparatus, electronic device, and storage medium
CN111507327B (en) Target detection method and device
CN111247525A (en) Lane detection method and device, lane detection equipment and mobile platform
CN111611906A (en) Obstacle detection method, system and medium for automatic parking
CN112200884A (en) Method and device for generating lane line
CN105469090A (en) Frequency-domain-residual-error-based small target detection method and apparatus in infrared image
CN111476099A (en) Target detection method, target detection device and terminal equipment
CN113723216A (en) Lane line detection method and device, vehicle and storage medium
CN111009011B (en) Method, device, system and storage medium for predicting vehicle direction angle
CN111860512B (en) Vehicle identification method, device, electronic equipment and computer readable storage medium
CN112837384A (en) Vehicle marking method and device and electronic equipment
CN116740145A (en) Multi-target tracking method, device, vehicle and storage medium
CN116935369A (en) Ship water gauge reading method and system based on computer vision
CN113870754B (en) Method and system for judging defects of panel detection electronic signals
CN113536867A (en) Object identification method, device and system
CN112308061B (en) License plate character recognition method and device
CN112477868B (en) Collision time calculation method and device, readable storage medium and computer equipment
CN114638947A (en) Data labeling method and device, electronic equipment and storage medium
CN113591543A (en) Traffic sign recognition method and device, electronic equipment and computer storage medium
CN113298759A (en) Water area detection method and device, electronic equipment and storage medium
CN112686155A (en) Image recognition method, image recognition device, computer-readable storage medium and processor
CN116543365B (en) Lane line identification method and device, electronic equipment and storage medium
CN116626630B (en) Object classification method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination