CN111967396A - Processing method, device and equipment for obstacle detection and storage medium - Google Patents
Processing method, device and equipment for obstacle detection and storage medium Download PDFInfo
- Publication number
- CN111967396A CN111967396A CN202010832737.5A CN202010832737A CN111967396A CN 111967396 A CN111967396 A CN 111967396A CN 202010832737 A CN202010832737 A CN 202010832737A CN 111967396 A CN111967396 A CN 111967396A
- Authority
- CN
- China
- Prior art keywords
- training
- obstacle detection
- obstacle
- image data
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention provides a processing method, a device, equipment and a storage medium for obstacle detection, wherein the method comprises the following steps: acquiring original image data of a road surface; preprocessing the original image data to obtain first image data; detecting the first image data based on an obstacle detection model obtained by pre-training to obtain a detection result; and if the obstacle is determined to exist according to the detection result, performing alarm processing, wherein the network architecture of the obstacle detection model at least comprises a bidirectional characteristic pyramid module improved based on a characteristic pyramid network, so that deep information and shallow information can be effectively fused, the small target detection effect is better improved, the small target detection effect is not easily influenced by illumination, and the obstacle detection accuracy is improved.
Description
Technical Field
The present invention relates to the field of technologies, and in particular, to a method, an apparatus, a device, and a storage medium for processing obstacle detection.
Background
With the rapid development of traffic technology, safety on traffic roads is particularly important, including roads with higher vehicle speeds such as tunnels and highways, which are very sensitive to obstacles, and traffic accidents become frequent, so that how to effectively and rapidly identify road obstacles and remind drivers in time becomes a key problem.
In the prior art, the obstacle on the road is usually identified by detecting a moving object based on an opencv frame difference method and an optical flow method, so as to remove a background and detect the obstacle.
However, the detection effect of the existing obstacle detection method model is easily influenced by illumination in a scene, so that the detection result is not accurate enough.
Disclosure of Invention
The embodiment of the invention provides a processing method, a processing device, processing equipment and a storage medium for obstacle detection, and aims to overcome the defects of low obstacle detection accuracy and the like in the prior art.
In a first aspect, an embodiment of the present invention provides a processing method for obstacle detection, including:
acquiring original image data of a road surface;
preprocessing the original image data to obtain first image data;
detecting the first image data based on an obstacle detection model obtained by pre-training to obtain a detection result, wherein a network architecture of the obstacle detection model at least comprises a bidirectional characteristic pyramid module improved based on a characteristic pyramid network;
and if the obstacle is determined to be present according to the detection result, performing alarm processing.
In a second aspect, an embodiment of the present invention provides a method for constructing an obstacle detection model, including:
acquiring training sample data, wherein the training sample data comprises training image samples of a road surface and label data corresponding to each training image sample, and the label data comprises marked and recorded area frame positioning coordinates and corresponding classification labels in the training image samples;
preprocessing the training image sample to obtain a first image sample;
training a pre-established obstacle detection network based on the first image sample and the label data;
judging whether the preset requirement is met or not according to the training detection result and a preset loss function;
if the preset requirement is met, stopping training, and outputting the obstacle detection model obtained by training;
the obstacle detection network is a neural network based on a bidirectional feature pyramid and an attention mechanism.
In a third aspect, an embodiment of the present invention provides a processing apparatus for obstacle detection, including:
the first acquisition module is used for acquiring original image data of a road surface;
the first preprocessing module is used for preprocessing the original image data to obtain first image data;
the detection module is used for detecting and processing the first image data based on an obstacle detection model obtained by pre-training to obtain a detection result, and the network architecture of the obstacle detection model at least comprises a bidirectional feature pyramid module improved based on a feature pyramid network;
and the processing module is used for carrying out alarm processing if the obstacle is determined to be present according to the detection result.
In a fourth aspect, an embodiment of the present invention provides an obstacle detection model building apparatus, including:
the second acquisition module is used for acquiring training sample data, wherein the training sample data comprises training image samples of a road surface and label data corresponding to each training image sample, and the label data comprises marked and recorded area frame positioning coordinates and corresponding classification labels in the training image samples;
the second preprocessing module is used for preprocessing the training image sample to obtain a first image sample;
the training module is used for training a pre-established obstacle detection network based on the first image sample and the label data;
the judging module is used for judging whether the preset requirement is met or not according to the training detection result and a preset loss function;
the output module is used for stopping training and outputting the obstacle detection model obtained by training if the preset requirement is met;
the obstacle detection network is a neural network based on a bidirectional feature pyramid and an attention mechanism.
In a fifth aspect, an embodiment of the present invention provides an electronic device, including: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method as set forth in the first aspect and various possible designs of the first aspect, or to perform the method as set forth in the second aspect and various possible designs of the second aspect.
In a fourth aspect, the embodiments of the present invention provide a computer-readable storage medium, in which computer-executable instructions are stored, and when a processor executes the computer-executable instructions, the method according to the first aspect and various possible designs of the first aspect is implemented, or the method according to the second aspect and various possible designs of the second aspect is implemented.
According to the obstacle detection processing method, device and equipment and the storage medium provided by the embodiment of the invention, the network architecture of the obstacle detection model adopted for the obstacle detection at least comprises the bidirectional FPN module based on the FPN improvement, so that the deep information and the shallow information can be effectively fused, the small target detection effect is better improved, and the influence of illumination is not easy to occur, so that the obstacle detection accuracy is improved, and the driving safety can be further improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a block diagram of a processing system according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a processing method for obstacle detection according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a processing method for obstacle detection according to another embodiment of the present invention;
fig. 4 is a schematic structural diagram of an obstacle detection network according to an embodiment of the present invention;
fig. 5 is a network structure diagram of the ResNet34 according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of an attention mechanism network according to an embodiment of the present invention;
fig. 7 is a schematic flowchart of a method for constructing an obstacle detection model according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a processing apparatus for obstacle detection according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an obstacle detection model building apparatus according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
With the above figures, certain embodiments of the invention have been illustrated and described in more detail below. The drawings and written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the concepts of the invention by those skilled in the art with reference to specific embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms to which the present invention relates will be explained first:
FPN: feature Pyramid Networks for Object Detection a Feature Pyramid Object Detection network.
ABiFDet: the Attention of Bi-directional Feature Pyramid for Object Detection, a new Network for Object Detection based on an FPN improved Bi-directional Feature Pyramid with Attention mechanism.
ResNet: a classification network based on residual connection.
With the rapid development of traffic technology, safety on traffic roads is particularly important, including roads with higher vehicle speeds such as tunnels and highways, which are very sensitive to obstacles, and traffic accidents become frequent, so that how to effectively identify road obstacles and remind drivers is a key problem. In the prior art, the obstacle on the road is usually identified by detecting a moving object based on an opencv frame difference method and an optical flow method, so as to remove a background and detect the obstacle. However, the existing obstacle detection method has the disadvantages that the detection effect of the model is easily affected by the illumination in the scene, the calculation amount is large, various parameters need to be set and adjusted, each scene needs to be processed independently, and the process is complicated.
Aiming at the problems in the prior art, the inventor conducts creative research and finds that the detection effect of the prior art model is easily influenced by the illumination in the scene, so that the detection result is inaccurate, in order to solve the above problems, the inventors have creatively found that an improvement is made to FPN (Feature Pyramid Networks), on the basis of bottom-up feature fusion of the original FPN, top-down feature fusion connection is added to form a bidirectional feature fusion network, so that deep information and shallow information can be fused, the detection effect of small targets is better improved, and combines the feature selection of attention (attention mechanism), can better perform feature screening on feature fusion, the network is easier to train, the network performance is improved, the influence of illumination is not easy to occur, and the method is suitable for various road scenes and is simple to operate. Therefore, the embodiment of the invention provides a processing method for obstacle detection, which detects a road obstacle by adopting an obstacle detection model based on a bidirectional FPN (field programmable gate array) which is improved on the basis of the FPN, and carries out alarm prompt in time so that a driver or a control system of an automatic driving vehicle can notice the obstacle to avoid in time, thereby improving the accuracy of obstacle detection and improving the safety of road driving.
The obstacle detection processing method provided by the embodiment of the invention is suitable for a scene of detecting the road obstacle in the running process of the vehicle, and the vehicle can be an automatic driving vehicle or a driver driving vehicle. Fig. 1 is a schematic diagram of an architecture of a processing system according to an embodiment of the present invention. The processing system can comprise electronic equipment and camera equipment, and can also comprise alarm equipment, such as voice playing equipment, display equipment and the like, wherein the camera equipment can be a vehicle-mounted automobile data recorder or a camera and the like, and is used for shooting road video or images and sending the road video or images to the electronic equipment, the electronic equipment receives video data or original image data, detects obstacles, and sends out alarm prompts in time when the obstacles are detected. The alarm prompt can be sent in a voice form, a screen display form or a voice and display form so that a driver can avoid the obstacle in time, and the alarm prompt can be sent to an automatic driving control system so that the automatic driving control system can automatically control the vehicle to avoid the obstacle, and the alarm prompt can be specifically set according to actual requirements. According to the obstacle detection model adopted by the embodiment of the invention for detecting the obstacle, the network architecture at least comprises the bidirectional FPN module based on FPN improvement, the deep information and the shallow information can be effectively fused, the small target detection effect is better improved, the influence of illumination is not easy to occur, and therefore the obstacle detection accuracy is improved.
The obstacle detection model adopted by the embodiment of the invention can be called an ABiFDet model, namely a new target detection network model based on a bidirectional feature pyramid with an attention mechanism improved by FPN.
Furthermore, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. In the description of the following examples, "plurality" means two or more unless specifically limited otherwise.
The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
An embodiment of the present invention provides a processing method for detecting an obstacle, which is used for detecting an obstacle on a road surface. The execution subject of the present embodiment is a processing device for obstacle detection, which may be provided in an electronic apparatus, which may be provided in a vehicle.
As shown in fig. 2, a schematic flow chart of a processing method for obstacle detection provided in this embodiment is shown, where the method includes:
Specifically, the original image data may be road surface image data captured by the image capturing apparatus, or may be image data of each frame in road surface video data captured by the image capturing apparatus. The image pickup apparatus is provided at a certain position on the vehicle. Specifically, the image pickup device picks up road surface image data and transmits the road surface image data to the electronic device, and the electronic device receives the image data transmitted by the image pickup device as original image data of the road surface. And if the image pickup equipment shoots the road surface video data and sends the road surface video data to the electronic equipment, the electronic equipment takes each frame of image data in the video data as the road surface original image data.
Specifically, after the original image data of the road surface is obtained, the original image data needs to be preprocessed, so that the image data meeting the preset requirements is obtained.
For example, the preprocessing specifically may include a normalization processing and a scaling processing, and for example, the normalization processing may include: and (3) carrying out an averaging process on the original image data, namely, respectively subtracting 127.5 from RGB three channels of the original image, and then carrying out a variance removing process, namely, respectively dividing 255.0 by the RGB three channels of the image after the averaging. The scaling process may include scaling the normalized image to a preset size, such as to a 412 x 412 image. The specific preset size can be set according to actual requirements.
103, detecting the first image data based on the obstacle detection model obtained by pre-training to obtain a detection result, wherein the network architecture of the obstacle detection model at least comprises a bidirectional feature pyramid module improved based on the feature pyramid network.
After preprocessing the original image data, the first image data may be detected based on an obstacle detection model obtained through pre-training, so as to obtain a detection result. The detection result includes a first probability that the target frame obtained by the detection belongs to the obstacle and optionally a second probability that the target frame does not belong to the obstacle. The sum of the first probability and the second probability is 1. Whether the target frame belongs to the obstacle may be determined based on the first probability and the second probability. For example, a first threshold may be set, and when the first probability is greater than the first threshold, it is determined that the target frame belongs to the obstacle. The specific judgment rule may be set according to actual requirements, and this embodiment is not limited.
It should be noted that the obstacle detection model needs to be obtained by training in advance, specifically, a large number of training image samples and label data corresponding to each training image sample may be obtained in advance, where the label data is an obstacle region frame included in the training image samples and a label 1 of the obstacle to which the region frame belongs, and may also include a region frame not belonging to the obstacle and a corresponding label 0. And training the pre-established obstacle detection network according to the training image sample and the label data to obtain an obstacle detection model. Two tasks of classification and region frame regression are involved in the training process, and two loss functions are needed to be combined to determine whether the training is finished. For example, for classification, focal length may be used, for region box regression, a square error loss function may be used, and the specific loss function may be set according to actual requirements.
And 104, if the obstacle is determined to be present according to the detection result, performing alarm processing.
Specifically, after the detection result is obtained, whether the original image data has an obstacle and the position of the obstacle, that is, whether the corresponding road surface has an obstacle or not, may be determined according to the detection result. If the right obstacle is determined according to the detection result, alarm processing can be carried out in time. For an autonomous vehicle, the alarm processing may specifically be sending alarm information to an autonomous driving control system of the autonomous vehicle, so that the autonomous driving control system may control the vehicle to avoid the obstacle in time. For a vehicle driven by a person, the alarm processing can be sending out an alarm prompt, such as a voice prompt, a screen display plus a voice prompt and the like, so that a driver can avoid an obstacle in time, and the driving safety is improved.
According to the obstacle detection processing method provided by the embodiment, the obstacle detection model adopted for the obstacle detection at least comprises the bidirectional FPN module based on FPN improvement, the deep information and the shallow information can be effectively fused, the small target detection effect is better promoted, the small target detection effect is not easily influenced by illumination, the obstacle detection accuracy is improved, and the driving safety can be improved.
The method provided by the above embodiment is further described in an additional embodiment of the present invention.
As shown in fig. 3, a schematic flow chart of the processing method for obstacle detection according to this embodiment is provided.
As a practical way, on the basis of the above embodiment, optionally, the obstacle detection model is obtained by training in the following way:
And 2013, inputting the first image sample into a pre-established obstacle detection network to obtain a training detection result.
And 2015, if the preset requirement is met, stopping training to obtain an obstacle detection model.
Specifically, a large number of training image samples and label data corresponding to each training image sample may be obtained in advance, where the label data includes an obstacle region frame (specifically, region frame coordinates, such as coordinate values of four vertices, or coordinate values of two vertices at the top left corner and the bottom right corner, which may be set specifically according to actual requirements) included in the training image sample, a classification label 1 that the region frame belongs to an obstacle, and a region frame that does not belong to an obstacle and a corresponding classification label 0. And training the pre-established obstacle detection network according to the training image sample and the label data to obtain an obstacle detection model. Two tasks of classification and region frame regression are involved in the training process, and two loss functions are needed to be combined to determine whether the training is finished. For example, for classification, a focal local loss function may be used, for region box regression, a square error loss function may be used, and the specific loss function may be set according to actual requirements.
For example, the classification loss function is L1, the regression loss function is L2, certain weights a and b may be set for the two loss functions, a + b is 1, and L ═ a × L1+ b × L2 is used as a comprehensive target loss function in the training process to determine the training effect, and the specific weight may be set and adjusted according to actual needs, which is not limited in this embodiment.
Optionally, the obstacle detection network includes an input layer, a resnet34 network module, a bidirectional feature pyramid module with attention mechanism based on feature pyramid network improvement, and an output layer.
Exemplarily, as shown in fig. 4, a schematic diagram of an obstacle detection network structure provided for the present embodiment is shown. Wherein, the gray module represents a convolution layer, such as a 3 × 3 convolution layer, which adds the feature fusion of the FPN of bottom-to-top on the basis of the FPN of original top-to-bottom (the part inside the dashed box). The predictor represents 2 task branches, one is a classification task (class) and one is a regression task (box) of the goal box. In addition, the backbone part in the figure is a feature extraction network module of the resnet34, and an attention module, namely an attention module, adopts the existing attention structure.
Illustratively, as shown in fig. 5, a network structure diagram of the resnet34 is provided for the present embodiment. Of the 3 × 3conv,64,/2, 3 × 3conv represents a convolution kernel of 3 × 3 size, 64 represents an output channel, and/2 represents 1/2 whose output is the original one. Other similar matters are not described in detail. This figure can be used as part of the backbone in figure 4. And is only exemplary, the specific network structure may be set according to actual requirements,
illustratively, as shown in fig. 6, a schematic diagram of an attention mechanism network structure is provided for the present embodiment. The network structure of the figure can be used as the network structure of the attention module in figure 4. Wherein, x represents the input of the input,
representing output, residual representing residual, global posing representing a global pooling layer, FC representing a fully-connected layer, sigmoid being an activation function, scale representing that each pixel point of the layer feature graph is equivalent to several pixel points in an input image, that is, representing multiplication of two inputs (output of residual and output of sigmoid) connected with the layer feature graph, c representing a network channel number, h representing a feature graph height, and w representing a feature graph width.
As another implementable manner, on the basis of the foregoing embodiment, optionally, preprocessing the original image data to obtain the first image data includes:
Specifically, the preprocessing may specifically include a normalization processing and a scaling processing, for example, the normalization processing may include: and (3) carrying out an averaging process on the original image data, namely, respectively subtracting 127.5 from RGB three channels of the original image, and then carrying out a variance removing process, namely, respectively dividing 255.0 by the RGB three channels of the image after the averaging. The scaling process may include scaling the normalized image to a preset size, such as to a 412 x 412 image. The specific preset size can be set according to actual requirements.
As another implementable manner, on the basis of the above embodiment, optionally, the detection result includes a target frame coordinate included in the first image and a first probability that the target frame belongs to the obstacle; the method further comprises the following steps:
step 2031, determining whether the target frame is an obstacle according to the first probability and a preset threshold.
Optionally, the detection result may further include a second probability that the target frame does not belong to the obstacle.
After the detection result is obtained, whether the target frame is an obstacle or not can be judged according to the first probability in the detection result and a preset threshold value. For example, the preset threshold is set to 0.7, that is, the target frame is determined to be an obstacle when the first probability is greater than 0.7.
It should be noted that the respective implementable modes in the embodiment may be implemented individually, or may be implemented in combination in any combination without conflict, and the present invention is not limited thereto.
The processing method for obstacle detection provided by the embodiment, an obstacle detection model is adopted for obstacle detection, the network architecture at least comprises a bidirectional FPN module based on FPN improvement, deep information and shallow information can be effectively fused, the small target detection effect is better promoted, an attention mechanism is combined, feature screening in feature fusion is better carried out, the network is easier to train, the network performance is promoted, the obtained model is not easily influenced by illumination, the obstacle detection accuracy is improved, and the driving safety can be improved.
Another embodiment of the present invention provides a method for constructing an obstacle detection model, which is used to obtain an obstacle detection model.
As shown in fig. 7, a flow chart of the method for constructing the obstacle detection model according to this embodiment is shown. The method comprises the following steps:
And 2045, stopping training if the preset requirement is met, and outputting the obstacle detection model obtained through training.
The obstacle detection network is a neural network based on a bidirectional feature pyramid and an attention mechanism.
Optionally, the output of the obstacle detection network includes two parts, one part is a region frame obtained by detection, and the other part is a classification result of the region frame, specifically, a probability that the region frame belongs to an obstacle.
It should be noted that the specific operations of the above steps have been described in detail in the above embodiments, and are not described again here.
According to the obstacle detection model construction method provided by the embodiment, the adopted network architecture at least comprises the bidirectional FPN module based on FPN improvement, deep information and shallow information can be effectively fused, the small target detection effect is better improved, the attention mechanism is combined, the feature screening in feature fusion is better carried out, the network is easier to train, the network performance is improved, the obtained model is not easily influenced by illumination, and the obstacle detection accuracy is improved.
Still another embodiment of the present invention provides a processing apparatus for obstacle detection, configured to execute the method of the foregoing embodiment.
As shown in fig. 8, is a schematic structural diagram of the processing device for obstacle detection provided in this embodiment. The processing device 30 for obstacle detection includes a first acquiring module 31, a first preprocessing module 32, a detecting module 33 and a processing module 34.
The first acquisition module is used for acquiring original image data of a road surface; the first preprocessing module is used for preprocessing original image data to obtain first image data; the detection module is used for detecting and processing the first image data based on an obstacle detection model obtained by pre-training to obtain a detection result, and the network architecture of the obstacle detection model at least comprises a bidirectional feature pyramid module improved based on a feature pyramid network; and the processing module is used for carrying out alarm processing if the obstacle is determined to be present according to the detection result.
The specific manner in which the respective modules perform operations has been described in detail in relation to the apparatus in this embodiment, and will not be elaborated upon here.
According to the processing apparatus that the barrier detected that this embodiment provided, to the barrier detection model that the detection of barrier adopted, its network architecture includes at least based on FPN modified two-way FPN module, can effectively fuse deep information and shallow information, better promotes little target detection effect, is difficult for being influenced by the illumination to improve the accuracy that the barrier detected, and then can improve the security of traveling.
The device provided by the above embodiment is further described in an additional embodiment of the present invention.
As an implementable manner, on the basis of the foregoing embodiment, optionally, the first obtaining module is further configured to obtain training sample data, where the training sample data includes training image samples and tag data corresponding to each training image sample;
the first preprocessing module is also used for preprocessing the training image sample to obtain a first image sample;
the detection module is also used for inputting the first image sample into a pre-established barrier detection network to obtain a training detection result;
and the processing module is further used for judging whether the preset requirement is met or not according to the training detection result and the preset loss function, and stopping training if the preset requirement is met.
Optionally, the obstacle detection network includes an input layer, a resnet34 network module, a bidirectional feature pyramid module with attention mechanism based on feature pyramid network improvement, and an output layer.
As another implementable manner, on the basis of the foregoing embodiment, optionally, the first preprocessing module is specifically configured to:
normalizing the original image data to obtain normalized image data;
and carrying out scaling processing on the normalized image data to obtain first image data with a preset size.
As another implementable manner, on the basis of the above embodiment, optionally, the detection result includes a target frame coordinate included in the first image and a first probability that the target frame belongs to the obstacle; and the processing module is also used for judging whether the target frame is an obstacle or not according to the first probability and a preset threshold value.
The specific manner in which the respective modules perform operations has been described in detail in relation to the apparatus in this embodiment, and will not be elaborated upon here.
It should be noted that the respective implementable modes in the embodiment may be implemented individually, or may be implemented in combination in any combination without conflict, and the present invention is not limited thereto.
According to the processing apparatus that the barrier detected of this embodiment, the barrier detection model that adopts to the detection of barrier, its network architecture includes at least based on FPN modified two-way FPN module, can effectively fuse deep information and shallow information, better promotion small object detection effect, and combined attention mechanism, the better characteristic screening that carries out on the feature fusion, make the network train more easily, promote the network performance, the model that obtains is difficult for being influenced by the illumination, thereby improve the accuracy that the barrier detected, and then can improve the security of traveling.
Another embodiment of the present invention provides an obstacle detection model building apparatus for building an obstacle detection model.
As shown in fig. 9, a schematic structural diagram of the obstacle detection model building apparatus provided in this embodiment is shown. The obstacle detection model construction apparatus 40 includes a second acquisition module 41, a second preprocessing module 42, a training module 43, a judgment module 44, and an output module 45.
The second acquisition module is used for acquiring training sample data, the training sample data comprises training image samples of the road surface and label data corresponding to each training image sample, and the label data comprises marked and recorded area frame positioning coordinates and corresponding classification labels in the training image samples; the second preprocessing module is used for preprocessing the training image sample to obtain a first image sample; the training module is used for training a pre-established barrier detection network based on the first image sample and the label data; the judging module is used for judging whether the preset requirement is met or not according to the training detection result and a preset loss function; the output module is used for stopping training and outputting the obstacle detection model obtained by training if the preset requirement is met; the obstacle detection network is a neural network based on a bidirectional feature pyramid and an attention mechanism.
It should be noted that the respective implementable modes in the embodiment may be implemented individually, or may be implemented in combination in any combination without conflict, and the present invention is not limited thereto.
According to the barrier detection model construction device of this embodiment, the network framework that adopts includes at least based on FPN modified two-way FPN module, can effectively fuse deep information and shallow information, better promotes little target detection effect to combined attention mechanism, better carry out the feature screening on the feature fusion, make the network train more easily, promote the network performance, the model that obtains is difficult for being influenced by the illumination, thereby improves the accuracy that the barrier detected.
Still another embodiment of the present invention provides an electronic device, configured to perform the method provided by the foregoing embodiment. The electronic device may be provided in a vehicle, and may specifically be a control device in the vehicle.
As shown in fig. 10, a schematic structural diagram of the electronic device provided in this embodiment is shown. The electronic device 50 includes: at least one processor 51 and memory 52;
the memory stores computer-executable instructions; the at least one processor executes computer-executable instructions stored by the memory to cause the at least one processor to perform a method as provided by any of the embodiments above.
According to the electronic equipment of this embodiment, the obstacle detection model that adopts to the detection of obstacle, its network architecture includes at least based on FPN modified two-way FPN module, can effectively fuse deep information and shallow information, better promotion small object detection effect, and combined attention mechanism, better carry out the feature screening on the feature fusion, make the network train more easily, promote the network performance, the model that obtains is difficult for being influenced by the illumination, thereby improve the accuracy that the obstacle detected, and then can improve the security of traveling.
Yet another embodiment of the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the method provided in any one of the above embodiments is implemented.
According to the computer readable storage medium of this embodiment, the obstacle detection model that adopts to the detection of obstacle, its network architecture includes at least based on FPN modified two-way FPN module, can effectively fuse deep information and shallow information, better promotion small object detection effect, and combined attention mechanism, the better characteristic screening that carries on in the feature fusion, make the network train more easily, promote the network performance, the model that obtains is difficult for being influenced by the illumination, thereby improve the accuracy that the obstacle detected, and then can improve the security of traveling.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A method for processing obstacle detection, comprising:
acquiring original image data of a road surface;
preprocessing the original image data to obtain first image data;
detecting the first image data based on an obstacle detection model obtained by pre-training to obtain a detection result, wherein a network architecture of the obstacle detection model at least comprises a bidirectional characteristic pyramid module improved based on a characteristic pyramid network;
and if the obstacle is determined to be present according to the detection result, performing alarm processing.
2. The method of claim 1, wherein the obstacle detection model is trained by:
acquiring training sample data, wherein the training sample data comprises training image samples and label data corresponding to each training image sample;
preprocessing the training image sample to obtain a first image sample;
inputting the first image sample into a pre-established barrier detection network to obtain a training detection result;
judging whether the preset requirement is met or not according to the training detection result and a preset loss function;
and if the preset requirement is met, stopping training.
3. The method of claim 2, wherein the obstacle detection network comprises an input layer, a resnet34 network module, a bidirectional feature pyramid with attention mechanism based on feature pyramid network refinement, and an output layer.
4. The method of claim 1, wherein pre-processing the raw image data to obtain first image data comprises:
normalizing the original image data to obtain normalized image data;
and carrying out scaling processing on the normalized image data to obtain first image data with a preset size.
5. The method according to any one of claims 1-4, wherein the detection result includes a target frame coordinate included in the first image and a first probability that the target frame belongs to an obstacle; the method further comprises the following steps:
and judging whether the target frame is an obstacle or not according to the first probability and a preset threshold value.
6. A method for constructing an obstacle detection model, comprising:
acquiring training sample data, wherein the training sample data comprises training image samples of a road surface and label data corresponding to each training image sample, and the label data comprises marked and recorded area frame positioning coordinates and corresponding classification labels in the training image samples;
preprocessing the training image sample to obtain a first image sample;
training a pre-established obstacle detection network based on the first image sample and the label data;
judging whether the preset requirement is met or not according to the training detection result and a preset loss function;
if the preset requirement is met, stopping training, and outputting the obstacle detection model obtained by training;
the obstacle detection network is a neural network based on a bidirectional feature pyramid and an attention mechanism.
7. A processing apparatus for obstacle detection, comprising:
the first acquisition module is used for acquiring original image data of a road surface;
the first preprocessing module is used for preprocessing the original image data to obtain first image data;
the detection module is used for detecting and processing the first image data based on an obstacle detection model obtained by pre-training to obtain a detection result, and the network architecture of the obstacle detection model at least comprises a bidirectional feature pyramid module improved based on a feature pyramid network;
and the processing module is used for carrying out alarm processing if the obstacle is determined to be present according to the detection result.
8. An obstacle detection model construction device characterized by comprising:
the second acquisition module is used for acquiring training sample data, wherein the training sample data comprises training image samples of a road surface and label data corresponding to each training image sample, and the label data comprises marked and recorded area frame positioning coordinates and corresponding classification labels in the training image samples;
the second preprocessing module is used for preprocessing the training image sample to obtain a first image sample;
the training module is used for training a pre-established obstacle detection network based on the first image sample and the label data;
the judging module is used for judging whether the preset requirement is met or not according to the training detection result and a preset loss function;
the output module is used for stopping training and outputting the obstacle detection model obtained by training if the preset requirement is met;
the obstacle detection network is a neural network based on a bidirectional feature pyramid and an attention mechanism.
9. An electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method of any one of claims 1-6.
10. A computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a processor, implement the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010832737.5A CN111967396A (en) | 2020-08-18 | 2020-08-18 | Processing method, device and equipment for obstacle detection and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010832737.5A CN111967396A (en) | 2020-08-18 | 2020-08-18 | Processing method, device and equipment for obstacle detection and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111967396A true CN111967396A (en) | 2020-11-20 |
Family
ID=73387836
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010832737.5A Pending CN111967396A (en) | 2020-08-18 | 2020-08-18 | Processing method, device and equipment for obstacle detection and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111967396A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818932A (en) * | 2021-02-26 | 2021-05-18 | 北京车和家信息技术有限公司 | Image processing method, obstacle detection device, medium, and vehicle |
| CN113012220A (en) * | 2021-02-02 | 2021-06-22 | 深圳市识农智能科技有限公司 | Fruit counting method and device and electronic equipment |
| CN113313668A (en) * | 2021-04-19 | 2021-08-27 | 石家庄铁道大学 | Subway tunnel surface disease feature extraction method |
| CN113361375A (en) * | 2021-06-02 | 2021-09-07 | 武汉理工大学 | Vehicle target identification method based on improved BiFPN |
| CN116206281A (en) * | 2023-04-27 | 2023-06-02 | 北京惠朗时代科技有限公司 | Sight line detection method and device, electronic equipment and storage medium |
-
2020
- 2020-08-18 CN CN202010832737.5A patent/CN111967396A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113012220A (en) * | 2021-02-02 | 2021-06-22 | 深圳市识农智能科技有限公司 | Fruit counting method and device and electronic equipment |
| CN112818932A (en) * | 2021-02-26 | 2021-05-18 | 北京车和家信息技术有限公司 | Image processing method, obstacle detection device, medium, and vehicle |
| CN113313668A (en) * | 2021-04-19 | 2021-08-27 | 石家庄铁道大学 | Subway tunnel surface disease feature extraction method |
| CN113361375A (en) * | 2021-06-02 | 2021-09-07 | 武汉理工大学 | Vehicle target identification method based on improved BiFPN |
| CN113361375B (en) * | 2021-06-02 | 2022-06-07 | 武汉理工大学 | Vehicle target identification method based on improved BiFPN |
| CN116206281A (en) * | 2023-04-27 | 2023-06-02 | 北京惠朗时代科技有限公司 | Sight line detection method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7052663B2 (en) | Object detection device, object detection method and computer program for object detection | |
| CN111967396A (en) | Processing method, device and equipment for obstacle detection and storage medium | |
| CN104616502B (en) | Car license recognition and alignment system based on combination type bus or train route video network | |
| US11371851B2 (en) | Method and system for determining landmarks in an environment of a vehicle | |
| CN112967283B (en) | Target identification method, system, equipment and storage medium based on binocular camera | |
| CN111209780A (en) | Lane line attribute detection method and device, electronic device and readable storage medium | |
| WO2020154990A1 (en) | Target object motion state detection method and device, and storage medium | |
| CN107909012B (en) | Real-time vehicle tracking detection method and device based on disparity map | |
| CN110348332B (en) | Method for extracting multi-target real-time trajectories of non-human machines in traffic video scene | |
| JP2014106685A (en) | Vehicle periphery monitoring device | |
| CN111932596B (en) | Method, device and equipment for detecting camera occlusion area and storage medium | |
| US11482007B2 (en) | Event-based vehicle pose estimation using monochromatic imaging | |
| CN112132071A (en) | Processing method, device and equipment for identifying traffic jam and storage medium | |
| CN115761668A (en) | Camera stain recognition method and device, vehicle and storage medium | |
| CN114841910A (en) | Vehicle-mounted lens shielding identification method and device | |
| CN114419603A (en) | Automatic driving vehicle control method and system and automatic driving vehicle | |
| JP2018073275A (en) | Image recognition device | |
| CN116363628A (en) | Mark detection method and device, nonvolatile storage medium and computer equipment | |
| Dai et al. | A driving assistance system with vision based vehicle detection techniques | |
| CN112686136B (en) | Object detection method, device and system | |
| CN115965934A (en) | Parking space detection method and device | |
| CN115565155A (en) | Training method of neural network model, generation method of vehicle view and vehicle | |
| CN104077566B (en) | Bayonet socket picture method for detecting human face based on color difference | |
| CN112215042A (en) | Parking space limiter identification method and system and computer equipment | |
| JP2021152826A (en) | Information processing device, subject classification method, and subject classification program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |