CN110210324B - Road target rapid detection early warning method and system - Google Patents

Road target rapid detection early warning method and system Download PDF

Info

Publication number
CN110210324B
CN110210324B CN201910383340.XA CN201910383340A CN110210324B CN 110210324 B CN110210324 B CN 110210324B CN 201910383340 A CN201910383340 A CN 201910383340A CN 110210324 B CN110210324 B CN 110210324B
Authority
CN
China
Prior art keywords
target
level
distance
detection
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910383340.XA
Other languages
Chinese (zh)
Other versions
CN110210324A (en
Inventor
陶文兵
石京磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN201910383340.XA priority Critical patent/CN110210324B/en
Publication of CN110210324A publication Critical patent/CN110210324A/en
Application granted granted Critical
Publication of CN110210324B publication Critical patent/CN110210324B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Abstract

The invention discloses a road target rapid detection early warning method and a system, belonging to the field of computer vision, wherein the method comprises the following steps: acquiring a road scene image by using image acquisition equipment, and extracting N-level interested areas from the road scene image according to the sequence of the target distance from near to far; sequentially carrying out target detection on the N-level interested areas according to the sequence of the target distance from near to far, carrying out early warning after a certain-level interested area with a near target distance detects a target, and not detecting the subsequent-level interested areas; wherein N is more than or equal to 2, and the target distance is the distance between the target and the image acquisition equipment. Compared with the existing detection and early warning algorithm for the targets such as pedestrians, vehicles and the like, the method can detect and early warn the targets appearing on the driving route more quickly and accurately, and has better practical value.

Description

Road target rapid detection early warning method and system
Technical Field
The invention belongs to the field of computer vision, and particularly relates to a road target rapid detection early warning method and system.
Background
The pedestrian and vehicle detection technology is widely applied to the driving assistance system. In urban traffic scenes, how to avoid traffic accidents is always a hot point problem. The current assistant driving mainly utilizes pedestrian and vehicle detection technology to detect and analyze images in front of running motor vehicles, and actively gives an alarm for pedestrians and vehicles, so that the assistant driving can assist drivers to avoid in advance and prevent traffic accidents. The current pedestrian and vehicle detection technology mainly utilizes a deep learning-based correlation algorithm to detect and identify objects such as pedestrians and vehicles in an input image. One important requirement of a driving assistance system is real-time performance. The vehicle is very fast for people, if the pedestrian can not be detected quickly and the alarm is given, the danger can not be avoided in time. In addition, the system needs to have higher accuracy, and the occurrence of missed alarm and false alarm conditions is reduced as much as possible.
At present, a plurality of excellent algorithms for deeply learning the pedestrian and the vehicle are provided, compared with the traditional algorithm, the accuracy and the robustness are much higher, and the algorithm can be quickly operated on a high-performance server. However, if these networks are directly transplanted to platforms such as embedded devices and mobile devices, the operation speed is greatly reduced due to the difference of computing performance of hardware platforms. The requirement for rapidity cannot be met. Therefore, the networks running on embedded devices and mobile devices must be lightweight neural networks. But simply, the pedestrian and vehicle detection network is simplified accurately and is directly used for detection and early warning of pedestrians and vehicles, although an obvious acceleration effect can be obtained, the detection precision is inevitably reduced more, and the actual use requirement of the vehicle-mounted portable pedestrian detection and early warning system cannot be met.
Therefore, the technical problem that the target on the driving route cannot be detected and early warned quickly and accurately exists in the prior art.
Disclosure of Invention
Aiming at the defects or improvement requirements of the prior art, the invention provides a road target rapid detection and early warning method and a system, so that the technical problem that the prior art cannot rapidly and accurately detect and early warn the target on a driving route is solved.
In order to achieve the above object, according to an aspect of the present invention, there is provided a method for quickly detecting and warning a road target, comprising the steps of:
(1) acquiring a road scene image by using image acquisition equipment, and extracting N-level interested areas from the road scene image according to the sequence of the target distance from near to far;
(2) sequentially carrying out target detection on the N-level interested areas according to the sequence of the target distance from near to far, carrying out early warning after a certain-level interested area with a near target distance detects a target, and not detecting the subsequent-level interested areas;
wherein N is more than or equal to 2, and the target distance is the distance between the target and the image acquisition equipment.
Further, the step (1) comprises the following steps:
collecting road scene images, setting a target height ratio threshold value Thr and a nearest target distance L0；
According to the relation f between the pixel height of the target and the target distance and the nearest target distance L0To obtain the pixel height H of the target1Is prepared from H1Calculating the width of the first-stage interested area by using the length of the first-stage interested area and the length-width ratio of the road scene image as the length of the first-stage interested area;
for the N-level region of interest, the target height is taken to be longer than the threshold Thr and the N-1 level region of interest HN-1Product of (A) and (B)NCalculating the width of the N-level interested area by using the length of the N-level interested area and the length-width ratio of the road scene image as the length of the N-level interested area;
obtaining the maximum distance L of the detection target of the N-1 level interested region according to the length and the width of the N level interested region and the relation f between the pixel height of the target and the target distanceN-1While L isN-1The nearest distance of the detection target of the Nth-level region of interest;
using the target height to make the ratio of the threshold Thr to the length H of the Nth order region of interestNThe maximum distance L of the detection target of the Nth-level interested area is obtained by the product of the maximum distance L and the maximum distance f of the targetN(ii) a The target distance detection range of the Nth-level interested region is (L)N-1，LN)。
Further, the step (1) further comprises:
collecting road scene image sets at different target distances, acquiring pixel height of a target and discrete data of the distance between the target and the road scene image sets, and obtaining the relation f between the pixel height of the target and the distance between the target and the discrete data.
Further, the value range of the target height ratio threshold Thr is (0, 1).
Further, the centers of the road scene image and the N-level region of interest are both lane line intersection points.
Further, the specific implementation manner of the target detection in the step (2) is as follows:
carrying out target detection on the N-level region of interest by using a target detection model, wherein the training of the target detection model comprises the following steps:
extracting N-level interested area samples from the sample road scene image, and marking the target with the target height ratio larger than the target height ratio threshold value in each interested area sample to obtain a training sample set;
constructing a target detection convolutional neural network comprising a feature extraction layer, and extracting features of N-level region of interest samples in a training sample set in the feature extraction layer to obtain a feature map;
setting a plurality of preselection frames in the characteristic diagram, calculating the overlapping rate of a marked target in a region of interest sample in a training sample set and each preselection frame, training a target detection convolutional neural network by using the preselection frames and the training sample set, wherein the overlapping rate is greater than a preset value, obtaining a training result, calculating a loss value of the training result, and performing back propagation by using the loss value to update the parameters of the target detection convolutional neural network, thereby obtaining a target detection model.
Further, the height ratio of the preselected box is greater than the target height ratio threshold.
According to another aspect of the present invention, there is provided a road target rapid detection and early warning system, comprising:
the interesting region extracting module is used for acquiring road scene images by utilizing image acquisition equipment and extracting N-level interesting regions from the road scene images according to the sequence of the target distance from near to far;
the target detection module is used for sequentially carrying out target detection on the N-level interested areas by using a target detection model according to the sequence of the target distance from near to far, carrying out early warning after a certain-level interested area with a near target distance detects a target, and not detecting the subsequent-level interested areas;
wherein N is more than or equal to 2, and the target distance is the distance between the target and the image acquisition equipment.
Further, the region of interest extracting module includes the following sub-modules:
an initialization submodule for acquiring road scene images, setting a target height ratio threshold value Thr and a nearest target distance L0；
A first-level interested region length and width calculation submodule used for calculating the length and the width of the interested region according to the relationship f between the pixel height of the target and the target distance and the distance L between the nearest target and the target0To obtain the pixel height H of the target1Is prepared from H1Calculating the width of the first-stage interested area by using the length of the first-stage interested area and the length-width ratio of the road scene image as the length of the first-stage interested area;
an Nth-level interested region length and width calculation sub-module, which is used for comparing the target height with a threshold Thr and the length H of the N-1 level interested region for the Nth-level interested regionN-1Product of (A) and (B)NCalculating the width of the N-level interested area by using the length of the N-level interested area and the length-width ratio of the road scene image as the length of the N-level interested area;
a target distance detection range calculation submodule of the Nth level interested area, which is used for obtaining the farthest distance L of the detection target of the N-1 level interested area according to the length and the width of the Nth level interested area and the relation f between the pixel height of the target and the target distanceN-1While L isN-1The nearest distance of the detection target of the Nth-level region of interest; using the target height to make the ratio of the threshold Thr to the length H of the Nth order region of interestNProduct of and pixel height of the target and the targetThe distance relation f is used for obtaining the farthest distance L of the detection target of the Nth-level interested regionN(ii) a The target distance detection range of the Nth-level interested region is (L)N-1，LN)。
Further, the training of the target detection model comprises:
the training sample set acquisition sub-module is used for extracting N-level interested area samples from the sample road scene image, and marking the target with the target height ratio larger than the target height ratio threshold value in each interested area sample to obtain a training sample set;
the characteristic diagram acquisition sub-module is used for constructing a target detection convolutional neural network comprising a characteristic extraction layer, and extracting the characteristics of N-level region of interest samples in the training sample set in the characteristic extraction layer to obtain a characteristic diagram;
the training submodule is used for setting a plurality of preselection frames in the characteristic diagram, calculating the overlapping rate of a marked target in an interested area sample in the training sample set and each preselection frame, training a target detection convolutional neural network by using the preselection frames with the overlapping rate larger than a preset value and the training sample set to obtain a training result, calculating the loss value of the training result, and performing back propagation by using the loss value to update the parameters of the target detection convolutional neural network to obtain a target detection model.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
(1) the invention adopts a grading target detection early warning strategy, so that the near targets can be detected at the fastest speed, the early warning time is shortened as much as possible for the most dangerous condition, and the targets with small distance can be detected. Therefore, compared with the existing detection and early warning algorithm for the targets such as pedestrians, vehicles and the like, the method and the device can accelerate the detection and early warning of the targets close to the vehicle to the maximum extent, can detect and early warn the targets far away from the vehicle, can detect and early warn the targets appearing on the driving route more quickly and accurately, and have better practical value.
(2) The existing target detection algorithm generally has worse detection accuracy for small targets than for large targets. The invention improves the network operation speed by simplifying the structural design of the target detection network. By adopting the strategy of hierarchical detection, each level of detection only needs to detect a larger target. This can increase the speed of the network.
(3) According to the invention, the interesting regions with the target height ratio larger than the target height ratio threshold are marked for training by utilizing all levels of interesting regions, so that the detection precision of the target detection network on the target in the actual picture is improved, and the detection of a smaller target is avoided.
Drawings
Fig. 1 is a schematic flow chart of a method for quickly detecting and warning a road target according to an embodiment of the present invention;
FIG. 2 is a flow chart of a preferred embodiment provided by an embodiment of the present invention;
fig. 3 is a three-level region of interest extraction method provided in the embodiment of the present invention;
FIG. 4 is a schematic flow chart of target detection model training data set labeling according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a feature diagram and a pre-selected box design provided by an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a target detection convolutional neural network according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
As shown in fig. 1, a rapid detection and early warning method for road targets includes the following steps:
(1) acquiring a road scene image by using image acquisition equipment, and extracting N-level interested areas from the road scene image according to the sequence of the target distance from near to far;
(2) sequentially carrying out target detection on the N-level interested areas according to the sequence of the target distance from near to far, carrying out early warning after a certain-level interested area with a near target distance detects a target, and not detecting the subsequent-level interested areas;
wherein N is more than or equal to 2, and the target distance is the distance between the target and the image acquisition equipment.
Further, the step (1) comprises the following steps:
collecting road scene images, setting a target height ratio threshold value Thr and a nearest target distance L0(ii) a Acquiring a road scene image set under different target distances, acquiring pixel height of a target and discrete data of the distance between the target and the road scene image set, and obtaining a relation f between the pixel height of the target and the distance between the target and the discrete data;
according to the relation f between the pixel height of the target and the target distance and the nearest target distance L0To obtain the pixel height H of the target1Is prepared from H1Calculating the width of the first-stage interested area by using the length of the first-stage interested area and the length-width ratio of the road scene image as the length of the first-stage interested area;
for the N-level region of interest, the target height is taken to be longer than the threshold Thr and the N-1 level region of interest HN-1Product of (A) and (B)NCalculating the width of the N-level interested area by using the length of the N-level interested area and the length-width ratio of the road scene image as the length of the N-level interested area;
obtaining the maximum distance L of the detection target of the N-1 level interested region according to the length and the width of the N level interested region and the relation f between the pixel height of the target and the target distanceN-1While L isN-1The nearest distance of the detection target of the Nth-level region of interest;
using the target height to make the ratio of the threshold Thr to the length H of the Nth order region of interestNThe maximum distance L of the detection target of the Nth-level interested area is obtained by the product of the maximum distance L and the maximum distance f of the targetN(ii) a Target distance detection of Nth-level region of interestRange of (L)N-1，LN)。
Further, the value range of the target height ratio threshold Thr is (O, 1), and preferably, the value of the target height ratio threshold Thr is 0.5.
Further, the centers of the road scene image and the N-level region of interest are both lane line intersection points.
Further, the specific implementation manner of the target detection in the step (2) is as follows:
carrying out target detection on the N-level region of interest by using a target detection model, wherein the training of the target detection model comprises the following steps:
extracting N-level interested area samples from the sample road scene image, and marking the target with the target height ratio larger than the target height ratio threshold value in each interested area sample to obtain a training sample set;
constructing a target detection convolutional neural network comprising a feature extraction layer, and extracting features of N-level region of interest samples in a training sample set in the feature extraction layer to obtain a feature map;
setting a plurality of preselection frames in the characteristic diagram, calculating the overlapping rate of a marked target in a region of interest sample in a training sample set and each preselection frame, training a target detection convolutional neural network by using the preselection frames and the training sample set, wherein the overlapping rate is greater than a preset value, obtaining a training result, calculating a loss value of the training result, and performing back propagation by using the loss value to update the parameters of the target detection convolutional neural network, thereby obtaining a target detection model. The height ratio of the preselection frame is greater than the target height ratio threshold, and the preset value is 0.5.
As shown in fig. 2, for the case that N is 3, acquiring a road scene image, and extracting three levels of regions of interest from the road scene image according to the order of the object distance from near to far; sequentially carrying out target detection on the three-level interested areas according to the sequence of the target distance from near to far, firstly carrying out first-level detection on the first-level interested area, if the target is detected, carrying out first-level alarm, otherwise carrying out second-level detection on the second-level interested area, if the target is detected, carrying out second-level alarm, otherwise carrying out third-level detection on the third-level interested area, if the target is detected, carrying out third-level alarm, and otherwise stopping detection.
And dividing the area on the driving track into three stages of interested areas which are respectively used for detecting near, middle and far targets. The method can accelerate the detection and early warning of the near target to the maximum extent, and can detect the target far away from the early warning at the same time.
As shown in fig. 3, for the case that N is 3, the setting of the three-level region of interest in step (1) specifically includes the following steps:
(21) analyzing the collected road scene image, and extracting the track and the intersection point of the lane line in the image; setting a target height ratio threshold Thr;
(22) analyzing the relation f between the pixel height of the target in the acquired image and the target distance;
(23) setting the distance a of the nearest detection target, and obtaining the pixel height H of the target at the moment according to the relation f1Is prepared from H1And as the length of the first-level interested area, calculating the width of the first-level interested area by using the length of the first-level interested area and the length-width ratio of the road scene image.
(24) Obtaining the minimum target pixel height H of the first-level region of interest detection according to the threshold value Thr2＝H1Thr. H is to be2And calculating the width of the second-level interested area by using the length of the second-level interested area and the length-width ratio of the road scene image as the length of the second-level interested area.
(25) From H2And obtaining the farthest distance b of the first-level interested region detection target according to the relation f, and simultaneously obtaining the nearest distance of the second-level interested region detection target.
(26) Obtaining the minimum target pixel height H of the second-level region-of-interest detection according to the threshold value Thr3＝H2Thr. H is to be3And as the length of the third-level interested area, calculating the width of the third-level interested area by using the length of the third-level interested area and the length-width ratio of the road scene image.
(27) From H3And obtaining the farthest distance of the second-level region of interest detection target according to the relation fAnd c is the nearest distance of the third-level region of interest detection target.
(28) Obtaining the minimum target pixel height H of the third-level region of interest detection according to the threshold Thr4＝H3*Thr。
(29) From H4And obtaining the farthest distance d of the third-level region of interest detection target according to the relation f.
(210) Taking the junction point as the center of each interested area, and sequentially intercepting the length of each interested area as H1，H2，H3As the first, second and third levels of interest.
The distances a, b, c and d in the steps (23), (25), (27) and (29) form target detection distance sections (a, b), (b, c) and (c, d) of the first, second and third levels of interest.
As shown in fig. 4, for the case where N is 3, the training method of the target detection model is as follows:
extracting three-level interesting regions from the sample road scene image to obtain input images, and marking a complete target with a height ratio larger than THR in each input image to obtain a training data set T; the marking method is that the coordinates of the upper left corner and the lower right corner of a rectangular box for recording and framing the target are recorded in an xml format file.
Constructing a target detection convolutional neural network, and extracting the characteristics of a training data set T on a characteristic layer of the target detection convolutional neural network;
as shown in fig. 5, designing a preselection frame on a feature layer of the target detection convolutional neural network, setting the size of the preselection frame to be slightly higher than Thr, and preliminarily obtaining the position of a target object with a high occupancy rate higher than Thr in an input image; preselected frame aspect ratio arIs 2, 3, 4; a preselected frame height of
Figure BDA0002053324390000091
Width is
Figure BDA0002053324390000092
And training the target detection convolutional neural network step by adopting a training data set T and a preselected frame arranged on the characteristic layer to obtain a target detection model.
As shown in fig. 6, during detection, an input image is input into a target feature extraction network to obtain a feature map, detection is performed to obtain a detection result, and subsequent processing is performed by using the detection result, that is, whether to perform early warning or not is determined.
Example 1
A road target rapid detection early warning method comprises the following steps when N is 3 and a target is a pedestrian:
step 1: collecting road scene images, analyzing the collected road scene images, and extracting tracks and junction points of lane lines in the images; and setting the center of the image as a lane line junction. Analyzing the relation f between the pixel height of the pedestrian in the collected image and the pedestrian distance; the detected pedestrian height ratio threshold Thr is set to 0.5.
The nearest detected pedestrian distance is set to 5 meters. And according to the relation f, the pixel height of the pedestrian at 5 m is 480, and the length and width of the first-level region of interest are 480. (the aspect ratio of the road scene image at this time is 1)
The first-level minimum pedestrian pixel height 240 to be detected is obtained by setting the length and width of the first-level interesting area sub-picture to be 480 and the set minimum pedestrian height ratio threshold value to be 0.5. At this time, the pedestrian distance is 10 meters, which is assumed to be the closest distance of the second-stage detection pedestrian. And the length and width of the second level region of interest are set to 240.
And obtaining the pixel height 120 of the minimum pedestrian to be detected at the second level by the length and the width of the sub-picture of the second-level interested area being 240 and the set minimum detected pedestrian height occupying ratio threshold value being 0.5. At this time, the pedestrian distance is 20 meters, which is assumed to be the closest distance of the third-level detected pedestrian. And the length and width of the second level region of interest are set to be 120.
The length and width of the third-level interested area sub-picture is 120, and the set minimum detected pedestrian height ratio threshold value is 0.5, so that the minimum pedestrian pixel height 60 to be detected at the third level is obtained. At this time, the pedestrian distance is 40 meters, which is the farthest distance of the third-level detection target.
Step 2: and intercepting all levels of interested areas containing pedestrians. The third-level interested region is the region obtained by the third-level interested region extraction method. And determining whether to mark the pedestrian according to whether the ratio of the pixel height of the pedestrian in the region of interest to the resolution height of the sub-picture is greater than Thr. Pedestrians with mark occupancy greater than Thr are noted. The labeling method is to record the coordinates of the upper left corner and the lower right corner of a rectangular frame for framing the pedestrian.
Designing a preselection frame on the characteristic layer L of the single-scale pedestrian detection convolutional neural network to preliminarily obtain the position of a pedestrian target;
selecting a part of feature layers behind the network, wherein the resolution of the feature layers is 3 multiplied by 3, and the dimension of the detected pedestrian is 0.7; the input image is divided into a 3 x 3 grid. The design principle of the pre-selection frame is concentric with the middle point of the grid area; preselected frame aspect ratio arIs 2, 3, 4; a preselected frame height of
Figure BDA0002053324390000101
Width is
Figure BDA0002053324390000102
And training the single-scale pedestrian detection convolutional neural network by adopting a training data set T and a preselection frame to obtain a pedestrian detection model.
And step 3: and sequentially carrying out pedestrian detection on the three-level regions of interest by utilizing a pedestrian detection model.
The invention relates to a road target rapid detection early warning method suitable for an auxiliary driving system. For detecting near, intermediate and far targets, respectively. The method can accelerate the detection and early warning of the near target to the maximum extent, and can detect the target far away from the early warning at the same time. And according to the strategy of the hierarchical detection, a rapid target detection network is designed. And performing target detection work by using the preselection frame which accords with the target characteristics and the single-scale characteristic diagram to obtain a rapid target detection network. The method realizes rapid and accurate detection and early warning of the target.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (8)

1. A road target rapid detection early warning method is characterized by comprising the following steps:
(1) acquiring a road scene image by using image acquisition equipment, and extracting N-level interested areas from the road scene image according to the sequence of the target distance from near to far;
(2) sequentially carrying out target detection on the N-level interested areas according to the sequence of the target distance from near to far, carrying out early warning after a certain-level interested area with a near target distance detects a target, and not detecting the subsequent-level interested areas;
wherein N is more than or equal to 2, and the target distance is the distance between the target and the image acquisition equipment;
the step (1) comprises the following steps:
collecting road scene images, setting a target height ratio threshold value Thr and a nearest target distance L0；
According to the relation f between the pixel height of the target and the target distance and the nearest target distance L0To obtain the pixel height H of the target1Is prepared from H1Calculating the width of the first-stage interested area by using the length of the first-stage interested area and the length-width ratio of the road scene image as the length of the first-stage interested area;
for the N-level region of interest, the target height is taken to be longer than the threshold Thr and the N-1 level region of interest HN-1Product of (A) and (B)NCalculating the width of the N-level interested area by using the length of the N-level interested area and the length-width ratio of the road scene image as the length of the N-level interested area;
obtaining the maximum distance L of the detection target of the N-1 level interested region according to the length and the width of the N level interested region and the relation f between the pixel height of the target and the target distanceN-1While L isN-1Is also the nearest of the N-th order region of interest detection targetA distance;
using the target height to make the ratio of the threshold Thr to the length H of the Nth order region of interestNThe maximum distance L of the detection target of the Nth-level interested area is obtained by the product of the maximum distance L and the maximum distance f of the targetN(ii) a The target distance detection range of the Nth-level interested region is (L)N-1,LN)。
2. The rapid detection and early warning method for road targets as claimed in claim 1, wherein the step (1) further comprises:
collecting road scene image sets at different target distances, acquiring pixel height of a target and discrete data of the distance between the target and the road scene image sets, and obtaining the relation f between the pixel height of the target and the distance between the target and the discrete data.
3. The method as claimed in claim 1, wherein the value range of the target height ratio threshold Thr is (0, 1).
4. The method as claimed in any one of claims 1 to 3, wherein the centers of the road scene image and the N-level region of interest are both intersection points of lane lines.
5. The rapid detection and early warning method for road targets as claimed in claim 1 or 3, wherein the specific implementation manner of the target detection in the step (2) is as follows:
carrying out target detection on the N-level region of interest by using a target detection model, wherein the training of the target detection model comprises the following steps:
extracting N-level interested area samples from the sample road scene image, and marking the target with the target height ratio larger than the target height ratio threshold value in each interested area sample to obtain a training sample set;
constructing a target detection convolutional neural network comprising a feature extraction layer, and extracting features of N-level region of interest samples in a training sample set in the feature extraction layer to obtain a feature map;
setting a plurality of preselection frames in the characteristic diagram, calculating the overlapping rate of a marked target in a region of interest sample in a training sample set and each preselection frame, training a target detection convolutional neural network by using the preselection frames and the training sample set, wherein the overlapping rate is greater than a preset value, obtaining a training result, calculating a loss value of the training result, and performing back propagation by using the loss value to update the parameters of the target detection convolutional neural network, thereby obtaining a target detection model.
6. The method as claimed in claim 5, wherein the height ratio of the pre-selection frame is greater than the target height ratio threshold.
7. The utility model provides a road target short-term test early warning system which characterized in that includes:
the interesting region extracting module is used for acquiring road scene images by utilizing image acquisition equipment and extracting N-level interesting regions from the road scene images according to the sequence of the target distance from near to far;
the target detection module is used for sequentially carrying out target detection on the N-level interested areas by using a target detection model according to the sequence of the target distance from near to far, carrying out early warning after a certain-level interested area with a near target distance detects a target, and not detecting the subsequent-level interested areas;
wherein N is more than or equal to 2, and the target distance is the distance between the target and the image acquisition equipment;
the region of interest extraction module comprises the following sub-modules:
an initialization submodule for acquiring road scene images, setting a target height ratio threshold value Thr and a nearest target distance L0；
A first-level interested region length and width calculation submodule used for calculating the length and the width of the interested region according to the relationship f between the pixel height of the target and the target distance and the distance L between the nearest target and the target0To obtain the pixel height H of the target1Is prepared from H1As a first stageThe length of the region of interest, and the width of the first-stage region of interest is calculated by utilizing the length of the first-stage region of interest and the length-width ratio of the road scene image;
an Nth-level interested region length and width calculation sub-module, which is used for comparing the target height with a threshold Thr and the length H of the N-1 level interested region for the Nth-level interested regionN-1Product of (A) and (B)NCalculating the width of the N-level interested area by using the length of the N-level interested area and the length-width ratio of the road scene image as the length of the N-level interested area;
a target distance detection range calculation submodule of the Nth level interested area, which is used for obtaining the farthest distance L of the detection target of the N-1 level interested area according to the length and the width of the Nth level interested area and the relation f between the pixel height of the target and the target distanceN-1While L isN-1The nearest distance of the detection target of the Nth-level region of interest; using the target height to make the ratio of the threshold Thr to the length H of the Nth order region of interestNThe maximum distance L of the detection target of the Nth-level interested area is obtained by the product of the maximum distance L and the maximum distance f of the targetN(ii) a The target distance detection range of the Nth-level interested region is (L)N-1,LN)。
8. The system of claim 7, wherein the training of the target detection model comprises:
the training sample set acquisition sub-module is used for extracting N-level interested area samples from the sample road scene image, and marking the target with the target height ratio larger than the target height ratio threshold value in each interested area sample to obtain a training sample set;
the characteristic diagram acquisition sub-module is used for constructing a target detection convolutional neural network comprising a characteristic extraction layer, and extracting the characteristics of N-level region of interest samples in the training sample set in the characteristic extraction layer to obtain a characteristic diagram;
the training submodule is used for setting a plurality of preselection frames in the characteristic diagram, calculating the overlapping rate of a marked target in an interested area sample in the training sample set and each preselection frame, training a target detection convolutional neural network by using the preselection frames with the overlapping rate larger than a preset value and the training sample set to obtain a training result, calculating the loss value of the training result, and performing back propagation by using the loss value to update the parameters of the target detection convolutional neural network to obtain a target detection model.
CN201910383340.XA 2019-05-08 2019-05-08 Road target rapid detection early warning method and system Active CN110210324B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910383340.XA CN110210324B (en) 2019-05-08 2019-05-08 Road target rapid detection early warning method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910383340.XA CN110210324B (en) 2019-05-08 2019-05-08 Road target rapid detection early warning method and system

Publications (2)

Publication Number Publication Date
CN110210324A CN110210324A (en) 2019-09-06
CN110210324B true CN110210324B (en) 2021-01-19

Family

ID=67785676

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910383340.XA Active CN110210324B (en) 2019-05-08 2019-05-08 Road target rapid detection early warning method and system

Country Status (1)

Country Link
CN (1) CN110210324B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111179533B (en) * 2020-01-11 2021-10-29 合肥源康信息科技有限公司 Target identification system based on shape analysis
CN114022860A (en) * 2020-07-16 2022-02-08 长沙智能驾驶研究院有限公司 Target detection method and device and electronic equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107066929A (en) * 2017-01-06 2017-08-18 重庆大学 The manifold freeway tunnel Parking hierarchical identification method of one kind fusion
CN107911697A (en) * 2017-10-30 2018-04-13 北京航空航天大学 Unmanned plane image motion object detection method based on area-of-interest layering
CN108399630A (en) * 2018-01-22 2018-08-14 北京理工雷科电子信息技术有限公司 Target fast ranging method in area-of-interest under a kind of complex scene

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107066929A (en) * 2017-01-06 2017-08-18 重庆大学 The manifold freeway tunnel Parking hierarchical identification method of one kind fusion
CN107911697A (en) * 2017-10-30 2018-04-13 北京航空航天大学 Unmanned plane image motion object detection method based on area-of-interest layering
CN108399630A (en) * 2018-01-22 2018-08-14 北京理工雷科电子信息技术有限公司 Target fast ranging method in area-of-interest under a kind of complex scene

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
Hierarchical region-of-interest detection;Huibao Lin等;《Optical Engineering 》;20060731;第45卷(第7期);第1-11页 *
KNOWLEDGE-BASED HIERARICAL REGION-OF-INEREST DETECTION;Huibao Lin等;《2002 IEEE International Conference on Acoustics,Speech,and Signal Processing》;20110407;第3628-3631页 *
Once for All: a Two-flow Convolutional Neural Network for Visual Tra cking;Kai Chen等;《arXiv:1604.07507v1 [cs.CV]》;20160426;第1-15页 *
SDBD: A Hierarchical Region-of-Interest Detection Approach in Large-Scale Remote Sensing Image;Tong Li等;《IEEE GEOSCIENCE AND REMOTE SENSING LETTERS》;20170531;第14卷(第5期);第699-703页 *
前方道路行人检测和距离估计研究;张治国;《中国博士学位论文全文数据库 工程科技Ⅱ辑》;20181015(第10期);C035-6 *
基于DM8127的多目标远距离检测定位系统;向根;《中国优秀硕士学位论文全文数据库 信息科技辑》;20170215(第2期);I138-2972 *

Also Published As

Publication number Publication date
CN110210324A (en) 2019-09-06

Similar Documents

Publication Publication Date Title
CN106919915B (en) Map road marking and road quality acquisition device and method based on ADAS system
CN110619279B (en) Road traffic sign instance segmentation method based on tracking
CN110415277B (en) Multi-target tracking method, system and device based on optical flow and Kalman filtering
CN105321342B (en) A kind of intersection vehicles queue length detection method based on video of taking photo by plane
CN113421289B (en) High-precision vehicle track data extraction method for overcoming unmanned aerial vehicle shooting disturbance
US10984263B2 (en) Detection and validation of objects from sequential images of a camera by using homographies
CN111340855A (en) Road moving target detection method based on track prediction
CN110348332B (en) Method for extracting multi-target real-time trajectories of non-human machines in traffic video scene
CN109727273B (en) Moving target detection method based on vehicle-mounted fisheye camera
CN107909012B (en) Real-time vehicle tracking detection method and device based on disparity map
CN107729843B (en) Low-floor tramcar pedestrian identification method based on radar and visual information fusion
CN104239867A (en) License plate locating method and system
CN111915583B (en) Vehicle and pedestrian detection method based on vehicle-mounted thermal infrared imager in complex scene
CN110458050B (en) Vehicle cut-in detection method and device based on vehicle-mounted video
JP6700373B2 (en) Apparatus and method for learning object image packaging for artificial intelligence of video animation
CN111091023A (en) Vehicle detection method and device and electronic equipment
CN106845458B (en) Rapid traffic sign detection method based on nuclear overrun learning machine
CN110210324B (en) Road target rapid detection early warning method and system
CN112990065A (en) Optimized YOLOv5 model-based vehicle classification detection method
CN107506753B (en) Multi-vehicle tracking method for dynamic video monitoring
CN110443142B (en) Deep learning vehicle counting method based on road surface extraction and segmentation
CN113361528B (en) Multi-scale target detection method and system
US11087150B2 (en) Detection and validation of objects from sequential images of a camera by using homographies
JP6681965B2 (en) Apparatus and method for extracting learning target image for autonomous driving
CN103258433B (en) Intelligent clear display method for number plates in traffic video surveillance

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant