CN114187328A - Object detection method and device and electronic equipment - Google Patents
Object detection method and device and electronic equipment Download PDFInfo
- Publication number
- CN114187328A CN114187328A CN202210135826.3A CN202210135826A CN114187328A CN 114187328 A CN114187328 A CN 114187328A CN 202210135826 A CN202210135826 A CN 202210135826A CN 114187328 A CN114187328 A CN 114187328A
- Authority
- CN
- China
- Prior art keywords
- detected
- radar
- detection target
- radar detection
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
Abstract
The invention discloses an object detection method, an object detection device and electronic equipment. The object detection method comprises the following steps: acquiring space-time synchronous radar data and image data in a monitoring area through roadside radar vision equipment; obtaining a radar detection target through radar data, and correcting the number of objects to be detected corresponding to the radar detection target by using image data; determining whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time; if the radar detection target does not appear for the first time, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result; and if the radar detection target appears for the first time, extracting the appearance characteristics of the object to be detected through image data, and obtaining the detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target. The technical scheme of the invention can reduce the consumption of computational resources under the condition of ensuring the detection precision.
Description
Technical Field
The invention relates to the technical field of intelligent traffic management, in particular to an object detection method, an object detection device and electronic equipment.
Background
At present, monitoring technology based on radar and video all-in-one machine is more and more emphasized in the field of intelligent traffic. The radar obtains the measurement information (spatial position and motion speed information) of the moving target with high detection probability, but the radar cannot obtain higher target identification rate; the video or the image can obtain the target identification information with high accuracy, but the motion information and the spatial position information of the target are not easy to obtain. If the radar and the video data are effectively fused, higher target identification accuracy, motion information and spatial position information can be obtained, so that the radar and video integrated machine (also called radar vision equipment) is widely applied.
In an intelligent traffic management scene, roadside radar equipment is an intelligent sensor specially designed for vehicle-road cooperation, and is a traffic sensor integrating a camera, a millimeter wave radar and a high-performance processor. The method comprises the steps that original video streams and radar data streams are simultaneously accessed into a Processor built in the roadside radar vision device through an MIPI (Mobile Industry Processor Interface) and an SPI (Serial Peripheral Interface), the built-in Processor is used for directly processing the original video streams and the radar data, fusion tracking is carried out on video targets and radar targets, and real-time monitoring of the roadside targets is achieved.
Research shows that in the present stage, a data fusion tracking algorithm based on the roadside radar vision equipment generally consumes more computational resources to obtain higher detection precision, and a processor of the roadside radar vision equipment is required to have extremely high computational performance.
Disclosure of Invention
In view of the above, the present invention provides an object detection method, an object detection apparatus, and an electronic device, which are used to reduce consumption of computational resources while ensuring detection accuracy.
According to a first aspect of the present invention, there is provided a detection method comprising: acquiring space-time synchronous radar data and image data in a monitoring area through roadside radar vision equipment; obtaining a radar detection target through radar data, and correcting the number of objects to be detected corresponding to the radar detection target by using image data to enable the corrected radar detection target to correspond to one object to be detected; determining whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time; if the object to be detected does not appear for the first time, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result; and if the object to be detected appears for the first time, extracting the appearance characteristics of the object to be detected through image data, and obtaining the detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target.
In some embodiments, the correcting the number of objects to be detected corresponding to the radar detection target by using the image data so that the corrected radar detection target corresponds to one object to be detected includes: determining whether the radar detection target corresponds to a plurality of objects to be detected or not by using the image data; and when the radar detection target corresponds to a plurality of objects to be detected, correcting the number of the objects to be detected corresponding to the radar detection target by using the image data.
In some embodiments, determining whether the radar detection target corresponds to a plurality of objects to be detected using the image data includes: intercepting an image area corresponding to the radar detection target from image data to obtain a sub-image; performing target classification on the sub-images by using a classification network; and if a plurality of classification results are obtained, determining that the radar detection target corresponds to a plurality of objects to be detected.
In some embodiments, when the radar detection target corresponds to a plurality of objects to be detected, correcting the number of the objects to be detected corresponding to the radar detection target by using the image data includes: dividing the radar detection target into a plurality of targets according to the position information of each classification result; and correcting the contour of the corresponding target according to the contour information of each classification result to obtain a corrected radar detection target.
In some embodiments, determining whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time includes: acquiring a historical detection result of the detected object, and determining a historical track of the detected object according to the historical detection result; determining whether a historical track matched with the object to be detected exists in the historical track or not according to the corrected measurement information and time information of the radar detection target; and if the matched historical track exists, determining that the object to be detected does not appear for the first time, and if the matched historical track does not exist, determining that the object to be detected appears for the first time.
In some embodiments, if the object to be detected appears for the first time, extracting the appearance feature of the object to be detected through image data, and obtaining the detection result of the object to be detected according to the corrected measurement information and appearance feature of the radar detection target, the method includes: judging whether the object to be detected is an original target recaptured after the roadside radar vision equipment is lost or not according to the appearance characteristics of the object to be detected; if the object to be detected is the original target, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result; and if the object to be detected is a new target, constructing a detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target.
In some embodiments, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result, including: predicting the spatial position of the object to be detected according to the movement speed in the measurement information and the historical track of the object to be detected; and adjusting the spatial position in the measurement information according to the predicted spatial position to obtain an updated detection result.
In some embodiments, determining whether the object to be detected is an original target recaptured after the roadside radar vision equipment is lost according to the appearance characteristics of the object to be detected includes: acquiring a historical detection result of a detected object, wherein the historical detection result comprises the appearance characteristic of the detected object; and if the appearance characteristics of a certain detected object in the historical detection result are the same as those of the object to be detected, determining that the object to be detected is the original target recaptured after the roadside radar vision equipment is lost.
According to a second aspect of the present invention, there is provided an object detecting apparatus comprising: the data acquisition unit is used for acquiring space-time synchronous radar data and image data in a monitoring area through roadside radar equipment; the first processing unit is used for obtaining a radar detection target through radar data, and correcting the number of objects to be detected corresponding to the radar detection target by using image data so that the corrected radar detection target corresponds to one object to be detected; the second processing unit is used for determining whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time; if the object to be detected does not appear for the first time, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result; and if the object to be detected appears for the first time, extracting the appearance characteristics of the object to be detected through image data, and obtaining the detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target.
According to a third aspect of the invention, there is provided an electronic device comprising a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the above object detection method.
According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing one or more programs which, when executed by a processor, implement the above-described object detection method.
The invention adopts at least one technical scheme which can achieve the following beneficial effects: the object detection method, the device and the electronic equipment in the embodiment of the invention are based on radar detection, when the radar detection target is obtained, the number of the objects to be detected corresponding to the radar detection target is verified by using the image data, and when the radar detection identifies a plurality of objects as one target by mistake, the embodiment corrects the number of the objects of the radar detection target by using the image data, so that the corrected radar detection target corresponds to one object to be detected, and the detection precision is further improved. In addition, the number of the objects to be detected corresponding to the radar detection target is verified, the image data can be used for verifying the number of the targets only by classifying the image data through the classification network, and target detection and target identification are not required to be carried out on the image data, so that the calculation force requirement can be remarkably reduced.
When the radar detection target is obtained, the object detection method, the device and the electronic equipment further verify whether the object to be detected appears in the monitoring area of the roadside radar vision equipment for the first time, the detection result of the object to be detected can be obtained only through the measurement information of radar data for the object to be detected which does not appear for the first time, feature extraction is carried out on image data only through the object to be detected which appears for the first time, and the detection result of the object to be detected is obtained by combining the appearance feature obtained from the image data and the measurement information obtained from the radar data. Since the present embodiment of the present invention performs feature extraction only once for a new target, the computational power requirement can be further reduced.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 shows a flow diagram of an object detection method according to an embodiment of the invention;
FIG. 2 shows a flow chart of an object detection method based on a roadside radar vision device according to one embodiment of the invention;
FIG. 3 is a block diagram of an object detection apparatus according to an embodiment of the present invention;
fig. 4 shows a schematic structural diagram of an electronic device according to an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein.
Fig. 1 shows a flowchart of an object detection method according to an embodiment of the present invention, and as shown in fig. 1, the method of the embodiment at least includes steps S110 to S150:
and step S110, acquiring space-time synchronous radar data and image data in the monitored area through roadside radar vision equipment.
The object detection method of the embodiment of the invention can be realized by a processor in the roadside radar equipment, and when the target detection is carried out, the space-time synchronous radar data and image data are required to be obtained firstly.
The roadside radar vision equipment comprises a millimeter wave radar and a camera, and the time-space synchronization refers to time synchronization and space synchronization.
In the calibration stage of the roadside radar vision equipment, the millimeter wave radar and the camera are jointly calibrated to obtain the relative posture between the millimeter wave radar and the camera, so that coordinate points between the millimeter wave radar and the camera can directly correspond to realize the spatial synchronization between the millimeter wave radar and the camera.
And in the initialization stage of the roadside radar video equipment, time alignment is carried out on the millimeter wave radar and the camera, so that time synchronization of the millimeter wave radar and the camera is realized.
And step S120, obtaining a radar detection target through the radar data, and correcting the number of the objects to be detected corresponding to the radar detection target by using the image data so that the corrected radar detection target corresponds to one object to be detected.
According to the method and the device, the target detection can be performed on the radar data by utilizing a machine learning algorithm, information such as a target profile, a target type, a target position and a target speed is recognized, and a radar detection target is obtained.
In practical applications, the millimeter wave radar may misrecognize a plurality of objects that are too close to each other as a radar detection target. And the image data has a target identification capability with higher accuracy. Therefore, when the radar detection target is obtained through the radar data, whether each radar detection target corresponds to a plurality of objects to be detected is further judged, and when the radar detection target corresponds to a plurality of objects to be detected, the target number of the radar detection target is corrected by using the image data, so that the corrected radar detection target corresponds to one object to be detected.
Because the position accuracy of the image detection result is far inferior to that of the radar detection result, the radar detection target corresponds to an object to be detected, and the spatial position of the radar detection target can not be corrected. Of course, if the position accuracy of the image detection result meets the requirement, when the radar detection target corresponds to an object to be detected, the position and the contour of the radar detection target can be corrected by using the image data.
And step S130, determining whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time.
The processor in the embodiment of the present application stores historical detection results, for example, historical detection results of each detected object in a certain period of time, where the historical detection results include historical trajectories of the detected objects.
When judging whether the object to be detected corresponding to the radar detection target at the current moment appears for the first time, judging whether the object to be detected appears for the first time according to the historical track of the detected object. For example, whether a historical track matched with the object to be detected exists in the historical tracks or not is judged according to the measurement information of the radar detection target, if yes, the object to be detected is determined not to appear for the first time, otherwise, if not, the object to be detected is determined to appear for the first time. The measured information refers to the spatial position and the motion speed obtained by radar data.
And step S140, if the object to be detected does not appear for the first time, updating the historical detection result of the object to be detected according to the corrected radar detection target to obtain an updated detection result.
And S150, if the object to be detected appears for the first time, extracting the appearance characteristics of the object to be detected through image data, and obtaining the detection result of the object to be detected according to the corrected radar detection target and the appearance characteristics.
In this embodiment, when it is determined that the object to be detected does not appear in the monitoring area for the first time, feature extraction of image data is not required, and only the measurement information of the radar detection target of the object to be detected is used to update the historical detection result of the object to be detected. And only when the object to be detected appears in the monitoring area for the first time, feature extraction is carried out on the image data to obtain the appearance features of the object to be detected, and the detection result of the object to be detected is obtained by combining the appearance features and the measurement information. The detection result of the present embodiment includes measurement information such as a spatial position and a movement speed obtained from radar data, and appearance characteristics such as a color and a contour obtained from image data.
It can be seen that, in the method shown in fig. 1, based on radar detection, when a radar detection target is obtained, on one hand, the number of objects to be detected corresponding to the radar detection target is verified by using image data, and when a plurality of objects are mistakenly identified as one target by radar detection, the number of targets of the radar detection target is corrected by using the image data in the embodiment, so that the corrected radar detection target corresponds to one object to be detected, and further, the detection precision is improved. In addition, the number of the objects to be detected corresponding to the radar detection target is verified, the image data can be used for verifying the number of the targets only by classifying the image data through the classification network, and target detection and target identification are not required to be carried out on the image data, so that the calculation force requirement can be remarkably reduced.
On the other hand, whether the object to be detected appears in the monitoring area of the roadside radar vision equipment for the first time or not is verified, the detection result of the object to be detected can be obtained only through the measurement information of radar data for the object to be detected which does not appear for the first time, feature extraction is carried out on the image data only through the object to be detected which appears for the first time, and the detection result of the object to be detected is obtained by combining the appearance feature obtained from the image data and the measurement information obtained from the radar data. Since the present embodiment of the present invention performs feature extraction only once for a new target, the computational power requirement can be further reduced.
In some embodiments, the correcting the number of objects to be detected corresponding to the radar detection target by using the image data so that the corrected radar detection target corresponds to one object to be detected includes:
determining whether the radar detection target corresponds to a plurality of objects to be detected or not by using the image data; and when the radar detection target corresponds to a plurality of objects to be detected, correcting the number of the objects to be detected corresponding to the radar detection target by using the image data.
For example, through joint calibration, an image area corresponding to a radar detection target is intercepted from image data to obtain a sub-image, the sub-image is subjected to target classification by using a classification network, if a plurality of classification results are obtained, it is determined that the radar detection target corresponds to a plurality of objects to be detected, and if one classification result is obtained, it is determined that the radar detection target corresponds to one object to be detected.
When the radar detection target corresponds to a plurality of objects to be detected, the radar detection target is divided into a plurality of targets according to the position information of each classification result, and the outline of the corresponding target is corrected according to the outline information of each classification result, so that the corrected radar detection target is obtained.
Certainly, in some embodiments, when the radar detection target is obtained through the radar data, the step of determining the number of objects to be detected corresponding to the radar detection target may not be performed, the sub-image corresponding to the radar detection target is directly intercepted from the image data, the sub-image is subjected to target classification by using a classification network, and the radar detection target is corrected according to the position information and the contour information of the classification result, so that the corrected radar detection target corresponds to one object to be detected.
When it is determined that the corrected radar detection target corresponds to an object to be detected, whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time can be judged.
The roadside radar vision device of the embodiment stores historical detection results of a detected object, wherein the historical detection results comprise historical tracks of the detected object and appearance characteristics of the detected object. The historical track comprises sampling point positions, sampling time, speed and the like, and the data information of the sampling point forms track data according to the sampling sequence. The sampling points in this embodiment are, for example, point clouds of objects in each frame of radar data.
In one embodiment, whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time is judged according to the historical track of the detected object:
acquiring a historical detection result of the detected object, and determining a historical track of the detected object according to the historical detection result; determining whether a historical track matched with the object to be detected exists in the historical track or not according to the corrected measurement information and time information of the radar detection target; for example, according to the motion speed and time information in the measurement information, the next sampling point position of the historical track of each detected object is predicted, if the deviation between the predicted sampling point position and the spatial position in the measurement information is in a reasonable range, a matched historical track is considered to exist, and if the deviation is large, a matched historical track does not exist.
And when the matched historical track exists, determining that the object to be detected does not appear for the first time, and when the matched historical track does not exist, determining that the object to be detected appears for the first time.
In practical application, when it is determined that the object to be detected appears in the monitored area for the first time through the above embodiment, it may be that the original target is lost by the millimeter wave radar due to environmental interference and other problems, and therefore, it should be further verified whether the object to be detected is an object recaptured after the tracking is lost.
In some embodiments, if the object to be detected appears for the first time, whether the object to be detected is an original target recaptured after the roadside radar vision equipment is lost is judged according to the appearance characteristics of the object to be detected; if the object to be detected is the original target, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result; and if the object to be detected is a new target, constructing a detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target.
The embodiment of the invention further verifies whether the radar detection target is an original target recaptured after the tracking loss according to the appearance characteristics, and when the radar detection target is determined to be the original target recaptured, the spatial position of the object to be detected is predicted according to the motion speed in the measurement information and the historical track of the object to be detected; and adjusting the spatial position in the measurement information according to the predicted spatial position to obtain an updated detection result.
In this embodiment, judging whether the object to be detected is an original target recaptured after the roadside radar vision equipment is lost according to the appearance characteristics of the object to be detected includes:
acquiring a historical detection result of a detected object, wherein the historical detection result comprises the appearance characteristic of the detected object; and if the appearance characteristics of a certain detected object in the historical detection result are the same as those of the object to be detected, determining that the object to be detected is the original target recaptured after the roadside radar vision equipment is lost.
For example, when the Set of appearance features of the detected object 1 in the historical detection result is Set1= { color, contour, type }, if the Set of appearance features obtained through the image data is Set2= { color, contour }, if the Set2 is a subset of the Set1, it can be determined that the appearance features of the detected object are the same as those of the detected object 1, and at this time, it is determined that the detected object is the detected object 1 recaptured after the roadside radar device is lost.
For the understanding of the above embodiments of the present invention, the following describes the detection method according to the embodiment of the present invention with reference to fig. 2.
As shown in fig. 2, when a radar detection target is obtained, the embodiment of the present invention verifies whether the radar detection target corresponds to a plurality of objects to be detected through an image detection target, and when it is determined that the radar detection target corresponds to the plurality of objects to be detected, corrects the number, position, and contour of the radar detection target by using the image detection target to obtain a corrected radar detection target, where the corrected radar detection target corresponds to one object to be detected.
Here, the image detection target is a classification result obtained by classifying the subgraph by using a neural classification network, and each classification result corresponds to one image detection target.
And then, judging whether the corrected radar detection target appears in the monitoring area for the first time by combining the track information. If the radar detection target appears for the first time, further combining the image data to verify whether the radar detection target is the original target recaptured after being lost, and if the radar detection target is determined to be the original target recaptured after being lost, the appearance characteristics of the object to be detected are the same as those in the historical detection result, so that the spatial position in the historical detection result is updated only by using the measurement information of the radar detection target.
In practical application, all data of each detected object can be saved as a historical detection result, so as to provide various data for various application requirements. For example, a data container is created for each detected object, and data such as radar data, image data, measurement information, appearance characteristics, and historical tracks are stored using the data container. When the object to be detected is determined to be a new target, a data container can be created for the new target in the cache of the roadside radar vision device to store all data of the new target. Therefore, in the next round of detection process, whether the radar detection target appears in the monitoring area for the first time or not and whether the radar detection target is the original target which is recaptured after being lost or not are judged based on historical data in all data containers in the cache.
In summary, in the embodiment of the invention, the radar detection and the image detection are matched, and the accuracy of the radar detection result is verified for multiple times by using the image detection result, so that the accuracy of the detection result can be effectively improved, the problem that the millimeter wave radar mistakenly identifies multiple problems with too close distance as one problem is avoided, and the problems that a camera is difficult to detect a shielding object and the target detection mainly based on an image is difficult to accurately position the spatial position of the target are avoided. In addition, the embodiment of the invention only needs to carry out feature extraction and target detection on the new target, and only needs to carry out classification processing on the original target by utilizing the neural classification network, so that the computational power requirement can be greatly reduced under the condition of ensuring the accuracy. The system stability of the roadside radar vision equipment can be ensured, and the equipment cost is reduced.
The present invention also provides an object detection apparatus, which can implement the object detection method in the foregoing embodiments.
Fig. 3 is a block diagram illustrating a structure of an object detecting apparatus according to an embodiment of the present invention, and as shown in fig. 3, the detecting apparatus 300 includes:
the data acquisition unit 310 is used for acquiring space-time synchronous radar data and image data in a monitoring area through roadside radar equipment;
the first processing unit 320 is configured to obtain a radar detection target through radar data, and correct the number of objects to be detected corresponding to the radar detection target by using image data, so that the corrected radar detection target corresponds to one object to be detected;
the second processing unit 330 is configured to determine whether an object to be detected corresponding to the corrected radar detection target first appears in the monitored area; if the object to be detected does not appear for the first time, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result; and if the object to be detected appears for the first time, extracting the appearance characteristics of the object to be detected through image data, and obtaining the detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target.
In some embodiments, the first processing unit 320 includes a first determining module, a modifying module, and a classifying module;
the first judgment module is used for determining whether the radar detection target corresponds to a plurality of objects to be detected or not by utilizing the image data;
and the correction module is used for correcting the number of the objects to be detected corresponding to the radar detection target by using the image data when the radar detection target corresponds to a plurality of objects to be detected.
In some embodiments, the classification module is configured to intercept an image region corresponding to the radar detection target from image data to obtain a sub-image; performing target classification on the sub-images by using a classification network;
and the first judgment module is used for determining that the radar detection target corresponds to a plurality of objects to be detected if a plurality of classification results are obtained.
In some embodiments, the correction module is configured to divide the radar detection target into a plurality of targets according to the position information of each classification result; and correcting the contour of the corresponding target according to the contour information of each classification result to obtain a corrected radar detection target.
In some embodiments, the second processing unit 330 includes a second determining module, a third determining module, a first obtaining module, and a second obtaining module;
the second judgment module is used for acquiring a historical detection result of the detected object and determining a historical track of the detected object according to the historical detection result; determining whether a historical track matched with the object to be detected exists in the historical track or not according to the corrected measurement information and time information of the radar detection target; and if the matched historical track exists, determining that the object to be detected does not appear for the first time, and if the matched historical track does not exist, determining that the object to be detected appears for the first time.
The third judgment module is used for judging whether the object to be detected is an original target recaptured after the roadside radar vision equipment is lost or not according to the appearance characteristics of the object to be detected if the object to be detected appears for the first time;
the first acquisition module is used for updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result if the object to be detected is the original target;
and the second acquisition module is used for constructing a detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target if the object to be detected is a new target.
In some embodiments, the first obtaining module is configured to predict a spatial position of the object to be detected according to a motion speed in the measurement information and a historical track of the object to be detected; and adjusting the spatial position in the measurement information according to the predicted spatial position to obtain an updated detection result.
In some embodiments, the third determining module is configured to obtain a historical detection result of the detected object, where the historical detection result includes an appearance characteristic of the detected object; and if the appearance characteristics of a certain detected object in the historical detection result are the same as those of the object to be detected, determining that the object to be detected is the original target recaptured after the roadside radar vision equipment is lost.
It can be understood that the object detection device can implement the steps of the object detection method provided in the foregoing embodiment, and the related explanations about the object detection method are applicable to the object detection device, and are not repeated herein.
It should be noted that:
FIG. 4 shows a schematic diagram of an electronic device according to one embodiment of the invention. Referring to fig. 4, at a hardware level, the electronic device includes a processor and a memory, and optionally further includes an internal bus and a network interface. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the interface module, the communication module, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 4, but that does not indicate only one bus or one type of bus.
A memory for storing computer executable instructions. The memory provides computer executable instructions to the processor through the internal bus.
A processor executing computer executable instructions stored in the memory and specifically configured to perform the following operations:
acquiring space-time synchronous radar data and image data in a monitoring area through roadside radar vision equipment;
obtaining a radar detection target through radar data, and correcting the number of objects to be detected corresponding to the radar detection target by using image data to enable the corrected radar detection target to correspond to one object to be detected;
determining whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time;
if the object to be detected does not appear for the first time, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result;
and if the object to be detected appears for the first time, extracting the appearance characteristics of the object to be detected through image data, and obtaining the detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target.
The functions performed by the object detection method disclosed in the embodiment of fig. 1 may be implemented in or by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software.
Embodiments of the present invention also provide a computer-readable storage medium storing one or more programs which, when executed by a processor, implement the object detection method provided in the foregoing embodiments.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention.
The above are merely examples of the present invention, and are not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (10)
1. An object detection method, comprising:
acquiring space-time synchronous radar data and image data in a monitoring area through roadside radar vision equipment;
obtaining a radar detection target through radar data, and correcting the number of objects to be detected corresponding to the radar detection target by using image data to enable the corrected radar detection target to correspond to one object to be detected;
determining whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time;
if the object to be detected does not appear for the first time, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result;
and if the object to be detected appears for the first time, extracting the appearance characteristics of the object to be detected through image data, and obtaining the detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target.
2. The method of claim 1, wherein the step of correcting the number of objects to be detected corresponding to the radar detection target by using the image data so that the corrected radar detection target corresponds to one object to be detected comprises:
determining whether the radar detection target corresponds to a plurality of objects to be detected or not by using the image data;
and when the radar detection target corresponds to a plurality of objects to be detected, correcting the number of the objects to be detected corresponding to the radar detection target by using the image data.
3. The method of claim 2, wherein determining whether the radar detection target corresponds to the plurality of objects to be detected using the image data comprises:
intercepting an image area corresponding to the radar detection target from image data to obtain a sub-image;
performing target classification on the sub-images by using a classification network;
and if a plurality of classification results are obtained, determining that the radar detection target corresponds to a plurality of objects to be detected.
4. The method of claim 3, wherein when the radar detection target corresponds to a plurality of objects to be detected, correcting the number of the objects to be detected corresponding to the radar detection target by using the image data comprises:
dividing the radar detection target into a plurality of targets according to the position information of each classification result;
and correcting the contour of the corresponding target according to the contour information of each classification result to obtain a corrected radar detection target.
5. The method of claim 1, wherein determining whether the object to be detected corresponding to the modified radar detection target is first present in the monitored area comprises:
acquiring a historical detection result of the detected object, and determining a historical track of the detected object according to the historical detection result;
determining whether a historical track matched with the object to be detected exists in the historical track or not according to the corrected measurement information and time information of the radar detection target;
and if the matched historical track exists, determining that the object to be detected does not appear for the first time, and if the matched historical track does not exist, determining that the object to be detected appears for the first time.
6. The method of claim 1, wherein if the object to be detected appears for the first time, extracting appearance features of the object to be detected through image data, and obtaining a detection result of the object to be detected according to the corrected measurement information and appearance features of the radar detection target, comprises:
judging whether the object to be detected is an original target recaptured after the roadside radar vision equipment is lost or not according to the appearance characteristics of the object to be detected;
if the object to be detected is the original target, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result;
and if the object to be detected is a new target, constructing a detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target.
7. The method according to claim 1 or 6, wherein updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result comprises:
predicting the spatial position of the object to be detected according to the movement speed in the measurement information and the historical track of the object to be detected;
and adjusting the spatial position in the measurement information according to the predicted spatial position to obtain an updated detection result.
8. The method according to claim 6, wherein judging whether the object to be detected is an original target recaptured after the roadside radar vision equipment is lost according to the appearance characteristics of the object to be detected comprises:
acquiring a historical detection result of a detected object, wherein the historical detection result comprises the appearance characteristic of the detected object;
and if the appearance characteristics of a certain detected object in the historical detection result are the same as those of the object to be detected, determining that the object to be detected is the original target recaptured after the roadside radar vision equipment is lost.
9. An object detecting device, comprising:
the data acquisition unit is used for acquiring space-time synchronous radar data and image data in a monitoring area through roadside radar equipment;
the first processing unit is used for obtaining a radar detection target through radar data, and correcting the number of objects to be detected corresponding to the radar detection target by using image data so that the corrected radar detection target corresponds to one object to be detected;
the second processing unit is used for determining whether the object to be detected corresponding to the corrected radar detection target appears in the monitoring area for the first time; if the object to be detected does not appear for the first time, updating the historical detection result of the object to be detected according to the corrected measurement information of the radar detection target to obtain an updated detection result; and if the object to be detected appears for the first time, extracting the appearance characteristics of the object to be detected through image data, and obtaining the detection result of the object to be detected according to the corrected measurement information and appearance characteristics of the radar detection target.
10. An electronic device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the object detection method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210135826.3A CN114187328B (en) | 2022-02-15 | 2022-02-15 | Object detection method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210135826.3A CN114187328B (en) | 2022-02-15 | 2022-02-15 | Object detection method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114187328A true CN114187328A (en) | 2022-03-15 |
| CN114187328B CN114187328B (en) | 2022-07-05 |
Family
ID=80545963
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210135826.3A Active CN114187328B (en) | 2022-02-15 | 2022-02-15 | Object detection method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114187328B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115131748A (en) * | 2022-08-26 | 2022-09-30 | 广州市德赛西威智慧交通技术有限公司 | Method and system for improving target tracking and identifying accuracy of radar-vision all-in-one machine |
Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104866823A (en) * | 2015-05-11 | 2015-08-26 | 重庆邮电大学 | Vehicle detection and tracking method based on monocular vision |
| CN107609522A (en) * | 2017-09-19 | 2018-01-19 | 东华大学 | A kind of information fusion vehicle detecting system based on laser radar and machine vision |
| CN109212514A (en) * | 2018-09-29 | 2019-01-15 | 河北德冠隆电子科技有限公司 | A kind of detections of radar equipment persistently tracks correlating method to movement and static target |
| CN109343050A (en) * | 2018-11-05 | 2019-02-15 | 浙江大华技术股份有限公司 | A kind of radar video monitoring method and device |
| CN110221286A (en) * | 2019-05-30 | 2019-09-10 | 安徽四创电子股份有限公司 | Adaptive targets detection method based on ground surveillance radar |
| CN110572577A (en) * | 2019-09-24 | 2019-12-13 | 浙江大华技术股份有限公司 | Method, device, equipment and medium for tracking and focusing |
| CN110598730A (en) * | 2019-07-29 | 2019-12-20 | 南京理工大学 | Flight path association algorithm based on decision tree |
| CN110632589A (en) * | 2019-10-17 | 2019-12-31 | 安徽大学 | Radar photoelectric information fusion technology |
| CN111856445A (en) * | 2019-04-11 | 2020-10-30 | 杭州海康威视数字技术股份有限公司 | Target detection method, device, equipment and system |
| CN112017431A (en) * | 2020-07-29 | 2020-12-01 | 河北德冠隆电子科技有限公司 | Active vehicle continuous tracking and positioning system and method based on multi-data fusion |
| CN112154444A (en) * | 2019-10-17 | 2020-12-29 | 深圳市大疆创新科技有限公司 | Target detection and tracking method, system, movable platform, camera and medium |
| CN112233097A (en) * | 2020-10-19 | 2021-01-15 | 中国科学技术大学 | Road scene other vehicle detection system and method based on space-time domain multi-dimensional fusion |
| CN112906777A (en) * | 2021-02-05 | 2021-06-04 | 北京邮电大学 | Target detection method and device, electronic equipment and storage medium |
| CN112991391A (en) * | 2021-03-31 | 2021-06-18 | 武汉大学 | Vehicle detection and tracking method based on radar signal and vision fusion |
| US20210231794A1 (en) * | 2020-01-28 | 2021-07-29 | Aptiv Technologies Limited | Imaging and Radar Fusion for Multiple-Object Tracking |
| US20210241026A1 (en) * | 2020-02-04 | 2021-08-05 | Nio Usa, Inc. | Single frame 4d detection using deep fusion of camera image, imaging radar and lidar point cloud |
| US20210287035A1 (en) * | 2020-03-10 | 2021-09-16 | Gm Cruise Holdings Llc | Image and lidar segmentation for lidar-camera calibration |
| CN113658256A (en) * | 2021-08-16 | 2021-11-16 | 智道网联科技(北京)有限公司 | Target detection method and device based on laser radar and electronic equipment |
| CN113903029A (en) * | 2021-12-10 | 2022-01-07 | 智道网联科技(北京)有限公司 | Method and device for marking 3D frame in point cloud data |
| CN113989755A (en) * | 2021-10-22 | 2022-01-28 | 智道网联科技(北京)有限公司 | Method, apparatus and computer readable storage medium for identifying an object |
-
2022
- 2022-02-15 CN CN202210135826.3A patent/CN114187328B/en active Active
Patent Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104866823A (en) * | 2015-05-11 | 2015-08-26 | 重庆邮电大学 | Vehicle detection and tracking method based on monocular vision |
| CN107609522A (en) * | 2017-09-19 | 2018-01-19 | 东华大学 | A kind of information fusion vehicle detecting system based on laser radar and machine vision |
| CN109212514A (en) * | 2018-09-29 | 2019-01-15 | 河北德冠隆电子科技有限公司 | A kind of detections of radar equipment persistently tracks correlating method to movement and static target |
| CN109343050A (en) * | 2018-11-05 | 2019-02-15 | 浙江大华技术股份有限公司 | A kind of radar video monitoring method and device |
| CN111856445A (en) * | 2019-04-11 | 2020-10-30 | 杭州海康威视数字技术股份有限公司 | Target detection method, device, equipment and system |
| CN110221286A (en) * | 2019-05-30 | 2019-09-10 | 安徽四创电子股份有限公司 | Adaptive targets detection method based on ground surveillance radar |
| CN110598730A (en) * | 2019-07-29 | 2019-12-20 | 南京理工大学 | Flight path association algorithm based on decision tree |
| CN110572577A (en) * | 2019-09-24 | 2019-12-13 | 浙江大华技术股份有限公司 | Method, device, equipment and medium for tracking and focusing |
| CN112154444A (en) * | 2019-10-17 | 2020-12-29 | 深圳市大疆创新科技有限公司 | Target detection and tracking method, system, movable platform, camera and medium |
| CN110632589A (en) * | 2019-10-17 | 2019-12-31 | 安徽大学 | Radar photoelectric information fusion technology |
| US20210231794A1 (en) * | 2020-01-28 | 2021-07-29 | Aptiv Technologies Limited | Imaging and Radar Fusion for Multiple-Object Tracking |
| US20210241026A1 (en) * | 2020-02-04 | 2021-08-05 | Nio Usa, Inc. | Single frame 4d detection using deep fusion of camera image, imaging radar and lidar point cloud |
| US20210287035A1 (en) * | 2020-03-10 | 2021-09-16 | Gm Cruise Holdings Llc | Image and lidar segmentation for lidar-camera calibration |
| CN112017431A (en) * | 2020-07-29 | 2020-12-01 | 河北德冠隆电子科技有限公司 | Active vehicle continuous tracking and positioning system and method based on multi-data fusion |
| CN112233097A (en) * | 2020-10-19 | 2021-01-15 | 中国科学技术大学 | Road scene other vehicle detection system and method based on space-time domain multi-dimensional fusion |
| CN112906777A (en) * | 2021-02-05 | 2021-06-04 | 北京邮电大学 | Target detection method and device, electronic equipment and storage medium |
| CN112991391A (en) * | 2021-03-31 | 2021-06-18 | 武汉大学 | Vehicle detection and tracking method based on radar signal and vision fusion |
| CN113658256A (en) * | 2021-08-16 | 2021-11-16 | 智道网联科技(北京)有限公司 | Target detection method and device based on laser radar and electronic equipment |
| CN113989755A (en) * | 2021-10-22 | 2022-01-28 | 智道网联科技(北京)有限公司 | Method, apparatus and computer readable storage medium for identifying an object |
| CN113903029A (en) * | 2021-12-10 | 2022-01-07 | 智道网联科技(北京)有限公司 | Method and device for marking 3D frame in point cloud data |
Non-Patent Citations (9)
| Title |
|---|
| BIN CHEN等: "Research on Target Detection Based on Distributed Track Fusion for Intelligent Vehicles", 《SENSORS》 * |
| XIAO WANG等: "On-Road Vehicle Detection and Tracking Using MMW Radar and Monovision Fusion", 《IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS》 * |
| 宋志勇等: "雷达回波序列中弱小飞机目标检测跟踪数据集", 《中国科学数据(中英文网络版)》 * |
| 宫铭钱: "基于激光雷达和相机信息融合的车辆识别与跟踪研究", 《中国优秀硕士学位论文全文数据库 工程科技II辑》 * |
| 尚昊等: "基于红外相机和毫米波雷达融合的夜间行人检测", 《传感技术学报》 * |
| 常昕等: "基于激光雷达和相机信息融合的目标检测及跟踪", 《光电工程》 * |
| 张雅雯等: "低空目标的雷达/可见光协同监视跟踪方法研究", 《计算机工程与应用》 * |
| 杨飞: "基于三维激光雷达的运动目标实时检测与跟踪", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 贺治华等: "雷达海面目标识别技术研究进展", 《科技导报》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115131748A (en) * | 2022-08-26 | 2022-09-30 | 广州市德赛西威智慧交通技术有限公司 | Method and system for improving target tracking and identifying accuracy of radar-vision all-in-one machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114187328B (en) | 2022-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107506763B (en) | Multi-scale license plate accurate positioning method based on convolutional neural network | |
| KR20180033037A (en) | Apparatus and method for detecting objects | |
| CN110264495B (en) | Target tracking method and device | |
| JP2018508078A (en) | System and method for object tracking | |
| US11170528B2 (en) | Object pose tracking method and apparatus | |
| CN113421305B (en) | Target detection method, device, system, electronic equipment and storage medium | |
| CN113447923A (en) | Target detection method, device, system, electronic equipment and storage medium | |
| CN112562093B (en) | Object detection method, electronic medium, and computer storage medium | |
| CN115063454B (en) | Multi-target tracking matching method, device, terminal and storage medium | |
| CN112037249A (en) | Method and device for tracking object in image of camera device | |
| CN114187328B (en) | Object detection method and device and electronic equipment | |
| CN115376109B (en) | Obstacle detection method, obstacle detection device, and storage medium | |
| CN111738036A (en) | Image processing method, device, equipment and storage medium | |
| CN105374049A (en) | Multi-angle-point tracking method based on sparse optical flow method and apparatus thereof | |
| CN112130153A (en) | Method for realizing edge detection of unmanned vehicle based on millimeter wave radar and camera | |
| CN113052019A (en) | Target tracking method and device, intelligent equipment and computer storage medium | |
| KR101806453B1 (en) | Moving object detecting apparatus for unmanned aerial vehicle collision avoidance and method thereof | |
| CN111814769A (en) | Information acquisition method and device, terminal equipment and storage medium | |
| CN116304992A (en) | Sensor time difference determining method, device, computer equipment and storage medium | |
| CN116543143A (en) | Training method of target detection model, target detection method and device | |
| CN114782496A (en) | Object tracking method and device, storage medium and electronic device | |
| CN111753901A (en) | Data fusion method, device and system and computer equipment | |
| CN113205144B (en) | Model training method and device | |
| CN110763232B (en) | Robot and navigation positioning method and device thereof | |
| CN116434030A (en) | Multi-sensor pre-fusion method and device based on single-sensor time sequence tracking result |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |