CN114241011A - Target detection method, device, equipment and storage medium - Google Patents
Target detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114241011A CN114241011A CN202210163037.0A CN202210163037A CN114241011A CN 114241011 A CN114241011 A CN 114241011A CN 202210163037 A CN202210163037 A CN 202210163037A CN 114241011 A CN114241011 A CN 114241011A
- Authority
- CN
- China
- Prior art keywords
- target
- detection
- area
- detection result
- target detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Abstract
The embodiment of the application provides a target detection method, a target detection device, target detection equipment and a storage medium. The target detection method comprises the following steps: obtaining a first target detection result of a region to be detected; determining a secondary detection area in the area to be detected according to the first target detection result; the secondary detection area includes at least one of: interrupting a target prediction region, a low confidence coefficient region and an image detection unmatched region; and carrying out target detection in the secondary detection area to obtain a second target detection result. In the embodiment of the application, the final detection result is composed of the first target detection result and the second target detection result, so that compared with the detection result obtained by directly carrying out primary target detection on the to-be-detected region on the basis of a larger range, the embodiment of the application can effectively recall the target which is missed in primary detection, and the performance of target detection is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a target detection method, a target detection device, target detection equipment and a storage medium.
Background
At present, in the field of target detection, target detection may be performed based on data collected by different types of sensors to obtain detection results such as types of targets and detection frames, for example: target detection may be performed based on point cloud data collected by a lidar, or may be performed based on image data collected by a camera, and so on.
The existing target detection algorithm generally has the problem of poor detection performance, such as: for some targets with small volume and incomplete observation, conditions such as missing detection and discontinuous detection may occur, so that the accuracy of the final target detection result is low.
Disclosure of Invention
In view of the above, embodiments of the present application provide an object detection method, apparatus, device and storage medium to at least partially solve the above problems.
According to a first aspect of embodiments of the present application, there is provided a target detection method, including:
obtaining a first target detection result of a region to be detected;
determining a secondary detection area in the area to be detected according to a first target detection result; the secondary detection area includes at least one of: interrupting a target prediction region, a low confidence coefficient region and an image detection unmatched region;
and carrying out target detection in the secondary detection area to obtain a second target detection result.
According to a second aspect of embodiments of the present application, there is provided an object detection apparatus, including:
the acquisition module is used for acquiring a first target detection result of the area to be detected;
a secondary detection area determining module, configured to determine a secondary detection area in the area to be detected according to the first target detection result, where the secondary detection area includes at least one of the following items: interrupting a target prediction region, a low confidence coefficient region and an image detection unmatched region;
and the secondary detection module is used for carrying out target detection in the secondary detection area to obtain a second target detection result.
According to a third aspect of embodiments of the present application, there is provided an electronic apparatus, including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the target detection method according to the first aspect.
According to a fourth aspect of embodiments of the present application, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the object detection method according to the first aspect.
According to the target detection method provided by the embodiment of the application, after the first target detection result is obtained, the secondary detection area with a smaller range and higher probability of missed detection is determined from the large-range area to be detected based on the first target detection result, and then the secondary detection area with the higher probability of missed detection is subjected to secondary target detection again, so that a second target detection result is obtained. That is to say, in the embodiment of the present application, the final detection result is composed of the first target detection result and the second target detection result, and therefore, compared with a detection result obtained by directly performing target detection once in the area to be detected on a large scale, the embodiment of the present application can effectively recall the target that is missed in the initial detection, and the performance of target detection is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1 is a flowchart illustrating steps of a target detection method according to a first embodiment of the present application;
FIG. 2 is a diagram illustrating an example of a scenario in the embodiment shown in FIG. 1;
FIG. 3 is a flowchart illustrating steps of a target detection method according to a second embodiment of the present application;
fig. 4 is a block diagram of a target detection apparatus according to a third embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application shall fall within the scope of the protection of the embodiments in the present application.
The following further describes specific implementations of embodiments of the present application with reference to the drawings of the embodiments of the present application.
Example one
Referring to fig. 1, fig. 1 is a flowchart illustrating steps of a target detection method according to a first embodiment of the present disclosure. Specifically, the target detection method provided by this embodiment includes the following steps:
In this step, the first target detection result may be a detection result obtained by performing target detection on the area to be detected based on any target detection algorithm. In the embodiment of the present application, a specific target detection algorithm used for obtaining the first target detection result is not limited.
For example: the method can be based on a laser radar target detection algorithm, and target detection is carried out on a region to be detected to obtain a point cloud target detection result; the target detection can also be carried out on the area to be detected by a camera-based detection algorithm (image-based target detection algorithm) to obtain an image target detection result; the method can also comprise the following steps: the method comprises a point cloud target detection result obtained based on a laser radar target detection algorithm, a camera-based detection algorithm, an obtained image target detection result and the like.
The region to be detected may be a region range which can be sensed by sensor equipment (such as a laser radar, a camera, and the like), and the region to be detected generally corresponds to a larger region range.
The first target detection result may include: the method comprises the steps of obtaining type information of a target in a region to be detected, position information of a detection frame where the target is located and confidence degree of detection corresponding to the target. Those skilled in the art will understand that: the first target detection result may include only 1 or 1 kind of target, or may include a plurality of or a plurality of different targets.
And 104, determining a secondary detection area in the area to be detected according to the first target detection result.
The secondary detection region may be a region in which the possibility of the presence of the undetected target is high. Specifically, in this embodiment of the present application, the secondary detection area may include at least one of the following items: interrupting a target prediction region, a low confidence region, an image detection unmatched region.
The target tracking algorithm may determine a target tracking target prediction region, where the target tracking interruption occurs, based on the target tracking algorithm. For sensor devices such as cameras and laser radars, data acquisition is usually performed at a certain frequency, and data acquired each time belong to the same frame. The data collected at the current moment form the current frame, and the data collected at the next moment form the next frame. The above-mentioned target (interrupt target) at which the tracking interrupt occurs may be: based on the target detection result of the current frame, the target tracking algorithm predicts that: the target should theoretically appear in the next frame but not in the next frame. For example: a certain pedestrian a appears in the target detection result of the current frame, and is predicted by adopting a target tracking algorithm: the pedestrian a theoretically appears in a certain area a in the next frame, but the target of the pedestrian a does not exist in the target detection result of the next frame actually, and at this time, it can be considered that the tracking interruption occurs for the pedestrian a, the pedestrian a is an interruption target where the tracking interruption occurs, and the area a is a terminal target prediction area. In general, the tracking interruption is caused by detection omission, a target leaving an observation range, and the like, so that the tracking interruption occurring in the observation range, and the prediction region of the interruption target is likely to be the region where the target omission occurs, and therefore, in the embodiment of the present application, the interruption target prediction region may be used as the secondary detection region.
The low confidence region may be a detection box region where the confidence value of the detection is within a preset range. Further, the preset range may be a confidence range greater than the first preset confidence threshold and less than or equal to the standard preset confidence threshold.
As will be understood by those skilled in the art, in the target detection algorithm, a standard preset confidence threshold is usually set for different types of targets, and when the calculated confidence is greater than the standard preset confidence threshold, it is determined that a corresponding type of target exists in the detection region. In this embodiment of the application, the preset range may be a confidence range that is greater than a first preset confidence threshold and is less than or equal to a standard preset confidence threshold, where the first preset confidence threshold is a preset threshold that is less than the standard preset confidence threshold.
In the embodiment of the application, specific values of the first preset confidence threshold and the standard preset confidence threshold are not limited, and can be set by self-definition according to actual needs.
Generally, in order to reduce false detection, the detection algorithm outputs a result to filter out targets with too low confidence (lower than a standard preset confidence threshold). However, due to incomplete observation, inaccurate regression of the detection algorithm, etc., the confidence of the real target in the detection result may be too low (smaller than the standard preset confidence threshold). Therefore, in the embodiment of the present application, in order to improve the detection effect and reduce the missing detection, the region where the target is located in the low confidence region may be set as the secondary detection region.
The image detection unmatched region is a predicted region of the unmatched target, and the unmatched target is a target only existing in the image detection result. The automatic driving system is usually provided with a laser radar sensor and a camera sensor at the same time, the laser radar can realize very accurate space distance measurement, but the point cloud data of the laser radar is sparse, and the camera can provide very abundant texture information, so that two sensors can be used at the same time to play a complementary effect. Specifically, if there is a certain observation target in the image target detection result (target detection result obtained based on the image captured by the camera), but there is no observation target in the corresponding position in the point cloud target detection result obtained based on the point cloud data, it may be considered that there is a missing detection in the position, and the observation target may become an unmatched target, and therefore, in the embodiment of the present application, the prediction region of the unmatched target may also be used as the secondary detection region.
The three regions are regions listed in the embodiment of the present application, which have a high possibility of missing detection, and in practical applications, one or more regions may be selected as secondary detection regions according to actual needs, it should be noted that, in the implementation process of the target detection method in the embodiment of the present application, one or two of the three regions may not exist in some frame data, and in this case, only the existing region may be determined as a secondary detection region according to actual situations. In addition, in addition to the three regions listed in the embodiments of the present application, other regions may also be set as secondary detection regions. In the embodiment of the present application, a specific setting manner of the secondary detection area is not limited.
And 106, carrying out target detection in the secondary detection area to obtain a second target detection result.
In this step, the specific detection algorithm used when the second target detection result is obtained is not limited, for example: target detection may be performed based on point cloud data acquired by a laser radar, or target detection may be performed based on image data acquired by a camera, resulting in a second target detection result, and so on.
In the current stage, a target detection network model for performing target detection based on point cloud data generally has three types: a point network PonitNet with granularity of three-dimensional points, a voxel network VoxelNet with granularity of three-dimensional grid bodies, and a two-dimensional convolution network with granularity of two-dimensional grids. In the embodiment of the application, the specific type of the target detection network model is not limited, and the target detection network model can be selected from the three types according to actual needs.
Referring to fig. 2, fig. 2 is a schematic view of a corresponding scenario in the embodiment of the present application, and the following describes the embodiment of the present application with a specific scenario example by referring to the schematic view shown in fig. 2:
acquiring a first target detection result of a region to be detected, and according to the result: detecting two targets, namely a building and a pedestrian, wherein the building is located in an area a, and the pedestrian is located in an area b; according to the first target detection result, three secondary detection areas are determined from the area to be detected: an interruption target prediction region c, a low confidence region d, and an image detection unmatched region e (fig. 2 illustrates only that the number of each region is only 1, and does not limit the number of each region); and performing target detection in the areas c, d and e to finally obtain a second target detection result: there is also a vehicle in the region c1 (where c1 belongs to c), that is, by the method of the embodiment of the present application, on the basis of the first object detection result, a missed detection object is found: the vehicle located in the area c1, so far, the final target detection result of the current frame is: buildings located in area a, pedestrians located in area b, and vehicles located in area c 1.
In the embodiment of the application, after the first target detection result is obtained, a secondary detection area with a smaller range and higher possibility of missed detection is determined from a large-range area to be detected based on the first target detection result, and then the secondary detection area with the higher possibility of missed detection is subjected to secondary target detection again, so that a second target detection result is obtained. That is to say, in the embodiment of the present application, the final detection result is composed of the first target detection result and the second target detection result, and therefore, compared with a detection result obtained by directly performing a target detection on a to-be-detected region in a large range, the embodiment of the present application can effectively recall the target that is missed in the initial detection, and the performance of the target detection is improved.
The object detection method of the present embodiment may be performed by any suitable electronic device having data processing capabilities, including but not limited to: servers and PCs, etc.
Example two
Referring to fig. 3, fig. 3 is a flowchart illustrating steps of a target detection method according to a second embodiment of the present application. Specifically, the target detection method provided by this embodiment includes the following steps:
In this embodiment of the application, the first target detection result may include: point cloud target detection results and/or image target detection results. Correspondingly, obtaining the first target detection result may include:
based on a laser radar target detection algorithm, obtaining a point cloud target detection result of a to-be-detected area;
and obtaining an image target detection result of the area to be detected based on a detection algorithm of the camera.
And step 304, determining a secondary detection area in the area to be detected according to the first target detection result.
Wherein the secondary detection area comprises at least one of: interrupting a target prediction region, a low confidence region, an image detection unmatched region. The interruption target prediction area is a prediction area of a target which is determined based on a tracking algorithm and has tracking interruption; the low confidence coefficient region is a detection frame region with a detection confidence value in a preset range; the image detection unmatched region is a predicted region of the unmatched target, and the unmatched target is a target only existing in the image detection result.
Alternatively, in some embodiments, a region in which a tracking interruption occurs may be determined as an interruption target prediction region in the region to be detected based on a target tracking algorithm.
Specifically, if the first target detection result is a point cloud target detection result, the interruption target prediction area may be determined by:
aiming at the point cloud target detection result of the current frame, establishing a motion information sequence of each target; determining a stable target appearing in the next frame and a prediction area of the stable target in the next frame according to the motion information sequence of each target; and determining an interruption target in which the tracking interruption occurs from the stable targets, and determining a prediction area of the interruption target in the next frame as an interruption target prediction area.
Specifically, the motion information sequence may include a position of the object in the current frame, a motion speed (including a direction), an object identifier, and the like. The stable target can be a target which is determined to appear in the next frame through a target tracking algorithm based on the motion information sequence; the interrupt target may be: based on the target detection result of the current frame, the target tracking algorithm predicts that: the target should theoretically appear in the next frame but not in the next frame.
In the embodiment of the present application, any existing target tracking algorithm may be used to determine the interrupt target prediction area, which is not limited herein.
Optionally, in some embodiments, if the first target detection result includes: the type of the target, the 3D detection frame (three-dimensional detection frame), and the point cloud target detection result of the confidence of detection may further: and taking the detection area with the detection confidence coefficient within the preset range as a low confidence coefficient area.
The preset range may be a confidence range that is greater than the first preset confidence threshold and less than or equal to the standard preset confidence threshold. Specifically, the method comprises the following steps: as will be understood by those skilled in the art, in the target detection algorithm, a standard preset confidence threshold is usually set for different types of targets, and when the calculated confidence is greater than the standard preset confidence threshold, it is determined that a corresponding type of target exists in the detection region. In this embodiment of the application, the preset range may be a confidence range that is greater than a first preset confidence threshold and is less than or equal to a standard preset confidence threshold, where the first preset confidence threshold is a preset threshold that is less than the standard preset confidence threshold. In the embodiment of the application, specific values of the first preset confidence threshold and the standard preset confidence threshold are not limited, and can be set by self-definition according to actual needs.
Optionally, in some embodiments, if the first target detection result is an image target detection result and a point cloud target detection result, determining a secondary detection region in the region to be detected according to the first target detection result, which may further include:
carrying out position matching on the image target detection result and the point cloud target detection result in a projection mode; if the image target detection result shows a target at the first position and the point cloud target detection result does not show a target, the area corresponding to the first position is an image detection unmatched area.
Further, the image target detection result may include: the type of each target, a two-dimensional detection frame where each target is located and a three-dimensional estimation frame corresponding to each two-dimensional detection frame; the point cloud target detection result may include: the type of each target and the three-dimensional detection frame where each target is located;
correspondingly, the performing position matching on the image target detection result and the point cloud target detection result in a projection manner may include: carrying out position matching on the two-dimensional detection frame where each target is located and the 3D detection frame where each target is located in a projection mode;
correspondingly, if the target appears in the image target detection result and the target does not appear in the point cloud target detection result at the first position, the area corresponding to the first position is an image detection unmatched area, and the method may include:
and if the first two-dimensional detection frame does not have a 3D detection frame with matched position, determining the area of the three-dimensional estimation frame corresponding to the first two-dimensional detection frame as an image detection unmatched area.
In the process, the image target detection result simultaneously comprises the two-dimensional detection frame where the target is located and the three-dimensional estimation frame corresponding to each two-dimensional detection frame, the position matching is firstly carried out on the two-dimensional detection frame and the 3D detection frame of each target in the point cloud target detection result, so that the first two-dimensional detection frame of the 3D detection frame without matching is determined, and the area where the three-dimensional estimation frame corresponding to the first two-dimensional detection frame is located is determined as an image detection unmatched area. In addition, in the secondary detection stage, the image detection unmatched area represented by the three-dimensional representation can be directly used as a new detection area, and a new point cloud target detection result is obtained as a second target detection result based on a laser radar target detection algorithm, so that the accuracy of the second target detection result can be effectively improved.
Specifically, any existing position matching algorithm may be adopted to perform position matching on the two-dimensional detection frame where each target is located and the 3D detection frame where each target is located, and the position matching algorithm adopted here is not limited.
The three-dimensional estimation frame can be directly obtained based on the image through the image detection model, that is, the image is input into the image detection model, and the image detection model can simultaneously output: the type of each target, the two-dimensional detection frame where each target is located, and the three-dimensional estimation frame corresponding to each two-dimensional detection frame.
In other embodiments, when the image detection model can only output the two-dimensional detection frame where the target is located, but cannot output the three-dimensional estimation frame corresponding to the two-dimensional detection frame, the three-dimensional estimation frame corresponding to the two-dimensional detection frame can be estimated based on the obtained two-dimensional detection frame by using a camera imaging principle. In the embodiment of the application, the specific method for estimating the three-dimensional estimation frame based on the two-dimensional detection frame is not limited, and the method can be selected from the existing estimation algorithm according to actual needs.
And step 306, performing range expansion processing on the secondary detection area to obtain an expanded detection area.
Since the secondary detection regions determined in step 304 are all estimation regions that may contain targets, the range accuracy may be low. Specifically, the interruption target prediction region, the low confidence region, and the image detection unmatched region may all have a certain uncertainty on the spatial boundary, so to ensure the effect of secondary detection, the range expansion processing may be performed on the determined secondary detection region first, so as to obtain an expanded detection region.
In the embodiment of the present application, the specific expansion policy is not limited, and may be flexibly set according to the actual situation and the computing capability of the hardware device executing the target detection method. For example: an expansion factor (greater than 1) can be preset, and after the secondary detection area is obtained, three different dimensions of the secondary detection area can be expanded respectively based on the expansion factor, so that the expanded detection area is obtained; for another example, the corresponding expansion strategy may also be determined according to the movement speed of the undetected target that may exist in the secondary detection area, the distance between the target and the lidar device, and other factors.
And 308, carrying out target detection in the expanded detection area to obtain a second target detection result.
In the step, a point cloud target detection result of the expanded detection area can be obtained as a second target detection result by adopting a laser radar-based target detection algorithm; or based on a detection algorithm of the camera, obtaining an image target detection result of the enlarged detection area as a second target detection result.
For a target detection algorithm based on the laser radar, a machine learning model may be specifically adopted to obtain a second target detection result: the point cloud data in the amplified detection area can be input into a pre-trained target detection network model, so that the target detection network model outputs a second target detection result.
Further, in this embodiment of the present application, a point network model may be used as a target detection network model. Specifically, the method comprises the following steps: the point network model PointNet has the smallest structure granularity, has strong perception capability on the spatial information of a point cloud area, has the largest computational overhead and smaller receptive field, and is not usually used independently in large-range point cloud input. In the embodiment of the application, compared with the region to be detected in the step 302, the detection region after being amplified has a significantly reduced range, the amount of point cloud data is limited, and the calculation amount by using PointNet is not very large in this case, so that the use of the PointNet structure is more suitable. The deep target detection network model can use a PointNet structure to regress a three-dimensional detection frame and confidence of a target.
In the embodiment of the application, after the first target detection result is obtained, a secondary detection area with a smaller range and higher possibility of missed detection is determined from a large-range area to be detected based on the first target detection result, and then the secondary detection area with the higher possibility of missed detection is subjected to secondary target detection again, so that a second target detection result is obtained. That is to say, in the embodiment of the present application, the final detection result is composed of the first target detection result and the second target detection result, and therefore, compared with a detection result obtained by directly performing a target detection on a to-be-detected region in a large range, the embodiment of the present application can effectively recall the target that is missed in the initial detection, and the performance of the target detection is improved.
The object detection method of the present embodiment may be performed by any suitable electronic device having data processing capabilities, including but not limited to: servers and PCs, etc.
EXAMPLE III
Referring to fig. 4, fig. 4 is a block diagram of a target detection apparatus according to a third embodiment of the present application. The target detection device provided by the embodiment of the application comprises:
an obtaining module 402, configured to obtain a first target detection result of a to-be-detected region;
a secondary detection area determining module 404, configured to determine a secondary detection area in the area to be detected according to the first target detection result, where the secondary detection area includes at least one of the following items: interrupting a target prediction region, a low confidence coefficient region and an image detection unmatched region;
a secondary detection module 406, configured to perform target detection in the secondary detection area to obtain a second target detection result;
the target interruption prediction area is a prediction area of a target with tracking interruption determined based on a tracking algorithm; the low confidence coefficient region is a detection frame region with a detection confidence value in a preset range; the image detection unmatched region is a predicted region of the unmatched target, and the unmatched target is a target only existing in the image detection result.
Optionally, in some embodiments, the obtaining module 402 is specifically configured to:
based on a laser radar target detection algorithm, obtaining a point cloud target detection result of a to-be-detected area;
and obtaining an image target detection result of the area to be detected based on a detection algorithm of the camera.
Optionally, in some embodiments, the secondary detection area determining module 404 is specifically configured to:
and determining an area with tracking interruption in the area to be detected as an interruption target prediction area based on a target tracking algorithm.
Optionally, in some embodiments, if the first target detection result is a point cloud target detection result, the secondary detection area determining module 404, when performing the step of determining, as the interruption target prediction area, an area where the tracking interruption occurs in the area to be detected, is specifically configured to:
aiming at the point cloud target detection result of the current frame, establishing a motion information sequence of each target;
determining a stable target appearing in the next frame and a prediction area of the stable target in the next frame according to the motion information sequence of each target;
and determining an interruption target in which the tracking interruption occurs from the stable targets, and determining a prediction area of the interruption target in the next frame as an interruption target prediction area.
Optionally, in some embodiments, if the first target detection result is a point cloud target detection result, the secondary detection module 406 is specifically configured to:
the point cloud target detection result comprises the category of the target, a 3D detection frame and the confidence degree of detection;
and taking the detection area with the detection confidence coefficient within the preset range as a low confidence coefficient area.
Optionally, in some embodiments, if the first target detection result is an image target detection result and a point cloud target detection result, the secondary detection module 406 is specifically configured to:
carrying out position matching on the image target detection result and the point cloud target detection result in a projection mode;
if the image target detection result shows a target at the first position and the point cloud target detection result does not show a target, the area corresponding to the first position is an image detection unmatched area.
Optionally, in some of the embodiments, the image target detection result includes: the type of each target, a two-dimensional detection frame where each target is located and a three-dimensional estimation frame corresponding to each two-dimensional detection frame; the point cloud target detection result comprises the following steps: the type of each target and the three-dimensional detection frame where each target is located;
the secondary detection module 406, when performing the step of performing position matching on the image target detection result and the point cloud target detection result in a projection manner, is specifically configured to:
carrying out position matching on the two-dimensional detection frame where each target is located and the 3D detection frame where each target is located in a projection mode;
the secondary detection module 406, when performing the step that if the target appears in the detection result of the image target and the target does not appear in the detection result of the point cloud target in the first position, the area corresponding to the first position is an image detection unmatched area, is specifically configured to:
and if the first two-dimensional detection frame does not have a 3D detection frame with matched position, determining the area of the three-dimensional estimation frame corresponding to the first two-dimensional detection frame as an image detection unmatched area.
Optionally, in some of the embodiments, the object detection apparatus further includes:
the expansion module is used for carrying out range expansion processing on the secondary detection area after the secondary detection area is determined to obtain an expanded detection area;
the secondary detection module 406 is specifically configured to: and carrying out target detection in the expanded detection area to obtain a second target detection result.
The target detection apparatus in the embodiment of the present application is used to implement the corresponding target detection method in the first method embodiment or the second method embodiment, and has the beneficial effects of the corresponding method embodiment, which are not described herein again. In addition, the function of each module in the target detection apparatus in the embodiment of the present application can be implemented by referring to the description of the corresponding part in the foregoing method embodiment one or embodiment two, and is not repeated here.
Example four
Referring to fig. 5, a schematic structural diagram of an electronic device according to a fourth embodiment of the present application is shown, and the specific embodiment of the present application does not limit a specific implementation of the electronic device.
As shown in fig. 5, the electronic device may include: a processor (processor)502, a Communications Interface 504, a memory 506, and a communication bus 508.
Wherein:
the processor 502, communication interface 504, and memory 506 communicate with one another via a communication bus 508.
A communication interface 504 for communicating with other electronic devices or servers.
The processor 502 is configured to execute the program 510, and may specifically perform relevant steps in the above-described target detection method embodiment.
In particular, program 510 may include program code that includes computer operating instructions.
The processor 502 may be a CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present application. The intelligent device comprises one or more processors which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 506 for storing a program 510. The memory 506 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 510 may specifically be used to cause the processor 502 to perform the following operations: obtaining a first target detection result of a region to be detected; determining a secondary detection area in the area to be detected according to the first target detection result; the secondary detection area includes at least one of: interrupting a target prediction region, a low confidence coefficient region and an image detection unmatched region; and carrying out target detection in the secondary detection area to obtain a second target detection result.
For specific implementation of each step in the program 510, reference may be made to corresponding steps and corresponding descriptions in units in the foregoing embodiments of the target detection method, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.
With the electronic device of this embodiment, after the first target detection result is obtained, a secondary detection area with a smaller range and a higher possibility of missed detection is determined from a large-scale area to be detected based on the first target detection result, and then the secondary detection area with the higher possibility of missed detection is subjected to secondary target detection again, so as to obtain a second target detection result. That is to say, in the embodiment of the present application, the final detection result is composed of the first target detection result and the second target detection result, and therefore, compared with a detection result obtained by directly performing a target detection on a to-be-detected region in a large range, the embodiment of the present application can effectively recall the target that is missed in the initial detection, and the performance of the target detection is improved.
An embodiment of the present application further provides a computer program product, which includes computer instructions, where the computer instructions instruct a computing device to execute an operation corresponding to any one of the target detection methods in the multiple method embodiments.
It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present application may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present application.
The above-described methods according to embodiments of the present application may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the methods described herein may be stored in such software processes on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that the computer, processor, microprocessor controller or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the object detection methods described herein. Further, when a general-purpose computer accesses code for implementing the object detection methods illustrated herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the object detection methods illustrated herein.
Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.
The above embodiments are only used for illustrating the embodiments of the present application, and not for limiting the embodiments of the present application, and those skilled in the relevant art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application, so that all equivalent technical solutions also belong to the scope of the embodiments of the present application, and the scope of patent protection of the embodiments of the present application should be defined by the claims.
Claims (12)
1. A method of target detection, comprising:
obtaining a first target detection result of a region to be detected;
determining a secondary detection area in the area to be detected according to a first target detection result; the secondary detection area includes at least one of: interrupting a target prediction region, a low confidence coefficient region and an image detection unmatched region;
and carrying out target detection in the secondary detection area to obtain a second target detection result.
2. The method of claim 1, wherein the obtaining a first target detection result of the region to be detected comprises:
based on a laser radar target detection algorithm, obtaining a point cloud target detection result of the area to be detected;
and obtaining an image target detection result of the area to be detected based on a detection algorithm of the camera.
3. The method of claim 2, wherein the determining a secondary detection area in the area to be detected according to the first target detection result comprises:
and determining an area with tracking interruption in the area to be detected as an interruption target prediction area based on a target tracking algorithm.
4. The method of claim 3, wherein if the first target detection result is a point cloud target detection result, the determining, based on a target tracking algorithm, an area in which tracking interruption occurs in the area to be detected as an interruption target prediction area comprises:
aiming at the point cloud target detection result of the current frame, establishing a motion information sequence of each target;
determining a stable target appearing in the next frame and a prediction area of the stable target in the next frame according to the motion information sequence of each target;
and determining an interruption target in which tracking interruption occurs from the stable targets, and determining a prediction area of the interruption target in the next frame as an interruption target prediction area.
5. The method of claim 2, wherein if the first target detection result is a point cloud target detection result, determining a secondary detection area in the area to be detected according to the first target detection result comprises:
the point cloud target detection result comprises the category of the target, a 3D detection frame and the confidence degree of detection;
and taking the detection area with the confidence coefficient of the detection within a preset range as a low confidence coefficient area.
6. The method according to claim 2, wherein if the first target detection result is an image target detection result and a point cloud target detection result, the determining a secondary detection area in the area to be detected according to the first target detection result comprises:
carrying out position matching on the image target detection result and the point cloud target detection result in a projection mode;
if the image target detection result shows a target at the first position and the point cloud target detection result does not show a target, the area corresponding to the first position is an image detection unmatched area.
7. The method of claim 6, wherein the image target detection result comprises: the type of each target, a two-dimensional detection frame where each target is located and a three-dimensional estimation frame corresponding to each two-dimensional detection frame; the point cloud target detection result comprises: the type of each target and the three-dimensional detection frame where each target is located;
the position matching of the image target detection result and the point cloud target detection result is carried out in a projection mode, and the method comprises the following steps:
carrying out position matching on a two-dimensional detection frame where each target is located and a 3D detection frame where each target is located in a projection mode;
if the image target detection result shows a target at the first position and the point cloud target detection result does not show a target, the area corresponding to the first position is an image detection unmatched area, and the method comprises the following steps:
and if the first two-dimensional detection frame does not have a 3D detection frame with matched position, determining the area of the three-dimensional estimation frame corresponding to the first two-dimensional detection frame as an image detection unmatched area.
8. The method of claim 1, wherein after the determining a secondary detection region, the method further comprises:
carrying out range expansion processing on the secondary detection area to obtain an expanded detection area;
the target detection is performed in the secondary detection area to obtain a second target detection result, and the method comprises the following steps:
and carrying out target detection in the expanded detection area to obtain a second target detection result.
9. An object detection device comprising:
the acquisition module is used for acquiring a first target detection result of the area to be detected;
a secondary detection area determining module, configured to determine a secondary detection area in the area to be detected according to the first target detection result, where the secondary detection area includes at least one of the following items: interrupting a target prediction region, a low confidence coefficient region and an image detection unmatched region;
and the secondary detection module is used for carrying out target detection in the secondary detection area to obtain a second target detection result.
10. An electronic device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the target detection method according to any one of claims 1-8.
11. A computer storage medium having stored thereon a computer program which, when executed by a processor, implements the object detection method of any one of claims 1-8.
12. A computer program product comprising computer instructions to instruct a computing device to perform operations corresponding to the object detection method according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210163037.0A CN114241011A (en) | 2022-02-22 | 2022-02-22 | Target detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210163037.0A CN114241011A (en) | 2022-02-22 | 2022-02-22 | Target detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114241011A true CN114241011A (en) | 2022-03-25 |
Family
ID=80747750
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210163037.0A Pending CN114241011A (en) | 2022-02-22 | 2022-02-22 | Target detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114241011A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116523970A (en) * | 2023-07-05 | 2023-08-01 | 之江实验室 | Dynamic three-dimensional target tracking method and device based on secondary implicit matching |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104063885A (en) * | 2014-07-23 | 2014-09-24 | 山东建筑大学 | Improved movement target detecting and tracking method |
| CN105335701A (en) * | 2015-09-30 | 2016-02-17 | 中国科学院合肥物质科学研究院 | Pedestrian detection method based on HOG and D-S evidence theory multi-information fusion |
| CN108230352A (en) * | 2017-01-24 | 2018-06-29 | 北京市商汤科技开发有限公司 | Detection method, device and the electronic equipment of target object |
| CN110991468A (en) * | 2019-12-13 | 2020-04-10 | 深圳市商汤科技有限公司 | Three-dimensional target detection and intelligent driving method, device and equipment |
| CN111723723A (en) * | 2020-06-16 | 2020-09-29 | 东软睿驰汽车技术（沈阳）有限公司 | Image detection method and device |
| CN112949782A (en) * | 2021-04-27 | 2021-06-11 | 上海芯物科技有限公司 | Target detection method, device, equipment and storage medium |
| CN113255481A (en) * | 2021-05-11 | 2021-08-13 | 北方工业大学 | Crowd state detection method based on unmanned patrol car |
| CN113361413A (en) * | 2021-06-08 | 2021-09-07 | 南京三百云信息科技有限公司 | Mileage display area detection method, device, equipment and storage medium |
| CN113705669A (en) * | 2021-08-27 | 2021-11-26 | 上海商汤临港智能科技有限公司 | Data matching method and device, electronic equipment and storage medium |
| CN113971653A (en) * | 2021-04-06 | 2022-01-25 | 国家气象中心 | Target detection method, device and equipment for remote sensing image and storage medium |
-
2022
- 2022-02-22 CN CN202210163037.0A patent/CN114241011A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104063885A (en) * | 2014-07-23 | 2014-09-24 | 山东建筑大学 | Improved movement target detecting and tracking method |
| CN105335701A (en) * | 2015-09-30 | 2016-02-17 | 中国科学院合肥物质科学研究院 | Pedestrian detection method based on HOG and D-S evidence theory multi-information fusion |
| CN108230352A (en) * | 2017-01-24 | 2018-06-29 | 北京市商汤科技开发有限公司 | Detection method, device and the electronic equipment of target object |
| CN110991468A (en) * | 2019-12-13 | 2020-04-10 | 深圳市商汤科技有限公司 | Three-dimensional target detection and intelligent driving method, device and equipment |
| CN111723723A (en) * | 2020-06-16 | 2020-09-29 | 东软睿驰汽车技术（沈阳）有限公司 | Image detection method and device |
| CN113971653A (en) * | 2021-04-06 | 2022-01-25 | 国家气象中心 | Target detection method, device and equipment for remote sensing image and storage medium |
| CN112949782A (en) * | 2021-04-27 | 2021-06-11 | 上海芯物科技有限公司 | Target detection method, device, equipment and storage medium |
| CN113255481A (en) * | 2021-05-11 | 2021-08-13 | 北方工业大学 | Crowd state detection method based on unmanned patrol car |
| CN113361413A (en) * | 2021-06-08 | 2021-09-07 | 南京三百云信息科技有限公司 | Mileage display area detection method, device, equipment and storage medium |
| CN113705669A (en) * | 2021-08-27 | 2021-11-26 | 上海商汤临港智能科技有限公司 | Data matching method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| SE-HOON PARK等: "Robust online multiple object tracking based on the confidence-based relative motion network and correlation filter", 《2016 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)》 * |
| 张财广等: "光学卫星遥感图像舰船目标检测综述", 《电波科学学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116523970A (en) * | 2023-07-05 | 2023-08-01 | 之江实验室 | Dynamic three-dimensional target tracking method and device based on secondary implicit matching |
| CN116523970B (en) * | 2023-07-05 | 2023-10-20 | 之江实验室 | Dynamic three-dimensional target tracking method and device based on secondary implicit matching |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11885872B2 (en) | System and method for camera radar fusion | |
| CN108960211B (en) | Multi-target human body posture detection method and system | |
| CN114022830A (en) | Target determination method and target determination device | |
| CN111402293A (en) | Vehicle tracking method and device for intelligent traffic | |
| CN112683228A (en) | Monocular camera ranging method and device | |
| CN115097419A (en) | External parameter calibration method and device for laser radar IMU | |
| CN114241011A (en) | Target detection method, device, equipment and storage medium | |
| CN110673607A (en) | Feature point extraction method and device in dynamic scene and terminal equipment | |
| CN111507999B (en) | Target tracking method and device based on FDSST algorithm | |
| CN112837384A (en) | Vehicle marking method and device and electronic equipment | |
| CN113256709A (en) | Target detection method, target detection device, computer equipment and storage medium | |
| CN112308917A (en) | Vision-based mobile robot positioning method | |
| CN116977671A (en) | Target tracking method, device, equipment and storage medium based on image space positioning | |
| CN114648639B (en) | Target vehicle detection method, system and device | |
| CN115527050A (en) | Image feature matching method, computer device and readable storage medium | |
| CN110689556A (en) | Tracking method and device and intelligent equipment | |
| CN110880003A (en) | Image matching method and device, storage medium and automobile | |
| CN115512281A (en) | Invader monitoring method and system combining video camera and laser radar | |
| CN115346184A (en) | Lane information detection method, terminal and computer storage medium | |
| CN114092778A (en) | Radar camera data fusion system and method based on characterization learning | |
| CN116433715A (en) | Time sequence tracking method, device and medium based on multi-sensor front fusion result | |
| CN112967399A (en) | Three-dimensional time sequence image generation method and device, computer equipment and storage medium | |
| CN113099210B (en) | Three-dimensional image restoration method and device, computer equipment and storage medium | |
| CN113837270B (en) | Target identification method, device, equipment and storage medium | |
| CN116681884B (en) | Object detection method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220325 |
|
| RJ01 | Rejection of invention patent application after publication |