CN111028264B - Rotation robust three-dimensional object detection optimization method and device - Google Patents
Rotation robust three-dimensional object detection optimization method and device Download PDFInfo
- Publication number
- CN111028264B CN111028264B CN201911017327.9A CN201911017327A CN111028264B CN 111028264 B CN111028264 B CN 111028264B CN 201911017327 A CN201911017327 A CN 201911017327A CN 111028264 B CN111028264 B CN 111028264B
- Authority
- CN
- China
- Prior art keywords
- rotation detection
- detection frame
- rotation
- intersection
- coordinates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses a rotation robust three-dimensional object detection optimization method and a device, wherein the method comprises the following steps: establishing a standard coordinate system in the first rotation detection frame and the second rotation detection frame, defining projection operation in the two standard coordinate systems to obtain two pairs of rectangles which are parallel or perpendicular to each other, wherein the two pairs of rectangles can obtain a first intersection area and a second intersection area according to a calculation method in the traditional rotation-free detection; selecting the minimum value of the first intersection area and the second intersection area, and multiplying the minimum value of the intersection areas by the cosine attenuation coefficient to obtain a target intersection area; and calculating the intersection ratio according to the target intersection area and the intersection ratio calculation method, obtaining a loss function according to the intersection ratio, adding the loss function into a regression loss function of the detection network for optimization, and detecting the three-dimensional object through the optimized detection network. The method can improve the detection precision of the three-dimensional object under multiple indexes.
Description
Technical Field
The invention relates to the technical field of computer vision and machine learning, in particular to a rotation robust three-dimensional object detection optimization method and device.
Background
The three-dimensional object detection technology plays an important role in practical application scenes such as automatic driving, intelligent mobile robots and the like. The laser radars loaded by the hardware platforms provide abundant spatial information, so that a data base is provided for three-dimensional object detection. However, the optimization method for three-dimensional object detection usually adopts 1 norm loss functions with mutually independent parameters, and is sensitive to the scale of a detection frame.
Three-dimensional object detection techniques based on radar point cloud data are roughly classified into four categories: directly carrying out three-dimensional convolution operation on the point cloud data; directly carrying out three-dimensional convolution on voxels obtained by point cloud segmentation; projecting the point cloud onto a plane to form a two-dimensional feature map; and filtering background points by using a segmentation network, and generating dense candidate boxes in the second stage for further optimization. These detection frameworks mimic the image detection method, using the smooth-l1 loss function. The loss function is independent of the optimization of each detection frame parameter, is sensitive to the size of the detection frame and is not beneficial to three-dimensional detection.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, an object of the present invention is to provide a rotation robust three-dimensional object detection optimization method, which uses an estimated value of a fitting degree of a detection result as an optimization target, mashups detection parameters into a whole, has a simple calculation process, and can be directly applied to an existing detection framework.
Another object of the present invention is to provide a rotation robust three-dimensional object detection optimization apparatus.
In order to achieve the above object, an embodiment of the invention provides a rotation robust three-dimensional object detection optimization method, including:
s1, acquiring a first rotation detection frame and a second rotation detection frame, and establishing a standard coordinate system in the first rotation detection frame and the second rotation detection frame respectively;
s2, calculating corner coordinates of the second rotation detection frame through a moving center point and a rotation coordinate axis in a standard coordinate system of the first rotation detection frame, obtaining coordinates of a second projection rectangle of the second rotation detection frame by extracting the maximum value of the corner coordinates of the second rotation detection frame, and calculating a first intersection area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle;
S3, calculating corner coordinates of the first rotation detection frame through a moving center point and a rotation coordinate axis in a standard coordinate system of the second rotation detection frame, obtaining coordinates of a first projection rectangle of the first rotation detection frame by extracting the maximum value of the corner coordinates of the first rotation detection frame, and calculating a second intersection area of the second rotation detection frame and the first projection rectangle according to the coordinates of the second rotation detection frame and the first projection rectangle;
s4, selecting the minimum value of the first intersection area and the second intersection area, and multiplying the minimum value of the intersection areas by a cosine attenuation coefficient to obtain a target intersection area;
and S5, calculating an intersection ratio according to the target intersection area and an intersection ratio calculation method, obtaining a loss function according to the intersection ratio, adding the loss function into a regression loss function of a detection network for optimization, and detecting the three-dimensional object through the optimized detection network.
According to the rotation robust three-dimensional object detection optimization method, two detection frames with rotation are given, standard coordinate systems are established at respective central points, projection operation is defined in the two standard coordinate systems, and two pairs of rectangles which are parallel or perpendicular to each other are obtained. The two pairs of rectangles can obtain respective intersection areas according to a calculation method in the conventional rotation-free detection. The two Intersection areas are then averaged to obtain the estimated rotation-robust Intersection area (Intersection). And substituting the interaction into a standard IoU calculation flow to further obtain a IoU loss function. The loss function can be directly added into an optimization target of an original three-dimensional object detection frame, and the three-dimensional object detection precision under multiple indexes can be remarkably improved.
In addition, the rotation robust three-dimensional object detection optimization method according to the above embodiment of the present invention may further have the following additional technical features:
further, in an embodiment of the present invention, the coordinates of the corner point of the second rotation detection box are:
corners p ＝corners p -[x g ,z g ] T
wherein the content of the first and second substances,g is the first rotation detection frame, p is the second rotation detection frame, r g For the first rotation detection frame to rotate around the y-axis by an angle, x g 、z g And coordinates of the central point of the first rotation detection frame on the ground.
Further, in one embodiment of the present invention, the coordinates of the second projection rectangle are:
among them, horners p,aligned And coordinates of the corner point of the second rotation detection frame.
Further, in an embodiment of the present invention, the target intersection area is:
Intersection＝min(Intersection1,Intersection2)*|cos(2*(r g -r p ))|
wherein the Intersection is the target Intersection area, the Intersection1 is the first Intersection area, and the Intersection2 is the second Intersection area, cos (2 × (r) g -r p ) Is an angle-dependent cosine attenuation coefficient, r g For the first rotation detection frame, the rotation angle r around the y-axis p The second rotation detection frame is rotated around the y-axis.
Further, in an embodiment of the present invention, the calculating an intersection ratio according to the target intersection area and an intersection ratio calculation method includes: obtaining the intersection ratio RI-IoU according to an intersection ratio calculation method in the classical non-rotation two-dimensional object detection, wherein the formula is as follows:
Union＝max(Intersection,l g *w g +l p *w p -Intersection)
Wherein the interaction is the Intersection area of the target,/ g Is the length, w, of the two-dimensional rectangular projection of the first rotation detection frame on the ground g Is the width of the two-dimensional rectangular projection of the first rotation detection frame on the ground p Is the length, w, of the two-dimensional rectangular projection of the second rotation detection frame on the ground p The width of the two-dimensional rectangular projection of the second rotation detection frame on the ground is obtained.
Further, in an embodiment of the present invention, when the target intersection area of the first rotation detection frame and the second rotation detection frame is smaller than a preset threshold, the calculating of the intersection ratio includes:
calculating the first minimum closure rectangle area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle;
calculating the second minimum closure rectangle area of the second rotation detection frame and the first projection rectangle according to the coordinates of the second rotation detection frame and the first projection rectangle;
and calculating the intersection ratio according to the first minimum closure rectangle area, the second minimum closure rectangle area and the target intersection area.
Further, in an embodiment of the present invention, when the target intersection area of the first rotation detection box and the second rotation detection box is smaller than a preset threshold, the intersection ratio RI-IoU is calculated as:
Universal＝max(Universal1,Universal2)
Union＝max(Intersection,l g *w g +l p *w p -Intersection)
The Universal1 is the first minimum occlusion rectangular area, the Universal2 is the second minimum occlusion rectangular area, the Universal is the maximum value of the first minimum occlusion rectangular area and the second minimum occlusion rectangular area, and the interaction is the target Intersection area.
Further, in one embodiment of the present invention, the objective function is:
wherein RI-IoU is the cross-over ratio.
In order to achieve the above object, another embodiment of the present invention provides a rotation robust three-dimensional object detection optimization apparatus, including:
the device comprises an acquisition module, a detection module and a control module, wherein the acquisition module is used for acquiring a first rotation detection frame and a second rotation detection frame and establishing a standard coordinate system in the first rotation detection frame and the second rotation detection frame respectively;
the first calculation module is used for calculating the corner coordinates of the second rotation detection frame through a moving center point and a rotation coordinate axis in a standard coordinate system of the first rotation detection frame, obtaining the coordinates of a second projection rectangle of the second rotation detection frame by extracting the maximum value of the corner coordinates of the second rotation detection frame, and calculating a first intersection area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle;
The second calculation module is used for calculating the corner coordinates of the first rotation detection frame through a moving center point and a rotation coordinate axis in a standard coordinate system of the second rotation detection frame, obtaining the coordinates of a first projection rectangle of the first rotation detection frame by extracting the maximum value of the corner coordinates of the first rotation detection frame, and calculating a second intersection area of the second rotation detection frame and the first projection rectangle according to the coordinates of the second rotation detection frame and the first projection rectangle;
the third calculation module is used for selecting the minimum value of the first intersection area and the second intersection area and multiplying the minimum value of the intersection areas by a cosine attenuation coefficient to obtain a target intersection area;
and the detection optimization module is used for calculating the intersection ratio according to the target intersection area and the intersection ratio calculation method, obtaining a loss function according to the intersection ratio, adding the loss function into a regression loss function of the detection network for optimization, and detecting the three-dimensional object through the optimized detection network.
According to the rotation robust three-dimensional object detection optimization device, two detection frames with rotation are given, standard coordinate systems are established at respective central points, projection operation is defined in the two standard coordinate systems, and two pairs of rectangles which are parallel or perpendicular to each other are obtained. The two pairs of rectangles can obtain respective intersection areas according to a calculation method in the conventional rotation-free detection. The two Intersection areas are averaged to obtain the maximum value and then multiplied by an angle coefficient to obtain the estimated rotation-robust Intersection area (Intersection). And substituting the interaction into a standard IoU calculation flow to further obtain a IoU loss function. The loss function can be directly added into an optimization target of an original three-dimensional object detection frame, and the three-dimensional object detection precision under multiple indexes can be remarkably improved.
In addition, the rotation robust three-dimensional object detection optimization device according to the above embodiment of the present invention may further have the following additional technical features:
further, in an embodiment of the present invention, when the target intersection area of the first rotation detection frame and the second rotation detection frame is smaller than a preset threshold, the method includes:
a fourth calculating module, configured to calculate a first minimum enclosing rectangle area of the first rotation detection frame and the second projection rectangle according to coordinates of the first rotation detection frame and the second projection rectangle;
a fifth calculating module, configured to calculate a second minimum bounding rectangle area of the second rotation detection frame and the first projection rectangle according to coordinates of the second rotation detection frame and the first projection rectangle;
a sixth calculating module, configured to calculate the intersection-to-intersection ratio according to the first minimum closure rectangle area, the second minimum closure rectangle area, and the target intersection area.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flow chart of a method for rotationally robust three-dimensional object detection optimization according to an embodiment of the present invention;
FIG. 2 is a diagram of a computational process of a rotation robust three-dimensional object detection optimization method according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a minimum closure area according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a rotation robust three-dimensional object detection optimization apparatus according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
In three-dimensional detection, the evaluation index of detection is based on Intersection over Union (IoU), and in order to overcome the problem of optimizing size-sensitive characteristics, attempts of direct optimization IoU appear in recent years in rotation-free classical two-dimensional detection, such as face detection, visual tracking, lesion localization, and the like. They add IoU to the loss function as shown below:
Classic IoU is always 0 when two detection frames do not intersect, and at this time, the network cannot update the parameters, some documents propose giou (generalized Intersection over union) to measure the similarity of the detection frames under general conditions, as shown in the following formula, where Universal is the area of the minimum closed rectangle of two rectangular frames:
in the application, a rotation robust three-dimensional detection optimization method is provided, and the problems that the existing detection optimization of the three-dimensional object with rotation is sensitive to the size of the object and the direct optimization of the rotation IoU is difficult are solved. The estimated value of the fitting degree of the detection result is used as an optimization target, the detection parameters are mixed into a whole, the calculation process is simple, and the method can be directly applied to the existing detection framework.
Since the rotation parameters detected by the three-dimensional object cannot be calculated IoU by using a numerical method, a projection method is also provided to estimate the intersection area of the detection frame with rotation, so as to obtain the final rotation robustness IoU and the loss function thereof. The method of the present application can be extended from IoU in two-dimensional form to GIoU and IoU in three-dimensional volume form, with better results in three-dimensional object localization and detection.
The following describes a rotation robust three-dimensional object detection optimization method and apparatus proposed according to an embodiment of the present invention with reference to the accompanying drawings.
First, a rotation robust three-dimensional object detection optimization method according to an embodiment of the present invention will be described with reference to the accompanying drawings.
Fig. 1 is a flowchart of a rotation robust three-dimensional object detection optimization method according to an embodiment of the present invention.
As shown in fig. 1, the rotation robust three-dimensional object detection optimization method includes the following steps:
step S1, acquiring a first rotation detection frame and a second rotation detection frame, and establishing a standard coordinate system in the first rotation detection frame and the second rotation detection frame, respectively.
As shown in fig. 2, two band rotation detection frames p and g are given, taking a two-dimensional top view as an example. A standard coordinate system is established in the two belt rotation detection frames p and g.
Step S2, calculating corner coordinates of the second rotation detection frame through the moving center point and the rotation coordinate axis in the standard coordinate system of the first rotation detection frame, obtaining coordinates of a second projection rectangle of the second rotation detection frame through extracting the maximum value of the corner coordinates of the second rotation detection frame, and calculating a first intersection area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle.
In the first rotation detection frame g, assuming that the rectangle g with rotation is represented by (x, z, l, w, r), g is represented by (0,0, l, w,0) in the standard coordinate system of g. Then in this standard coordinate system, recalculating the corner coordinates of the second rotation detection frame p requires moving the center point and rotating the coordinate axes as follows:
corners p ＝corners p -[x g ,z g ] T
wherein g is a first rotation detection frame, p is a second rotation detection frame, and r g For the first rotation detection frame to rotate around the y-axis by an angle, x g 、z g And coordinates of the central point of the first rotation detection frame on the ground.
The coordinates of the coordinates p' of the first projection rectangle can be obtained by extracting the most significant value of the p corner point, as follows:
among them, horners p,aligned Coordinates of the corner point of the second rotation detection box.
In the standard coordinate system of the first rotation detection frame, the first projection rectangle is not rotated, and the Intersection area interaction 1 between the first rotation detection frame and the first projection rectangle can be easily obtained. Further, a first minimum closure area Universal1 of the first rotation detection box and the first projection rectangle can be obtained, as shown in fig. 3, wherein the minimum closure corresponds to the area inside the solid line.
Specifically, in the standard coordinate system of the first rotation detection frame, the corner coordinates of the first rotation detection frame are (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), and the corner coordinates of the second projection rectangle are (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), which are all arranged clockwise in the upper left, upper right, lower right, and lower left directions. The intersection area is max { min { X2, X2 }. max { X1, X1},0}, and the minimum bounding rectangle area is max { X2, X2 }. min { X1, X1 }.
Step S3, calculating the corner coordinates of the first rotation detection frame through the moving center point and the rotation coordinate axis in the standard coordinate system of the second rotation detection frame, obtaining the coordinates of the first projection rectangle of the first rotation detection frame through extracting the maximum value of the corner coordinates of the first rotation detection frame, and calculating the second intersection area of the second rotation detection frame and the first projection rectangle according to the coordinates of the second rotation detection frame and the first projection rectangle.
The first rotation detection frame g and the second rotation detection frame p are exchanged, and the same process as that in step S2 is performed, so as to obtain another set of the second Intersection area interaction 2 and the second minimum occlusion area Universal 2.
And step S4, selecting the minimum value of the first intersection area and the second intersection area, and multiplying the minimum value of the intersection areas by the cosine attenuation coefficient to obtain the target intersection area.
Selecting the minimum value from the first intersection area and the second intersection area obtained above, and multiplying by an angle-dependent cosine attenuation coefficient to obtain the finally estimated rotation robust intersection area, as shown in the following formula:
Intersection＝min(Intersection1,Intersection2)*|cos(2*(r g -r p ))|
wherein, the Intersection is the target Intersection area, the Intersection1 is the first Intersection area, the Intersection2 is the second Intersection area, cos (2 × (r) g -r p ) Is an angle-dependent cosine attenuation coefficient, r g For the first rotation detection frame, the rotation angle r around the y-axis p The second rotation detection frame is rotated around the y-axis.
And step S5, calculating the intersection ratio according to the target intersection area and the intersection ratio calculation method, obtaining a loss function according to the intersection ratio, adding the loss function into a regression loss function of the detection network for optimization, and detecting the three-dimensional object through the optimized detection network.
After the intersection area is obtained, a rotation-robust intersection ratio IoU, namely RI-IoU, can be obtained according to the IoU calculation method in the classical rotation-free two-dimensional object detection, as follows:
Union＝max(Intersection,l g *w g +l p *w p -Intersection)
wherein, the Intersection is the target Intersection area,/ g Is the length, w, of the two-dimensional rectangular projection of the first rotation detection frame on the ground g Is the width of the two-dimensional rectangular projection of the first rotation detection frame on the ground p Is the length, w, of the two-dimensional rectangular projection of the second rotation detection frame on the ground p The width of the two-dimensional rectangular projection of the second rotation detection frame on the ground is obtained.
Further from the cross-over ratio, the corresponding loss function is obtained as follows:
the loss function is directly added into the regression loss function of the detection framework.
And adding the loss function into a regression loss function of the detection network for optimization, and detecting the three-dimensional object through the optimized detection network.
Further, in an embodiment of the present invention, when the intersection area of the targets of the first rotation detection frame and the second rotation detection frame is smaller than a preset threshold, the calculating of the intersection ratio includes:
calculating the first minimum closure rectangle area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle;
calculating the second minimum occlusion rectangle area of the second rotation detection frame and the first projection rectangle according to the coordinates of the second rotation detection frame and the first projection rectangle;
and calculating the intersection ratio according to the first minimum closure rectangular area, the second minimum closure rectangular area and the target intersection area.
Specifically, when the target intersection area is smaller than a preset threshold value, namely the two detection boxes are low in coincidence or not coincident, adjustment is needed in calculating the cross-over ratio, the form of the cross-over ratio is changed, and the RI-IoU in the form of GIoU is introduced.
Calculating a first minimum closure area and a second minimum closure area according to the step S2 and the step S3 during calculation, selecting the maximum value of the two closure areas to calculate the intersection ratio, as shown in the following formula:
Universal＝max(Universal1,Universal2)
Union＝max(Intersection,l g *w g +l p *w p -Intersection)
the Universal1 is a first minimum occlusion rectangular area, the Universal2 is a second minimum occlusion rectangular area, the Universal is a maximum value of the first minimum occlusion rectangular area and the second minimum occlusion rectangular area, and the interaction is a target Intersection area.
And S1-S5, the obtained intersection ratio RI-IoU is a two-dimensional condition under a top view angle, and if a height parameter is introduced, the intersection ratio RI-IoU in a volume form can be obtained. Since the height parameter of the detection box is independent of the rotation, the evolution method of the two-dimensional intersection ratio RI-IoU to three-dimensional is the same as that of the standard non-rotation detection case.
The RI-IoU and the variation thereof are added into the optimization target of the detection framework as a loss function, so that the detection accuracy is improved, and the whole architecture of the network is not influenced.
It can be understood that the IoU loss function directly optimizing the rotation robustness can be directly applied to the existing detection network, and the value in the calculation process includes numerical operation, so that the error back propagation condition is satisfied. And the rotation robust IoU is expanded in a value range and a three-dimensional space, so that the universality of the method is increased.
According to the rotation robust three-dimensional object detection optimization method provided by the embodiment of the invention, two detection frames with rotation are given, a standard coordinate system is respectively established at the central point of each detection frame, and projection operation is defined in the two standard coordinate systems to obtain two pairs of rectangles which are parallel or vertical to each other. The two pairs of rectangles can obtain respective intersection areas according to a calculation method in the conventional rotation-free detection. The two Intersection areas are then averaged to obtain the estimated rotation-robust Intersection area (Intersection). And substituting the interaction into a standard IoU calculation flow to further obtain a IoU loss function. The loss function can be directly added into an optimization target of an original three-dimensional object detection frame, and the three-dimensional object detection precision under multiple indexes can be remarkably improved.
Next, a rotation robust three-dimensional object detection optimization apparatus according to an embodiment of the present invention will be described with reference to the drawings.
Fig. 4 is a schematic structural diagram of a rotation robust three-dimensional object detection optimization apparatus according to an embodiment of the present invention.
As shown in fig. 4, the rotation robust three-dimensional object detection optimizing apparatus includes: an acquisition module 100, a first calculation module 200, a second calculation module 300, a third calculation module 400, and a detection optimization module 500.
The obtaining module 100 is configured to obtain a first rotation detection frame and a second rotation detection frame, and establish a standard coordinate system in the first rotation detection frame and the second rotation detection frame, respectively.
The first calculating module 200 is configured to calculate, in a standard coordinate system of the first rotation detection frame, corner coordinates of the second rotation detection frame through the moving center and the rotation coordinate axis, obtain coordinates of a second projection rectangle of the second rotation detection frame by extracting a maximum value of the corner coordinates of the second rotation detection frame, and calculate a first intersection area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle.
The second calculating module 300 is configured to calculate, in a standard coordinate system of the second rotation detection frame, corner coordinates of the first rotation detection frame through the moving center and the rotation coordinate axis, obtain coordinates of a first projection rectangle of the first rotation detection frame by extracting a maximum value of the corner coordinates of the first rotation detection frame, and calculate a second intersection area of the second rotation detection frame and the first projection rectangle according to the coordinates of the second rotation detection frame and the first projection rectangle.
The third calculating module 400 is configured to select a minimum value of the first intersection area and the second intersection area, and multiply the minimum value of the intersection areas by the cosine attenuation coefficient to obtain a target intersection area.
And the detection optimization module 500 is configured to calculate an intersection ratio according to the target intersection area and an intersection ratio calculation method, obtain a loss function according to the intersection ratio, add the loss function into a regression loss function of the detection network for optimization, and detect the three-dimensional object through the optimized detection network.
The device regards the estimated value of the fitting degree of the detection result as an optimization target, mashups the detection parameters into a whole, has a simple calculation process, and can be directly applied to the existing detection framework.
Further, in an embodiment of the present invention, when the intersecting area of the targets of the first rotation detection frame and the second rotation detection frame is smaller than a preset threshold, the method includes:
a fourth calculating module 600, configured to calculate the first minimum enclosing rectangle area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle.
A fifth calculating module 700, configured to calculate a second minimum enclosing rectangle area of the second rotation detecting frame and the first projection rectangle according to the coordinates of the second rotation detecting frame and the first projection rectangle.
A sixth calculating module 800, configured to calculate an intersection-to-intersection ratio according to the first minimum closure rectangle area, the second minimum closure rectangle area, and the target intersection area.
It should be noted that the foregoing explanation of the embodiment of the rotation robust three-dimensional object detection optimization method is also applicable to the apparatus of the embodiment, and is not repeated herein.
According to the rotation robust three-dimensional object detection optimization device provided by the embodiment of the invention, two detection frames with rotation are given, a standard coordinate system is respectively established at the central point of each detection frame, and projection operation is defined in the two standard coordinate systems to obtain two pairs of rectangles which are parallel or vertical to each other. The two pairs of rectangles can obtain respective intersection areas according to a calculation method in the conventional rotation-free detection. The two Intersection areas are then averaged to obtain the estimated rotation-robust Intersection area (Intersection). And substituting the interaction into a standard IoU calculation flow to further obtain a IoU loss function. The loss function can be directly added into an optimization target of an original three-dimensional object detection frame, and the three-dimensional object detection precision under multiple indexes can be remarkably improved.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the description of the specification, reference to the description of "one embodiment," "some embodiments," "an example," "a specific example," or "some examples" or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Moreover, various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without being mutually inconsistent.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (8)
1. A rotation robust three-dimensional object detection optimization method is characterized by comprising the following steps:
s1, acquiring a first rotation detection frame and a second rotation detection frame, and establishing a standard coordinate system in the first rotation detection frame and the second rotation detection frame respectively;
s2, calculating corner coordinates of the second rotation detection frame through a moving center point and a rotation coordinate axis in a standard coordinate system of the first rotation detection frame, obtaining coordinates of a second projection rectangle of the second rotation detection frame by extracting the maximum value of the corner coordinates of the second rotation detection frame, and calculating a first intersection area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle;
s3, calculating corner coordinates of the first rotation detection frame through a moving center point and a rotation coordinate axis in a standard coordinate system of the second rotation detection frame, obtaining coordinates of a first projection rectangle of the first rotation detection frame by extracting the maximum value of the corner coordinates of the first rotation detection frame, and calculating a second intersection area of the second rotation detection frame and the first projection rectangle according to the coordinates of the second rotation detection frame and the first projection rectangle;
S4, selecting the minimum value of the first intersection area and the second intersection area, and multiplying the minimum value of the intersection areas by a cosine attenuation coefficient to obtain a target intersection area;
s5, calculating an intersection ratio according to the target intersection area and an intersection ratio calculation method, obtaining a loss function according to the intersection ratio, adding the loss function into a regression loss function of a detection network for optimization, and detecting the three-dimensional object through the optimized detection network;
wherein, the corner point coordinates of the second rotation detection frame are:
corners p ＝corners p -[x g ,z g ] T
wherein g is the first rotation detection frame, p is the second rotation detection frame, r g For the first rotation detection frame to rotate around the y-axis by an angle, x g 、z g Coordinates of the central point of the first rotation detection frame on the ground;
wherein the coordinates of the second projection rectangle are:
among them, horners p,aligned And coordinates of the corner point of the second rotation detection frame.
2. The rotation-robust three-dimensional object detection optimization method according to claim 1, wherein the target intersection area is:
Intersection＝min(Intersection1,Intersection2)*|cos(2*(r g -r p ))|
wherein the Intersection is the target Intersection area, the Intersection1 is the first Intersection area, and the Intersection2 is the second Intersection area, cos (2 × (r) g -r p ) Is an angle-dependent cosine attenuation coefficient, r g For the first rotation detection frame, the rotation angle r around the y-axis p The second rotation detection frame is rotated around the y-axis.
3. The rotation-robust three-dimensional object detection optimization method according to claim 1, wherein the calculating of the intersection ratio according to the target intersection area and the intersection ratio calculation method comprises: obtaining the intersection ratio RI-IoU according to an intersection ratio calculation method in classical non-rotation two-dimensional object detection, wherein the formula is as follows:
Union＝max(Intersection,l g *w g +l p *w p -Intersection)
whereinThe Intersection is the Intersection area of the target,/ g Is the length, w, of the two-dimensional rectangular projection of the first rotation detection frame on the ground g Is the width of the two-dimensional rectangular projection of the first rotation detection frame on the ground p Is the length, w, of the two-dimensional rectangular projection of the second rotation detection frame on the ground p The width of the two-dimensional rectangular projection of the second rotation detection frame on the ground is obtained.
4. The rotation-robust three-dimensional object detection optimization method according to claim 1, wherein when the target intersection area of the first rotation detection box and the second rotation detection box is smaller than a preset threshold, the calculation of the intersection ratio comprises:
calculating the first minimum closure rectangle area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle;
Calculating the second minimum closure rectangle area of the second rotation detection frame and the first projection rectangle according to the coordinates of the second rotation detection frame and the first projection rectangle;
and calculating the intersection ratio according to the first minimum closure rectangle area, the second minimum closure rectangle area and the target intersection area.
5. The rotation-robust three-dimensional object detection optimization method according to claim 4, wherein when the target intersection area of the first rotation detection box and the second rotation detection box is smaller than a preset threshold, the intersection ratio RI-IoU is calculated as:
Universal＝max(Universal1,Universal2)
Union＝max(Intersection,l g *w g +l p *w p -Intersection)
wherein, Universal1 is the first minimum occlusion rectangle area, Universal2 is the second minimum occlusion rectangle area, Universal is the maximum of the first minimum occlusion rectangle area and the second minimum occlusion rectangle area, interaction is the target Intersection area, l is the target Intersection area g Is the length, w, of the two-dimensional rectangular projection of the first rotation detection frame on the ground g Is the width of the two-dimensional rectangular projection of the first rotation detection frame on the ground p Is the length, w, of the two-dimensional rectangular projection of the second rotation detection frame on the ground p The width of the two-dimensional rectangular projection of the second rotation detection frame on the ground is obtained.
6. The rotation-robust three-dimensional object detection optimization method of claim 1, wherein the objective function is:
wherein RI-IoU is the cross-over ratio.
7. A rotation-robust three-dimensional object detection optimization apparatus, comprising:
the device comprises an acquisition module, a detection module and a control module, wherein the acquisition module is used for acquiring a first rotation detection frame and a second rotation detection frame and respectively establishing a standard coordinate system in the first rotation detection frame and the second rotation detection frame;
the first calculation module is used for calculating the corner coordinates of the second rotation detection frame through a moving center point and a rotation coordinate axis in a standard coordinate system of the first rotation detection frame, obtaining the coordinates of a second projection rectangle of the second rotation detection frame by extracting the maximum value of the corner coordinates of the second rotation detection frame, and calculating a first intersection area of the first rotation detection frame and the second projection rectangle according to the coordinates of the first rotation detection frame and the second projection rectangle;
the second calculation module is used for calculating the corner coordinates of the first rotation detection frame through a moving center point and a rotation coordinate axis in a standard coordinate system of the second rotation detection frame, obtaining the coordinates of a first projection rectangle of the first rotation detection frame by extracting the maximum value of the corner coordinates of the first rotation detection frame, and calculating a second intersection area of the second rotation detection frame and the first projection rectangle according to the coordinates of the second rotation detection frame and the first projection rectangle;
The third calculation module is used for selecting the minimum value of the first intersection area and the second intersection area and multiplying the minimum value of the intersection areas by a cosine attenuation coefficient to obtain a target intersection area;
the detection optimization module is used for calculating an intersection ratio according to the target intersection area and an intersection ratio calculation method, obtaining a loss function according to the intersection ratio, adding the loss function into a regression loss function of a detection network for optimization, and detecting the three-dimensional object through the optimized detection network;
wherein, the corner point coordinates of the second rotation detection frame are:
corners p ＝corners p -[x g ,z g ] T
wherein g is the first rotation detection frame, p is the second rotation detection frame, r g For the first rotation detection frame to rotate around the y-axis by an angle, x g 、z g Coordinates of the central point of the first rotation detection frame on the ground;
wherein the coordinates of the second projection rectangle are:
among them, horners p,aligned And coordinates of the corner point of the second rotation detection frame.
8. The rotation-robust three-dimensional object detection optimization apparatus according to claim 7, wherein when the target intersection area of the first rotation detection box and the second rotation detection box is smaller than a preset threshold, the apparatus comprises:
A fourth calculating module, configured to calculate a first minimum enclosing rectangle area of the first rotation detection frame and the second projection rectangle according to coordinates of the first rotation detection frame and the second projection rectangle;
a fifth calculating module, configured to calculate a second minimum bounding rectangle area of the second rotation detection frame and the first projection rectangle according to coordinates of the second rotation detection frame and the first projection rectangle;
a sixth calculating module, configured to calculate the intersection ratio according to the first minimum closure rectangle area, the second minimum closure rectangle area, and the target intersection area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911017327.9A CN111028264B (en) | 2019-10-24 | 2019-10-24 | Rotation robust three-dimensional object detection optimization method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911017327.9A CN111028264B (en) | 2019-10-24 | 2019-10-24 | Rotation robust three-dimensional object detection optimization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111028264A CN111028264A (en) | 2020-04-17 |
| CN111028264B true CN111028264B (en) | 2022-08-12 |
Family
ID=70200374
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911017327.9A Active CN111028264B (en) | 2019-10-24 | 2019-10-24 | Rotation robust three-dimensional object detection optimization method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111028264B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114882489B (en) * | 2022-07-07 | 2022-12-16 | 浙江智慧视频安防创新中心有限公司 | Method, device, equipment and medium for horizontally correcting rotating license plate |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011002919A (en) * | 2009-06-17 | 2011-01-06 | Seiko Epson Corp | Method and device for detecting object |
| CN109145756A (en) * | 2018-07-24 | 2019-01-04 | 湖南万为智能机器人技术有限公司 | Object detection method based on machine vision and deep learning |
| CN109872366A (en) * | 2019-02-25 | 2019-06-11 | 清华大学 | Object dimensional method for detecting position and device based on depth fitting degree assessment network |
| CN109977945A (en) * | 2019-02-26 | 2019-07-05 | 博众精工科技股份有限公司 | Localization method and system based on deep learning |
| WO2019157924A1 (en) * | 2018-02-13 | 2019-08-22 | 视辰信息科技(上海)有限公司 | Real-time detection method and system for three-dimensional object |
| CN110223302A (en) * | 2019-05-08 | 2019-09-10 | 华中科技大学 | A kind of naval vessel multi-target detection method extracted based on rotary area |
-
2019
- 2019-10-24 CN CN201911017327.9A patent/CN111028264B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011002919A (en) * | 2009-06-17 | 2011-01-06 | Seiko Epson Corp | Method and device for detecting object |
| WO2019157924A1 (en) * | 2018-02-13 | 2019-08-22 | 视辰信息科技(上海)有限公司 | Real-time detection method and system for three-dimensional object |
| CN109145756A (en) * | 2018-07-24 | 2019-01-04 | 湖南万为智能机器人技术有限公司 | Object detection method based on machine vision and deep learning |
| CN109872366A (en) * | 2019-02-25 | 2019-06-11 | 清华大学 | Object dimensional method for detecting position and device based on depth fitting degree assessment network |
| CN109977945A (en) * | 2019-02-26 | 2019-07-05 | 博众精工科技股份有限公司 | Localization method and system based on deep learning |
| CN110223302A (en) * | 2019-05-08 | 2019-09-10 | 华中科技大学 | A kind of naval vessel multi-target detection method extracted based on rotary area |
Non-Patent Citations (2)
| Title |
|---|
| Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression;Hamid Rezatofighi等;《2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)》;20190620;全文 * |
| Monocular 3D Object Detection and Box Fitting Trained End-to-End Using Intersection-over-Union Loss;Eskil Jorgensen等;《arXiv:1906.08070v2 [cs:CV]》;20190630;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111028264A (en) | 2020-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111665842B (en) | Indoor SLAM mapping method and system based on semantic information fusion | |
| US10922844B2 (en) | Image positioning method and system thereof | |
| CN112102409B (en) | Target detection method, device, equipment and storage medium | |
| CN111080662A (en) | Lane line extraction method and device and computer equipment | |
| CN110579764B (en) | Registration method and device for depth camera and millimeter wave radar, and electronic equipment | |
| CN108074237B (en) | Image definition detection method and device, storage medium and electronic equipment | |
| CN112798811B (en) | Speed measurement method, device and equipment | |
| CN111123242B (en) | Combined calibration method based on laser radar and camera and computer readable storage medium | |
| CN112700552A (en) | Three-dimensional object detection method, three-dimensional object detection device, electronic apparatus, and medium | |
| CN105957107A (en) | Pedestrian detecting and tracking method and device | |
| CN104751458A (en) | Calibration angle point detection method based on 180-degree rotating operator | |
| Al Haj et al. | Reactive object tracking with a single PTZ camera | |
| Jiang et al. | An accurate and flexible technique for camera calibration | |
| CN112215308A (en) | Single-order detection method and device for hoisted object, electronic equipment and storage medium | |
| CN111028264B (en) | Rotation robust three-dimensional object detection optimization method and device | |
| CN112405526A (en) | Robot positioning method and device, equipment and storage medium | |
| CN110458857B (en) | Central symmetry primitive detection method and device, electronic equipment and readable storage medium | |
| KR100933304B1 (en) | An object information estimator using the single camera, a method thereof, a multimedia device and a computer device including the estimator, and a computer-readable recording medium storing a program for performing the method. | |
| Song et al. | 3D vehicle model-based PTZ camera auto-calibration for smart global village | |
| Amamra et al. | Real-time multiview data fusion for object tracking with RGBD sensors | |
| CN114371475A (en) | Method, system, equipment and computer storage medium for optimizing calibration parameters | |
| CN111860161B (en) | Target shielding detection method | |
| CN116245730A (en) | Image stitching method, device, equipment and storage medium | |
| GB2362793A (en) | Image processing apparatus | |
| Su | Vanishing points in road recognition: A review |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |