CN110991534A - Point cloud data processing method, device, equipment and computer readable storage medium - Google Patents
Point cloud data processing method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN110991534A CN110991534A CN201911221610.3A CN201911221610A CN110991534A CN 110991534 A CN110991534 A CN 110991534A CN 201911221610 A CN201911221610 A CN 201911221610A CN 110991534 A CN110991534 A CN 110991534A
- Authority
- CN
- China
- Prior art keywords
- cloud data
- point cloud
- anchor
- data processing
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 31
- 238000012545 processing Methods 0.000 claims abstract description 62
- 238000001514 detection method Methods 0.000 claims abstract description 27
- 238000007499 fusion processing Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 14
- 238000000605 extraction Methods 0.000 claims description 4
- 238000000034 method Methods 0.000 description 19
- 238000010586 diagram Methods 0.000 description 11
- 206010061274 Malocclusion Diseases 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The application provides a point cloud data processing method, a point cloud data processing device and a computer readable storage medium, and the implementation scheme comprises the following steps: acquiring point cloud data in a target area; clustering the point cloud data according to a plurality of anchor point frames with different yaw angles; extracting data characteristics of the point cloud data in each anchor point frame; and generating a feature map according to the extracted data features. According to the point cloud data processing method, the point cloud data processing device, the point cloud data processing equipment and the computer readable storage medium, the point cloud data are clustered by adopting the anchor point frames, and the anchor point frames used for clustering the point cloud data all have different yaw angles, so that the point cloud data obtained by clustering have diversity, and the extracted low-dimensional data feature types are more abundant. Corresponding feature maps can be generated by extracting the data features of the cloud data, and further, the vehicle detection results can be obtained by directly utilizing the deep convolutional network for processing.
Description
Technical Field
The present disclosure relates to a target detection technology, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for processing point cloud data.
Background
Deep learning is a technical field that has been rapidly developed and studied in recent years. Since the rise of deep learning, convolutional neural networks have been widely used in image recognition and target detection. Compared with a target detection algorithm using an image as input data, the input data of the three-dimensional target detection is mainly laser point cloud data. However, the laser point cloud has the characteristics of sparseness and irregularity, and it is generally difficult to directly utilize the deep convolutional network to process the point cloud data, so that the development of the deep convolutional network is restricted, and therefore, the technical problem to be solved by the technical staff in the field is urgent.
Disclosure of Invention
The application provides a point cloud data processing method, a point cloud data processing device and a computer readable storage medium, and aims to solve the technical problem that point cloud data cannot be processed directly by using a deep convolution network in the prior art.
A first aspect of the present application provides a point cloud data processing method, including:
acquiring point cloud data in a target area;
clustering the point cloud data according to a plurality of anchor point frames with different yaw angles;
extracting data characteristics of the point cloud data in each anchor point frame;
and generating a feature map according to the extracted data features.
Another aspect of the present application is to provide a point cloud data processing apparatus including:
the acquisition module is used for acquiring point cloud data in a target area;
the clustering module is used for clustering the point cloud data according to a plurality of anchor point frames with different yaw angles;
the extraction module is used for extracting the data characteristics of the point cloud data in each anchor point frame;
and the generating module is used for generating a feature map according to the extracted data features.
Yet another aspect of the present application provides a point cloud data processing apparatus including:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the point cloud data processing method.
Yet another aspect of the present application is to provide a computer-readable storage medium, having stored thereon a computer program,
the computer program is executed by a processor to implement the point cloud data processing method.
The technical effects of the point cloud data processing method, the point cloud data processing device, the point cloud data processing equipment and the computer readable storage medium are as follows:
the application provides a point cloud data processing method, a point cloud data processing device and a computer readable storage medium, wherein the point cloud data processing device comprises the following steps: acquiring point cloud data in a target area; clustering the point cloud data according to a plurality of anchor point frames with different yaw angles; extracting data characteristics of the point cloud data in each anchor point frame; and generating a feature map according to the extracted data features. According to the point cloud data processing method, the point cloud data processing device, the point cloud data processing equipment and the computer readable storage medium, the point cloud data are clustered by adopting the anchor point frames, and the anchor point frames used for clustering the point cloud data all have different yaw angles, so that the point cloud data obtained by clustering have diversity, and the extracted low-dimensional data feature types are more abundant. Corresponding feature maps can be generated by extracting the data features of the cloud data, so that the deep convolutional network can be directly utilized for processing and obtaining a vehicle detection result, and the applicability of the deep convolutional network is improved.
Drawings
Fig. 1 is a flowchart illustrating a point cloud data processing method according to an exemplary embodiment of the present application;
FIG. 2 is a flow chart of a point cloud data processing method according to another exemplary embodiment of the present application;
FIG. 3 is a schematic spatial view of a three-dimensional point cloud as shown in an exemplary embodiment of the present application;
FIG. 4 is a bird's eye view of a three-dimensional point cloud as shown in an exemplary embodiment of the present application;
FIG. 5 is a schematic diagram of a signature graph generation process shown in an exemplary embodiment of the present application;
FIG. 6 is a schematic diagram illustrating a subsequent processing of a feature map according to an exemplary embodiment of the present application;
FIG. 7 is a block diagram of a point cloud data processing apparatus according to an exemplary embodiment of the present application;
FIG. 8 is a block diagram of a point cloud data processing apparatus according to another exemplary embodiment of the present application;
FIG. 9 is a block diagram of a point cloud data processing apparatus according to yet another exemplary embodiment of the present application;
fig. 10 is a block diagram of a point cloud data processing apparatus according to an exemplary embodiment of the present application.
Detailed Description
In order to process point cloud data by using a conventional deep convolution network, it is a conventional practice to perform voxelization on point cloud data (a Voxel-net is a three-dimensional target detection algorithm), cut a three-dimensional space into a plurality of small three-dimensional voxels, and perform 3d convolution in the three-dimensional space of the whole target. However, this method has a disadvantage that the 3d convolution speed is slow. Furthermore, unlike anchor-box (anchor-point box) in two-dimensional image target detection algorithms, which are all rectangular boxes, there are many possibilities for the yaw angle of a target (e.g., a vehicle) in three-dimensional target detection.
Since the anchor-box of the three-dimensional target detection generally has various yaw angles, although the right-angle rectangular frame of the anchor-box can form a good matching effect with the right-angle movement of the convolution operation in the two-dimensional target detection, the yaw angle of the anchor-box in the three-dimensional target detection has various possibilities, and the 3d convolution operation along the right-angle direction is difficult to form a good matching effect with the anchor-box with various yaw angles.
In another approach, a Frustum-pointet (i.e., a cone point cloud network), a three-dimensional target detection algorithm, directly processes point cloud data through Pointnet. The method extracts the data characteristics of the point cloud data in the Frustum target area through Pointnet. Then dividing each point cloud data into a foreground and a background, and connecting Rpn networks by using the data characteristics in the foreground data point cloud for detection and classification. However, this method requires that the target be detected on the image in advance by a two-dimensional target detection algorithm, and if the two-dimensional target is lost, this method will fail.
In the scheme provided by the embodiment of the application, a plurality of anchor point frames are adopted for clustering point cloud data aiming at the problems, and the anchor point frames for clustering the point cloud data all have different yaw angles, so that the point cloud data obtained by clustering has diversity and the extracted low-dimensional data feature types are more abundant. Corresponding feature maps can be generated by extracting the data features of the cloud data, and further, the vehicle detection results can be obtained by directly utilizing the deep convolutional network for processing.
Fig. 1 is a flowchart illustrating a point cloud data processing method according to an exemplary embodiment of the present application.
As shown in fig. 1, the method for processing point cloud data provided by this embodiment includes:
The point cloud data processing method provided by the embodiment may be executed by an electronic device with computing capability, for example, may be executed by a computer. The method provided by the embodiment can be arranged in the electronic equipment in a software form.
Specifically, the electronic device may store the point cloud data to be processed in advance, or store the point cloud data in another device, and at this time, the electronic device may access the device in which the point cloud data is stored, so as to read the point cloud data.
Further, the point cloud data may be directly or real-time acquired by a laser radar, which may be installed near a target area to be detected, and may acquire the point cloud data in the target area in real time when a target, such as a vehicle, in the target area moves.
Referring to fig. 3 and 4, the point cloud data represents the position of the vehicle in the three-dimensional space of the target area on the x-axis coordinate, the y-axis coordinate, and the z-axis coordinate, in the form of (x, y, z, r). Where r represents the reflectivity. For example, when the x-axis coordinate is (0, 70.4), this represents the forward direction. When the y-axis coordinate is (-40, 40), this represents the left-to-right direction. When the z-axis coordinate is (0, 4), this indicates that the vertical ground is pointed in the sky direction. Similar coordinates refer to the above examples and are not described in detail herein.
And step 102, clustering the point cloud data according to a plurality of anchor point frames with different yaw angles.
Therefore, after the point cloud data are clustered through the anchor point frames with different yaw angles, the types of data features can be richer, the obtained point cloud data are ensured to have diversity, and the extracted low-dimensional data feature types are richer.
When the feature map is generated by using the data features of the cloud data, the obtained feature map can be directly processed by using the deep convolutional network, and the vehicle detection result can be effectively obtained after the data processing is performed by using the deep convolutional network, so that the applicability of the deep convolutional network is expanded.
Referring to fig. 5, in one embodiment, the generation manner of the anchor blocks includes: generating image meshes in an x-y plane of the target area, each of the image meshes generating a predetermined number of the anchor boxes. For example, in the generation of the anchor boxes, image meshes corresponding to feature maps may be generated or divided within the x-y plane of the target area, and then a predetermined number of the anchor boxes may be generated in each of the image meshes.
Of course, when generating the anchor point frame, a person skilled in the art may also perform a generating operation according to other similar rules, and after generating the anchor point frame, it is only required that the anchor point frames are uniformly distributed in the x-y plane of the target area, and each position has a plurality of anchor point frames with different yaw angles, which is not limited herein.
Continuing with the illustration of fig. 5, an image mesh corresponding to Feature-map (Feature map) may be divided on the x-y plane, i.e., the bird's eye view plane, at intervals of 0.4 meters, thereby dividing the image mesh size to 0.4x0.4, by way of example. The y-axis coordinate range may be set to-40, 40 in meters. The x-axis coordinate range is set to 0, 70.4 in meters. Thus, there are 80/0.4, i.e., 200, intervals (grids) along the y-axis and 70.4/0.4, i.e., 176, intervals (grids) along the x-axis. Therefore, the number of image meshes of the corresponding feature map at this time is 200 × 176.
In addition, when the x-y plane of the target area is divided, or when the x-y plane of the target area generates the image mesh, a person skilled in the art may also divide the image mesh according to other sizes to obtain an image mesh with an appropriate size, which is not limited herein.
In a preferred embodiment, and with continued reference to FIG. 5, 4 anchor boxes may be generated at the center of each of the image grids, with different yaw angles. At this time, anchor blocks are generated in the center of each image grid, and the shape of each feature map is 200 × 176. From the above, the ranges on the corresponding x-y plane are (-40, 40) and (0, 70.4). Therefore, 200 coordinate points can be uniformly obtained in the range of-40 to 40, and 176 coordinate points can be uniformly obtained in the range of 0 to 70.4, wherein the coordinate points of the x axis and the y axis can be used as the center of the anchor point frame, and finally the coordinate point matrix with the shape of 200x176 is formed in a combined mode. Meanwhile, preferably, the coordinate of the z-axis may be selected to be 0.5, that is, it may be assumed that the center points of the vehicle targets are all 0.5 m high from the ground.
For example, the length, width and height of the anchor point frame may all be selected (3.9, 1.6 and 1.56), and the yaw angles of the generated 4 different anchor point frames may be selected (0, 0.79, 1.57 and 2.37), respectively. Therefore, in this embodiment, the number of anchor blocks is 200x176x4 ═ 140800, and the shape of the generated matrix of anchor blocks is 200x176x4x 7. Note that "7" in the last dimension represents the coordinates (x, y, z) of the center point of the anchor point frame, the width, length, and height (w, l, h) of the anchor point frame, and the yaw angle (θ) of the anchor point frame.
Additionally, in one embodiment, the clustering the point cloud data according to anchor point boxes with different yaw angles comprises:
and projecting the point cloud data along an x-y plane of the target area, and gathering the point cloud data projected into the same anchor point frame into one type.
Therefore, when clustering is carried out on the point cloud data, a mode of projecting to an x-y plane can be adopted, namely, the point cloud data of a target area is projected to the x-y plane, and the point cloud data projected to the same anchor point frame can be clustered into one class, so that clustering of the point cloud data can be completed.
The clustering mode can cluster various point cloud data in anchor point frames with different yaw angles, and can ensure the richness of data characteristics when the data characteristics of the point cloud data are subsequently extracted.
And 103, extracting the data characteristics of the point cloud data in each anchor point frame.
Next, features need to be extracted from the clustered point cloud data, so that a feature map can be generated through the extracted data features. A predetermined number of point cloud data may be selected within each anchor box when extracting the data features.
For example, 128 point clouds can be randomly selected from each anchor point box, and data features of the 128 point cloud data are extracted one by one. And putting the data features into corresponding positions of the feature map pair, thereby generating a required pair feature map.
The characteristic graph clusters point cloud data through a plurality of anchor point frames with different yaw angles and then extracts data characteristics one by one, so that the data types of the characteristic graph are rich, and the characteristic graph is suitable for processing by utilizing a deep convolution network.
In one embodiment, the extracting the data feature of the point cloud data in each of the anchor boxes comprises:
and selecting a predetermined amount of point cloud data in each anchor point frame, and extracting the data characteristics of the selected point cloud data according to a Pointernet network.
At this time, after a predetermined number of point cloud data are randomly selected in each anchor point frame, features of the randomly selected point cloud data may be extracted one by one using a pointent network.
With continued reference to fig. 7, for example, if 128 points of cloud data are randomly selected from each anchor box as described above, a feature map with a size of 200x176x128 may be obtained. Meanwhile, if 4 anchor blocks with different yaw angles are generated in the center of each image grid, the shape of the feature diagram obtained finally is 4x200x176x 128.
And 104, generating a feature map according to the extracted data features.
At this time, after extracting features from the point cloud data clustered in the anchor point frames located in each position of the feature map, the corresponding features are placed in corresponding positions on the feature map, so that the required feature map can be formed. In addition, the manner of generating the feature map may also be generated in other manners, and a person skilled in the art may select an appropriate manner according to requirements to form a required feature map by using the extracted data features, which is not limited herein.
The present embodiment provides a method for detecting a vehicle, which is performed by a device provided with the method of the present embodiment, which is typically implemented in hardware and/or software.
The point cloud data processing method provided by the embodiment comprises the following steps: acquiring point cloud data in a target area; clustering the point cloud data according to a plurality of anchor point frames with different yaw angles; extracting data characteristics of the point cloud data in each anchor point frame; and generating a feature map according to the extracted data features. According to the point cloud data processing method, the point cloud data processing device, the point cloud data processing equipment and the computer readable storage medium, the point cloud data are clustered by adopting the anchor point frames, and the anchor point frames used for clustering the point cloud data all have different yaw angles, so that the point cloud data obtained by clustering have diversity, and the extracted low-dimensional data feature types are more abundant. Corresponding feature maps can be generated by extracting the data features of the cloud data, so that the deep convolutional network can be directly utilized for processing and obtaining a vehicle detection result, and the applicability of the deep convolutional network is improved.
Fig. 2 is a flowchart illustrating a point cloud data processing method according to another exemplary embodiment of the present application.
As shown in fig. 2, the point cloud data processing method provided in this embodiment includes:
And 202, clustering the point cloud data according to a plurality of anchor point frames with different yaw angles.
And step 204, generating a feature map according to the extracted data features.
The specific principle and implementation of steps 201 to 204 are similar to those of steps 101 to 104, and reference may be made to the related descriptions above, which are not repeated herein.
And step 205, performing feature fusion processing on the feature map, and performing regression processing and classification processing on the processed feature map to generate a vehicle detection result.
Referring to fig. 2 and 6, after the feature map is obtained in the above manner, the feature map may be subjected to feature fusion processing, and the processed feature map may be subjected to regression processing and classification processing to generate a vehicle detection result.
Specifically, it can be known from the above that after clustering point-to-point cloud data and extracting data features, a corresponding feature map can be obtained. As shown in fig. 6, optionally, the feature map may perform feature fusion processing on the feature map according to the nernet network structure, and the high-resolution feature may be obtained after performing fusion processing on the feature map through the nernet network structure, that is, the size of the feature map may not be changed.
In addition, a resnet network structure or an vgg network structure may be used to fuse the feature maps, and the feature maps after the resnet network structure or the vgg network structure are fused become smaller, so that the feature maps need to be restored to the original size by deconvolution or upsampling after the feature maps are fused by the resnet network structure or the vgg network structure.
The feature fusion processing of the feature map according to the hnnet network structure is exemplified, and when the obtained feature map is subjected to the feature fusion processing through the hnnet network structure, the data features with high resolution can be extracted, and the size of the feature map after the fusion processing is not changed.
As shown in fig. 6, for example, if the size of the feature map obtained before the fusion process is 200x176x128, then the feature map needs to be subjected to a feature stitching process before the fusion process, and a 200x176x512 feature map is obtained first. Then, after the fusion processing of the hrnet network structure, the scale of the feature map becomes 200x176x 128. At this time, the obtained data features with the highest resolution are input into Rpn for regression processing and classification processing.
Before that, after the anchor frame is generated according to the foregoing rule, the anchor frame and the Ground-route (the real value of the vehicle target in the target area) generated by the above process may be encoded.
The group-route indicates: if there are 3 vehicles in the target area, then the matrix of the group-route is 3x7, where the last dimension "7" has the same meaning as the last dimension "7" of the anchor point frame, i.e. the center point coordinates (x, y, z), the width, length, height (w, l, h) of the anchor point frame, and the yaw angle (θ).
Firstly, the score of each anchor frame needs to be calculated according to the anchor frame and the group-route, and the score calculation rule is as follows:
and calculating the iou (intersection ratio) of all anchor point frames and the group-truth in the x-y axis plane. The "intersection" in the intersection ratio refers to the overlapping area of the two regions, and refers to the sum of the areas of the two regions minus the overlapping area. For example, if the intersection area of the two regions is 20 and the union area is 100, the intersection ratio is 0.2.
Therefore, the specific calculation rule is that if the intersection ratio of the anchor frame and the group-channel is greater than 0.6, the classification label of the anchor frame is marked as a positive class, and the maximum group-channel corresponding to the intersection ratio is selected to be used for calculating the regression label. If the intersection ratio of the anchor box and the group-channel is less than 0.45, the classification label of the anchor box is marked as a negative class. If the iou of the anchor box and all group-truth is between 0.45 and 0.6, the anchor box does not participate in the calculation.
For the anchor point frame with the classification label as the positive class, the regression label of the anchor point frame can be obtained by encoding in the following way. The coding formula is as follows:
Δθ＝θg-θa。
wherein, Δ x, Δ y, Δ z, Δ l, Δ w, Δ h, Δ θ are regression labels of the anchor frames after encoding, g superscripts in each formula represent values of the group-channel corresponding to the anchor frame of the positive type, and a superscripts in each formula represent the anchor frames. For anchor blocks labeled as positive classes, the regression label is calculated using the above formula.
In addition, for the anchor point frame marked as the positive type, the type of the yaw angle is coded at the same time, namely when the yaw angle of the group-route corresponding to the positive type anchor point frame is larger than 0, the anchor point frame is marked as the positive type, and when the yaw angle of the group-route corresponding to the positive type anchor point frame is larger than 0, the anchor point frame is marked as the negative type.
The regression label of the anchor block after being coded is obtained through the formula, and the shape of the regression label is 200x176x4x 7. The shape of the encoded class label is 200x176x4x2, where "2" in the last dimension indicates one-hot encoding for the 0, 1 class. The shape of the encoded yaw angle classification label is 200x176x4x2, where "2" in the last dimension indicates one-hot encoding for the 0, 1 category.
The feature map is then input into Rpn networks for regression and classification processing. Rpn the network consists of regression branches and classification branches. Both the regression branch and the classification branch consist of multiple layers of convolutions. The regression branch is responsible for predicting the deviation between the anchor frame and the group-truth, so that the regression branch outputs the deviation relative to the anchor frame, namely Δ x, Δ y, Δ z, Δ w, Δ l, Δ h, Δ θ. With continued reference to fig. 6, in this embodiment, the matrix shape output at this time is 200x176x4x7, the shape corresponding to the regression label obtained above.
The classification branch comprises a vehicle classification branch and a yaw angle classification branch.
The vehicle classification is responsible for predicting whether a vehicle is, and the probability of each classification is obtained. The vehicle classification has two categories, i.e., vehicle or not. The matrix shape of the vehicle classification branch output is 200x176x4x2, and the shape corresponds to the obtained classification label.
The yaw angle classification is responsible for predicting the yaw angle class, which has two classes, positive or negative. The matrix shape of the yaw angle classification branch output is 200x176x4x2, and the shape corresponds to the yaw angle classification label described above.
Therefore, three kinds of labels, namely a regression label, a vehicle classification label and a yaw angle classification label, can be obtained finally. The Rpn-header of the network output also has three outputs, namely:
car _ cls _ header, header _ cls _ header, and reg _ header.
The matrix shapes of the three tag outputs are 200x176x4x2, 200x176x4x2, and 200x176x4x7, respectively. For the car _ cls _ header loss function, focal-loss is selected, the header _ cls _ header loss function, focal-loss is selected, and the reg _ header loss function, smooth-l1 loss is selected. The learning strategy can select adam, the initial learning rate is 0.001, and 10 epochs become 0.5 of the original rate as the iterative training is gradually reduced. And (5) performing iterative training by 100 epochs, and finally saving the trained model.
And finally, inputting the point cloud data into the trained model to output a result, and decoding and screening the result. Namely, the point cloud is output to a model to obtain a classification result and a regression result, the classification result is firstly sorted, the sorting is carried out from large to small according to the score of the automobile, and the classification result and the regression result of anchor point frames with the highest score and the preset number are reserved. The retained classification results and regression results are then decoded. For example, the first 1000 anchor blocks with the highest score may be reserved, or other numbers of anchor blocks may be reserved according to actual requirements. Wherein, the formula of decoding is as follows:
lg＝Δl*la+la，
wg＝Δw*wa+wa，
hg＝Δh*ha+ha，
θg＝Δθ+θa。
through the decoding, real output can be obtained by combining the yaw angle classification result, and the optimal result is screened through nms by using the 1000 real outputs and the scores thereof, and the vehicle detection result is output as a final result.
Where nms is the non-maxima suppression algorithm. For example, for 1000 three-dimensional output frames and their corresponding scores, the three-dimensional output frame with the highest classification score is first selected and stored. Then, the three-dimensional output frame is 3d iou with the remaining 999 three-dimensional output frames.
At this time, if the calculated 3d iou is greater than a predetermined threshold, for example, greater than 0.1, it is considered that the two three-dimensional output frames of the comparison overlap too much, and the score of the two output frames is low is discarded. And if the calculated 3d iou is less than 0.1, reserving the corresponding three-dimensional output frame. The predetermined threshold may be set according to actual conditions, and is not limited herein.
And after the calculation, further repeating the calculation of the rest according to the mode. For example, if 500 three-dimensional output frames are reserved in the previous calculation, the three-dimensional output frame with the highest score is taken out from the 500 three-dimensional output frames and stored, and then the three-dimensional output frame and the rest 499 three-dimensional output frames are made into 3d iou. And continuing to use 0.1 as a preset threshold, if the calculated 3d iou is smaller than 0.1, reserving the corresponding three-dimensional output frame, and discarding the corresponding three-dimensional output frame larger than 0.1. The above operations are repeated, and the final result is the saved three-dimensional output frame. Those skilled in the art can use the algorithm to calculate the final result, which is not described herein.
Fig. 7 is a block diagram of a point cloud data processing apparatus according to an exemplary embodiment of the present application.
As shown in fig. 7, the present embodiment provides a point cloud data processing apparatus, including:
an obtaining module 31, configured to obtain point cloud data in a target area;
a clustering module 32, configured to cluster the point cloud data according to a plurality of anchor frames with different yaw angles;
an extracting module 33, configured to extract data features of the point cloud data in each anchor point frame;
and a generating module 34, configured to generate a feature map according to the extracted data features.
The present embodiment provides a point cloud data processing apparatus, including: an obtaining module 31, configured to obtain point cloud data in a target area; a clustering module 32, configured to cluster the point cloud data according to a plurality of anchor frames with different yaw angles; an extracting module 33, configured to extract data features of the point cloud data in each anchor point frame; and a generating module 34, configured to generate a feature map according to the extracted data features.
In the device that this embodiment provided, adopted a plurality of anchor point frames to cluster point cloud data, and a plurality of anchor point frames that are used for clustering point cloud data all have different yaw angles, can guarantee from this that the point cloud data that the clustering obtained have the variety, the low-dimensional data characteristic kind of extraction is abundanter. Corresponding feature maps can be generated by extracting the data features of the cloud data, and further, the vehicle detection results can be obtained by directly utilizing the deep convolutional network for processing.
The specific principle and implementation of the point cloud data processing apparatus provided in this embodiment are similar to those of the related technical contents in the method embodiment shown in fig. 1, and the related technical contents may refer to the foregoing descriptions and are not described herein again.
Fig. 8 is a block diagram of a point cloud data processing apparatus according to another exemplary embodiment of the present application.
As shown in fig. 8, on the basis of the above embodiment, optionally, the clustering module 32 includes a point cloud projection unit 321, configured to project the point cloud data along an x-y plane of the target area, and cluster the point cloud data projected into the same anchor point frame into a cluster.
Optionally, the clustering module 32 includes a grid generating unit 322, configured to generate image grids in an x-y plane of the target region, where each image grid generates a predetermined number of anchor point boxes.
Optionally, the center of each image grid generates 4 anchor point frames with different yaw angles.
Optionally, the extracting module 33 is specifically configured to:
and selecting a predetermined amount of point cloud data in each anchor point frame, and extracting the data characteristics of the selected point cloud data according to a Pointernet network.
The specific principle and implementation of the point cloud data processing apparatus provided in this embodiment are similar to those of the embodiment shown in fig. 7, and are not described here again.
Fig. 9 is a block diagram of a point cloud data processing apparatus according to still another exemplary embodiment of the present application.
As shown in fig. 9, the present embodiment provides a point cloud data processing apparatus, including:
an obtaining module 41, configured to obtain point cloud data in a target area;
a clustering module 42, configured to cluster the point cloud data according to a plurality of anchor frames with different yaw angles;
an extracting module 43, configured to extract data features of the point cloud data in each anchor point frame;
a generating module 44, configured to generate a feature map according to the extracted data features;
and the processing module 45 is configured to perform feature fusion processing on the feature map, and generate a vehicle detection result after performing regression processing and classification processing on the processed feature map.
The present embodiment provides a point cloud data processing apparatus, including: an obtaining module 41, configured to obtain point cloud data in a target area; a clustering module 42, configured to cluster the point cloud data according to a plurality of anchor frames with different yaw angles; an extracting module 43, configured to extract data features of the point cloud data in each anchor point frame; a generating module 44, configured to generate a feature map according to the extracted data features; and the processing module 45 is configured to perform feature fusion processing on the feature map, and generate a vehicle detection result after performing regression processing and classification processing on the processed feature map.
In the device that this embodiment provided, adopted a plurality of anchor point frames to cluster point cloud data, and a plurality of anchor point frames that are used for clustering point cloud data all have different yaw angles, can guarantee from this that the point cloud data that the clustering obtained have the variety, the low-dimensional data characteristic kind of extraction is abundanter. Corresponding feature maps can be generated by extracting the data features of the cloud data, and further the deep convolutional network can be directly utilized for processing. Then, the vehicle detection result can be finally obtained through the subsequent fusion network or classification network.
The specific principle and implementation of the point cloud data processing apparatus provided in this embodiment are similar to those of the related technical contents in the method embodiment shown in fig. 2, and the related technical contents may refer to the foregoing descriptions and are not described herein again.
On the basis of the foregoing embodiment, in the point cloud data processing apparatus provided in this embodiment, optionally, the processing module 45 is specifically configured to:
and performing feature fusion processing on the feature map, and generating a vehicle detection result after regression processing and classification processing on the processed feature map.
Optionally, when performing the feature fusion processing on the feature map, the feature fusion processing is performed on the feature map according to the Hrnet network structure.
The specific principle and implementation of the apparatus provided in this embodiment are similar to those of the embodiment shown in fig. 9, and are not described herein again.
Fig. 10 is a block diagram of a point cloud data processing apparatus according to an exemplary embodiment of the present application.
As shown in fig. 10, the present embodiment provides a point cloud data processing apparatus, including:
a memory 51;
a processor 52; and
a computer program;
wherein the computer program is stored in the memory 51 and configured to be executed by the processor 52 to implement any one of the point cloud data processing methods as described above.
The present embodiments also provide a computer-readable storage medium, having stored thereon a computer program,
the computer program is executed by the processor 52 to implement the point cloud data processing method as described above.
The present embodiment also provides a computer program comprising a program code for executing any one of the point cloud data processing methods described above when the computer program is executed by a computer.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A point cloud data processing method is characterized by comprising the following steps:
acquiring point cloud data in a target area;
clustering the point cloud data according to a plurality of anchor point frames with different yaw angles;
extracting data characteristics of the point cloud data in each anchor point frame;
and generating a feature map according to the extracted data features.
2. The point cloud data processing method of claim 1, wherein the feature map is subjected to feature fusion processing, and the processed feature map is subjected to regression processing and classification processing to generate a vehicle detection result.
3. The point cloud data processing method of claim 2, wherein the feature map is subjected to feature fusion processing according to an Hrnet network structure.
4. The point cloud data processing method of any of claims 1-3, wherein the clustering the point cloud data according to anchor blocks of different yaw angles comprises:
and projecting the point cloud data along an x-y plane of the target area, and gathering the point cloud data projected into the same anchor point frame into one type.
5. The point cloud data processing method of claim 4, wherein the anchor boxes are generated in a manner that includes:
generating image meshes in an x-y plane of the target area, each of the image meshes generating a predetermined number of the anchor boxes.
6. The point cloud data processing method of claim 5, wherein the center of each image grid generates 4 anchor boxes with different yaw angles.
7. The point cloud data processing method of any of claims 1-3, wherein the extracting data features of the point cloud data in each of the anchor boxes comprises:
and selecting a predetermined amount of point cloud data in each anchor point frame, and extracting the data characteristics of the selected point cloud data according to a Pointernet network.
8. A point cloud data processing apparatus, comprising:
the acquisition module is used for acquiring point cloud data in a target area;
the clustering module is used for clustering the point cloud data according to a plurality of anchor point frames with different yaw angles;
the extraction module is used for extracting the data characteristics of the point cloud data in each anchor point frame;
and the generating module is used for generating a feature map according to the extracted data features.
9. A point cloud data processing apparatus, characterized by comprising:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the point cloud data processing method of any of claims 1-7.
10. A computer-readable storage medium, having stored thereon a computer program,
the computer program is executed by a processor to implement the point cloud data processing method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911221610.3A CN110991534A (en) | 2019-12-03 | 2019-12-03 | Point cloud data processing method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911221610.3A CN110991534A (en) | 2019-12-03 | 2019-12-03 | Point cloud data processing method, device, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110991534A true CN110991534A (en) | 2020-04-10 |
Family
ID=70089613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911221610.3A Pending CN110991534A (en) | 2019-12-03 | 2019-12-03 | Point cloud data processing method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110991534A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539949A (en) * | 2020-05-12 | 2020-08-14 | 河北工业大学 | Point cloud data-based lithium battery pole piece surface defect detection method |
| CN111667522A (en) * | 2020-06-04 | 2020-09-15 | 上海眼控科技股份有限公司 | Three-dimensional laser point cloud densification method and equipment |
| CN111860493A (en) * | 2020-06-12 | 2020-10-30 | 北京图森智途科技有限公司 | Target detection method and device based on point cloud data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018133851A1 (en) * | 2017-01-22 | 2018-07-26 | 腾讯科技（深圳）有限公司 | Point cloud data processing method and apparatus, and computer storage medium |
| CN109559300A (en) * | 2018-11-19 | 2019-04-02 | 上海商汤智能科技有限公司 | Image processing method, electronic equipment and computer readable storage medium |
| CN109948661A (en) * | 2019-02-27 | 2019-06-28 | 江苏大学 | A kind of 3D vehicle checking method based on Multi-sensor Fusion |
| CN109993103A (en) * | 2019-03-29 | 2019-07-09 | 华南理工大学 | A kind of Human bodys' response method based on point cloud data |
| CN110472534A (en) * | 2019-07-31 | 2019-11-19 | 厦门理工学院 | 3D object detection method, device, equipment and storage medium based on RGB-D data |
-
2019
- 2019-12-03 CN CN201911221610.3A patent/CN110991534A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018133851A1 (en) * | 2017-01-22 | 2018-07-26 | 腾讯科技（深圳）有限公司 | Point cloud data processing method and apparatus, and computer storage medium |
| CN109559300A (en) * | 2018-11-19 | 2019-04-02 | 上海商汤智能科技有限公司 | Image processing method, electronic equipment and computer readable storage medium |
| CN109948661A (en) * | 2019-02-27 | 2019-06-28 | 江苏大学 | A kind of 3D vehicle checking method based on Multi-sensor Fusion |
| CN109993103A (en) * | 2019-03-29 | 2019-07-09 | 华南理工大学 | A kind of Human bodys' response method based on point cloud data |
| CN110472534A (en) * | 2019-07-31 | 2019-11-19 | 厦门理工学院 | 3D object detection method, device, equipment and storage medium based on RGB-D data |
Non-Patent Citations (3)
| Title |
|---|
| WALEED ALI ET AL.: "YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud" * |
| WEN HAO ET AL.: "Slice-based building facade reconstruction from 3D point clouds" * |
| 李艳红: "车载移动测量系统数据配准与分类识别关键技术研究" * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539949A (en) * | 2020-05-12 | 2020-08-14 | 河北工业大学 | Point cloud data-based lithium battery pole piece surface defect detection method |
| CN111539949B (en) * | 2020-05-12 | 2022-05-13 | 河北工业大学 | Point cloud data-based lithium battery pole piece surface defect detection method |
| CN111667522A (en) * | 2020-06-04 | 2020-09-15 | 上海眼控科技股份有限公司 | Three-dimensional laser point cloud densification method and equipment |
| CN111860493A (en) * | 2020-06-12 | 2020-10-30 | 北京图森智途科技有限公司 | Target detection method and device based on point cloud data |
| CN111860493B (en) * | 2020-06-12 | 2024-02-09 | 北京图森智途科技有限公司 | Target detection method and device based on point cloud data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11295517B2 (en) | Generating realistic point clouds | |
| CN111443359B (en) | Positioning method, device and equipment | |
| CN108416266B (en) | Method for rapidly identifying video behaviors by extracting moving object through optical flow | |
| CN110991534A (en) | Point cloud data processing method, device, equipment and computer readable storage medium | |
| CN112613378B (en) | 3D target detection method, system, medium and terminal | |
| CN111144304A (en) | Vehicle target detection model generation method, vehicle target detection method and device | |
| JP4567660B2 (en) | A method for determining a segment of an object in an electronic image. | |
| CN111476242A (en) | Laser point cloud semantic segmentation method and device | |
| CN110930503B (en) | Clothing three-dimensional model building method, system, storage medium and electronic equipment | |
| CN113012063B (en) | Dynamic point cloud repairing method and device and computer equipment | |
| CN111754618B (en) | Object-oriented live-action three-dimensional model multi-level interpretation method and system | |
| CN112418165B (en) | Small-size target detection method and device based on improved cascade neural network | |
| CN114648640B (en) | Target object monomer method, device, equipment and storage medium | |
| CN113344110B (en) | Fuzzy image classification method based on super-resolution reconstruction | |
| CN112154448A (en) | Target detection method and device and movable platform | |
| CN111783722B (en) | Lane line extraction method of laser point cloud and electronic equipment | |
| CN111783721A (en) | Lane line extraction method of laser point cloud and electronic equipment | |
| CN108090455B (en) | Cascade mechanism-based parking space line vertex positioning method, system, terminal and medium | |
| CN116030445A (en) | Automatic driving real-time three-dimensional target detection method combining point cloud shape characteristics | |
| CN113343987B (en) | Text detection processing method and device, electronic equipment and storage medium | |
| Li et al. | Primitive fitting using deep geometric segmentation | |
| Liu | Robust segmentation of raw point clouds into consistent surfaces | |
| CN116091887A (en) | Weak supervision-based target detection model training method and device | |
| JP3944034B2 (en) | Partition data creation method and apparatus | |
| CN106909936B (en) | Vehicle detection method based on double-vehicle deformable component model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| AD01 | Patent right deemed abandoned | ||
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20240517 |