CN115330630A - Point cloud data enhancement method and device for automatic driving data set construction in mining area - Google Patents
Point cloud data enhancement method and device for automatic driving data set construction in mining area Download PDFInfo
- Publication number
- CN115330630A CN115330630A CN202211005162.5A CN202211005162A CN115330630A CN 115330630 A CN115330630 A CN 115330630A CN 202211005162 A CN202211005162 A CN 202211005162A CN 115330630 A CN115330630 A CN 115330630A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- target
- frame
- enhanced
- ground
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000005065 mining Methods 0.000 title claims abstract description 52
- 238000010276 construction Methods 0.000 title claims description 3
- 230000004888 barrier function Effects 0.000 claims abstract description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 7
- 230000003014 reinforcing effect Effects 0.000 claims description 2
- 238000004519 manufacturing process Methods 0.000 description 10
- 230000036544 posture Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 3
- 230000001965 increasing effect Effects 0.000 description 3
- 230000002349 favourable effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a point cloud data enhancement method and device, a storage medium and computer equipment which are built for an automatic driving data set of a mining area, wherein the method comprises the following steps: determining a point cloud frame to be enhanced from a plurality of basic point cloud frames, wherein the basic point cloud frames are obtained based on a conventional mining area environment; determining at least one target point cloud from a plurality of sample point clouds, wherein each sample point cloud corresponds to a target obstacle and is provided with an area label, and the target point cloud is obtained based on a preset environment; and superposing the at least one target point cloud to the point cloud frame to be enhanced based on the area label of the target point cloud to obtain an enhanced point cloud frame. This application is through in overlapping at least one target point cloud that corresponds the target barrier to treating the enhancement point cloud frame, and then obtains the point cloud frame after the enhancement, can promote sample abundance greatly, and maneuverability is stronger simultaneously.
Description
Technical Field
The application relates to the technical field of data processing, in particular to a point cloud data enhancement method and device, a storage medium and computer equipment which are built for an automatic driving data set of a mining area.
Background
In an automatic driving scheme of mining equipment in a mining area environment, a deep learning method based on laser radar point cloud is an effective means for improving three-dimensional environment perception performance. However, when sample data for training the deep learning model is acquired, the richness of the sample data is often influenced due to the difficulty in data acquisition of some moving obstacles in a mining area. On one hand, as the mining area is a non-open scene, a plurality of non-production vehicles such as minibuses, pickup trucks and other passenger vehicles can enter the limited mining area environment only under the condition of acquiring limited road right, and the entering time is strictly controlled; on the other hand, the collected engineering vehicles or non-production vehicles have limited scenes and postures in the mining area. Under the condition, the method for enhancing the point cloud data commonly used for public roads, such as random rotation, random turning and the like, can only increase the richness of a small amount of sample data, and cannot fundamentally solve the problem that the richness of the sample data is seriously insufficient in the mining area environment.
Disclosure of Invention
In view of this, the application provides a point cloud data enhancement method and apparatus, a storage medium, and a computer device which are built for an automatic driving data set in a mining area, and at least one target point cloud corresponding to a target obstacle is superimposed on a point cloud frame to be enhanced, so that an enhanced point cloud frame is obtained, the sample richness can be greatly improved, and meanwhile, the operability is strong.
According to one aspect of the application, a point cloud data enhancement method constructed for an automatic driving data set of a mining area is provided, and comprises the following steps:
determining a point cloud frame to be enhanced from a plurality of basic point cloud frames, wherein the basic point cloud frames are obtained based on a conventional mining area environment;
determining at least one target point cloud from a plurality of sample point clouds, wherein each sample point cloud corresponds to a target obstacle and is provided with an area label, and the target point cloud is obtained based on a preset environment;
and superposing the at least one target point cloud to the point cloud frame to be enhanced based on the area label of the target point cloud to obtain an enhanced point cloud frame.
According to another aspect of this application, a point cloud data reinforcing means towards mining area autopilot data set is put up is provided, includes:
the device comprises a first determining module, a second determining module and a third determining module, wherein the first determining module is used for determining a point cloud frame to be enhanced from a plurality of basic point cloud frames, and the basic point cloud frames are obtained based on a conventional mining area environment;
the second determining module is used for determining at least one target point cloud from a plurality of sample point clouds, each sample point cloud corresponds to a target obstacle and is provided with an area label, and the target point cloud is obtained based on a preset environment;
and the point cloud overlapping module is used for overlapping the at least one target point cloud to the point cloud frame to be enhanced based on the area label of the target point cloud to obtain an enhanced point cloud frame.
According to yet another aspect of the present application, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements a method of enhancing point cloud data constructed for a mine autodrive dataset as described above.
According to still another aspect of the application, a computer device is provided, which includes a storage medium, a processor, and a computer program stored on the storage medium and operable on the processor, and when the processor executes the program, the processor implements the above point cloud data enhancement method built for the mine area automatic driving dataset.
By means of the technical scheme, the point cloud data enhancement method and device, the storage medium and the computer equipment which are set up facing the automatic driving data set of the mining area can firstly find out a point cloud frame to be enhanced from a plurality of basic point cloud frames. One or more target point clouds may then be determined from the plurality of sample point clouds. Here, each sample point cloud may carry an area label. After one or more target point clouds are determined, the corresponding target point clouds can be superposed in the point cloud frame to be enhanced on the basis of the area label corresponding to each target point cloud, and finally the enhanced point cloud frame is obtained. The embodiment of the application superposes at least one target point cloud corresponding to the target barrier to the point cloud frame to be enhanced, so that the enhanced point cloud frame is obtained, the sample richness can be greatly improved, and meanwhile, the operability is strong.
The above description is only an overview of the technical solutions of the present application, and the present application may be implemented in accordance with the content of the description so as to make the technical means of the present application more clearly understood, and the detailed description of the present application will be given below in order to make the above and other objects, features, and advantages of the present application more clearly understood.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 shows a schematic flow diagram of a point cloud data enhancement method constructed for an automatic driving data set in a mining area according to an embodiment of the present application;
fig. 2 shows a schematic flow diagram of another point cloud data enhancement method built for a mining area automatic driving data set according to the embodiment of the present application;
FIG. 3 is a schematic diagram illustrating a target occlusion area provided by an embodiment of the application;
fig. 4 shows a schematic structural diagram of a point cloud data enhancement device constructed for an automatic driving data set in a mining area according to an embodiment of the present application.
Detailed Description
The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
In this embodiment, a point cloud data enhancement method constructed for an automatic driving data set in a mining area is provided, as shown in fig. 1, the method includes:
according to the point cloud data enhancement method for the mining area automatic driving data set building, firstly, a point cloud frame to be enhanced can be found out from a plurality of basic point cloud frames. Specifically, the point cloud frame to be enhanced may be randomly selected from the base point cloud frame, or may be obtained according to other preset rules. The basic point cloud frame can comprise conventional scenes, conventional vehicles and the like under conventional mining area environments, namely, the object point cloud in the basic point cloud frame can be the point cloud corresponding to objects frequently encountered in the daily working process of the mining area automatic driving equipment.
in this embodiment, one or more target point clouds may then be determined from the plurality of sample point clouds. Here, the sample point cloud may be a point cloud corresponding to a target obstacle, and the target obstacle may be an object that the mine automatic driving equipment can encounter at a specific time or under a specific condition, for example, since the mine is a non-open scene, many non-production vehicles such as passenger cars and pickup trucks can enter a limited mine environment only when limited road rights are obtained, and the entering time is strictly controlled, so these non-production vehicles such as passenger cars and pickup trucks can be used as the target obstacle. In addition, the target obstacle may also be an object in a specific position and a specific posture, for example, some collectable engineering vehicles or non-production vehicles in a mine area have limited postures, and then the engineering vehicles or the non-production vehicles in the specific position and the specific posture can also be used as the target obstacle. By constructing the preset environment, the preset environment can contain the target obstacles which can be met only under the conditions of specific time, specific conditions and the like, the point clouds of the target obstacles can be obtained, and the method and the device are very favorable for increasing the sample richness subsequently. Similarly, the target point cloud may be randomly selected from the sample point cloud, or may be obtained according to other preset rules. Each sample point cloud may carry an area label. The area tag may include a position and a size corresponding to the 3D detection frame or bounding box of the target obstacle, and a heading of the target obstacle.
And 103, superposing the at least one target point cloud to the point cloud frame to be enhanced based on the area label of the target point cloud to obtain an enhanced point cloud frame.
In this embodiment, after one or more target point clouds are determined, the corresponding target point clouds may be superimposed on the point cloud frame to be enhanced based on the area tag corresponding to each target point cloud, so as to obtain an enhanced point cloud frame. The enhanced point cloud frame can be subsequently used for training the unmanned deep learning model, and is also used as a sample of the unmanned deep learning model.
By applying the technical scheme of the embodiment, firstly, a point cloud frame to be enhanced can be found out from a plurality of basic point cloud frames. One or more target point clouds may then be determined from the plurality of sample point clouds. Here, each sample point cloud may carry an area label. After one or more target point clouds are determined, the corresponding target point clouds can be superimposed on the point cloud frame to be enhanced on the basis of the area label corresponding to each target point cloud, and an enhanced point cloud frame is finally obtained. According to the method and the device, at least one target point cloud corresponding to the target barrier is superposed to the point cloud frame to be enhanced, the enhanced point cloud frame is obtained, sample richness can be greatly improved, and meanwhile operability is high.
Further, as a refinement and an extension of the specific implementation of the embodiment, in order to fully explain the specific implementation process of the embodiment, another point cloud data enhancement method constructed for the mining area autopilot data set is provided, as shown in fig. 2, the method includes:
in this embodiment, first, one point cloud frame to be enhanced may be found from a plurality of base point cloud frames. Specifically, the point cloud frame to be enhanced may be randomly selected from the base point cloud frame, or may be obtained according to other preset rules. The basic point cloud frame can comprise conventional scenes, conventional vehicles and the like under conventional mining area environments, namely, the object point cloud in the basic point cloud frame can be the point cloud corresponding to objects frequently encountered in the daily working process of the mining area automatic driving equipment.
in this embodiment, one or more target point clouds may then be determined from the plurality of sample point clouds. Here, the sample point cloud may be a point cloud corresponding to a target obstacle, and the target obstacle may be an object that the mine automatic driving equipment can encounter at a specific time or under a specific condition, for example, since the mine is a non-open scene, many non-production vehicles such as passenger cars and pickup trucks can enter a limited mine environment only when limited road rights are obtained, and the entering time is strictly controlled, so these non-production vehicles such as passenger cars and pickup trucks can be used as the target obstacle. In addition, the target obstacle may also be an object in a specific position and a specific posture, for example, some collectable engineering vehicles or non-production vehicles in a mine area have limited postures, and then the engineering vehicles or the non-production vehicles in the specific position and the specific posture can also be used as the target obstacle. According to the embodiment of the application, the preset environment is constructed, the target obstacles which can be met only under the conditions of specific time, specific conditions and the like can be contained in the preset environment, the point cloud of the target obstacles can be obtained, and the method is very favorable for increasing the sample richness subsequently. Similarly, the target point cloud may be randomly selected from the sample point cloud, or may be obtained according to other preset rules. Each sample point cloud may carry an area label. The area tag may include a position and a size corresponding to the 3D detection frame or bounding box of the target obstacle, and a heading of the target obstacle.
in this embodiment, since each target point cloud corresponds to one area tag, the position of the 3D detection frame or bounding box of the target obstacle corresponding to each target point cloud may be specifically identified from the area tags on the basis of the area tags, and the target placement position of the corresponding target point cloud in the point cloud frame to be enhanced may be determined according to the position. Because the target point cloud and the point cloud frame to be enhanced can be placed in the same coordinate system, the target placing position can be determined in the point cloud frame to be enhanced according to the position of the 3D detection frame or the bounding box of the target obstacle corresponding to the target point cloud.
in this embodiment, after determining the target placement position of any target point cloud, it may further determine whether there is a non-ground point cloud point in a ground placement area of the target point cloud in the point cloud frame to be enhanced, where the ground placement area refers to an area occupied by the target point cloud on the ground plane. The point cloud points in the point cloud frame to be enhanced can be divided into ground point cloud points and non-ground point cloud points, wherein the ground point cloud points are also point cloud points belonging to the ground, and the non-ground point cloud points are other point cloud points except the ground point cloud points.
in this embodiment, if it is determined that there are no non-ground point cloud points in the ground placement area, it is reasonable to say that the target point cloud is placed there, and at this time, the target point cloud may be superimposed on the target placement position in the point cloud frame to be enhanced. In addition, the target point cloud may further have a name tag, where the name tag may include a name of a target obstacle corresponding to the target point cloud, such as a minibus, a pickup, and the like. After the target point cloud is superposed on the point cloud frame to be enhanced, the name tag and the area tag corresponding to the target point cloud can be added into the tag file corresponding to the point cloud frame to be enhanced, so that the information included in the tag file corresponding to the point cloud frame to be enhanced is more comprehensive.
In this embodiment, after the target placement position corresponding to each target point cloud is determined, the target occlusion area corresponding to the target point cloud may also be determined according to the target placement position. The target placement position can be a three-dimensional space occupied position of the target point cloud in the point cloud frame to be enhanced, and when the mining area automatic driving equipment acquires the point cloud frame by laser scanning, a region corresponding to one surface of the target point cloud horizontally emitting towards the laser can be determined as a target shielding region. As shown in fig. 3, the right side surface of the cuboid is a surface facing the horizontal emission of the laser, and the region corresponding to the surface is the target shielding region. In the process of horizontal laser emission, the target shielding area has a blocking effect on laser emitted by the laser emitter, so that an object far away from the laser emitter cannot be scanned by taking the target shielding area as a dividing plane, target point cloud points which cannot be obtained due to the target shielding area can be determined from the first point cloud points, and the target point cloud points are deleted from the point cloud frame to be enhanced, so that the enhanced point cloud frame is obtained. Here, the first point cloud point refers to a point cloud point in the point cloud frame to be enhanced. According to the method and the device, the cloud points of the target points which cannot appear due to the influence of the target shielding area in the point cloud frame to be enhanced are deleted, so that the enhanced point cloud frame conforms to the laser radar point cloud imaging principle, and the accuracy and the authenticity of the enhanced point cloud frame are stronger.
In this embodiment of the application, optionally, before the step 204 of "determining whether there is a non-ground point cloud point in the ground placement area of any one of the target point clouds in the point cloud frame to be enhanced", the method further includes: and determining target attributes corresponding to each first point cloud point in the point cloud frame to be enhanced based on a preset ground fitting algorithm, and marking the first point cloud points one by utilizing the target attributes, wherein the target attributes comprise ground point cloud points and non-ground point cloud points.
In this embodiment, before determining whether there are non-ground point cloud points in the ground placement area, a target attribute may be marked on each first point cloud point in the point cloud frame to be enhanced, where the target attribute may include a ground point cloud point and a non-ground point cloud point. Specifically, the ground may be fitted using a preset ground fitting algorithm. Since the ground is a flat open point cloud, the preset ground fitting algorithm herein can fit the ground plane, for example, global ground fitting can be performed by ransac or ground fitting can be performed after dividing into a plurality of regions. Then, the relation between each first point cloud point and the fitted ground plane can be judged according to the ground fitting result, and whether each first point cloud point is a ground point cloud point or a non-ground point cloud point is finally determined.
In this embodiment, optionally, when the non-ground-point cloud point exists in the ground placement area, the method further includes: if the number of the non-ground point cloud points is less than or equal to a preset number threshold, overlapping any one target point cloud to the target placement position in the point cloud frame to be enhanced; and if the number of the non-ground point cloud points is larger than the preset number threshold, rotating the target placement position based on a preset angle, and judging the number of the non-ground point cloud points in the corresponding ground placement area again according to the rotated target placement position until the number of the non-ground point cloud points in the ground placement area is smaller than or equal to the preset number threshold or the number of rotation times exceeds the preset number threshold.
In this embodiment, if there are non-ground cloud points in the ground placement area, the next determination may be continued. If the number of the non-ground point cloud points in the ground placement area is smaller than or equal to the preset number threshold, an error may exist during ground plane fitting, and the corresponding target point cloud can be continuously superposed into the point cloud frame to be enhanced. If the number of the non-ground point cloud points in the ground placement area is larger than the preset number threshold, it is unreasonable to superimpose the target point cloud at the target placement position, at the moment, the target placement position of the target point cloud can be rotated, the rotating base point can be determined according to the preset rotation rule, and the rotating base point can be rotated by the preset angle each time. After the rotation, the corresponding ground placement area can be determined according to the rotated target placement position, the number of the non-ground point cloud points in the ground placement area corresponding to the rotated target placement position is judged again, and the judgment can be executed again according to the number until the number of the non-ground point cloud points in the ground placement area is less than or equal to the preset number threshold, or the judgment is finished when the rotation times of the target placement position exceed the preset number threshold. After the target placing position is rotated every time, the area label corresponding to the target point cloud can be updated. When the target placing position is unreasonable, the target placing position is continuously rotated to find a reasonable target placing position, the problem that the sample richness is difficult to promote due to the fact that the target placing position is unreasonable and overlapping is abandoned is avoided, and the sample richness is favorably stably increased.
In this embodiment of the present application, optionally, before step 201, the method further includes: acquiring a first point cloud frame acquired by the mining area automatic driving equipment in a conventional mining area environment, generating a label file corresponding to the first point cloud frame, and acquiring a basic point cloud frame based on the first point cloud frame and the label file; and acquiring a second point cloud frame acquired by the mining area automatic driving equipment in a preset environment, and determining the sample point cloud based on the second point cloud frame, wherein the preset environment comprises the target obstacle.
In this embodiment, before determining the point cloud frame to be enhanced, a first point cloud frame acquired by the mine area automatic driving device in a normal mine area environment may be acquired, where the normal mine area environment is also a daily working environment of the mine area automatic driving device. The first point cloud frame may be collected by the mining area autopilot device under different weather and time conditions in a conventional mining area environment. After the first point cloud frame is obtained, different objects in the first point cloud frame can be identified and labeled, and therefore a corresponding label file is generated. Specifically, object name information, object position information, object size information, object heading information, and the like of different objects may be included in the tag file. And then, obtaining a basic point cloud frame based on the first point cloud frame and the corresponding tag file, namely, the basic point cloud frame is the first point cloud frame with the tag file. Subsequently, when the basic point cloud frame is used as the point cloud frame to be enhanced and the target point cloud is superposed on the point cloud frame to be enhanced, the label file can be updated by using the name label and the area label of the target point cloud.
In addition, a second point cloud frame acquired by the mining area automatic driving equipment in a preset environment can be acquired, and then the sample point cloud can be determined based on the second point cloud frame. Wherein the preset environment may include a target obstacle. The second point cloud frame may be a point cloud frame of the target obstacle at a different position, a different relative pose, from the mine autopilot. Specifically, the mine area automatic driving equipment used for collecting the second point cloud frame can be arranged on the open platform, the mine area automatic driving equipment is arranged in the center of the platform, the size of the platform is based on the actual mine area condition, the open platform and the large platform are used as selection standards, and when the platform is small, the position of the mine area automatic driving equipment in the platform can be correspondingly adjusted, so that enough space is reserved for the movement of a target obstacle. When the position of the mine area automatic driving equipment is fixed, a target obstacle is placed on the platform, and point cloud frames moving at various positions and various relative poses relative to the mine area automatic driving equipment are collected. Then, a sample point cloud can be extracted from the second point cloud frame, that is, a point cloud corresponding to the target obstacle can be extracted from the second point cloud frame.
In this embodiment of the present application, optionally, the "determining the sample point cloud based on the second point cloud frame" includes: sorting the second point cloud frames based on the acquisition time of the second point cloud frames, and determining reference frames and search frames corresponding to the reference frames according to the sorted second point cloud frames; identifying the target obstacle in the reference frame, and labeling the point cloud corresponding to the target obstacle by using first target information to obtain a first sample point cloud, wherein the first target information comprises the area tag; determining the target attribute corresponding to each second point cloud point in the search frame based on the preset ground fitting algorithm, determining a maximum cluster closest to the target obstacle in the reference frame through an Euclidean distance clustering algorithm according to non-ground point cloud points in the second point cloud points and the first target information, and taking the maximum cluster as the point cloud corresponding to the target obstacle in the search frame; and determining a target bounding box according to the point cloud corresponding to the target obstacle in the search frame, determining second target information corresponding to the target obstacle based on the target bounding box and the first target information, and labeling the point cloud corresponding to the target obstacle in the search frame by using the second target information to obtain a second sample point cloud.
In this embodiment, when extracting the sample point cloud from the second point cloud frame, first, the second point cloud frames corresponding to the same target obstacle may be sorted according to the acquisition time of the second point cloud frames, and specifically, the second point cloud frames may be sorted according to the order of the acquisition time. After sorting, reference frames, and search frames corresponding to each reference frame, may be determined therefrom. For example, after the reference frame and the search frame are determined, a target obstacle in the reference frame can be identified, a point cloud corresponding to the target obstacle is labeled on the basis of the first target information, and a first sample point cloud can be obtained after labeling. The first target information may be obtained by manual calculation and sorting. And then, similarly, determining the target attribute of each second point cloud point in the search frame by using a preset ground fitting algorithm, wherein the search frame refers to a frame subsequent to the reference frame, namely a point cloud frame acquired by the frame subsequent to the reference frame, and the target attribute also comprises ground point cloud points and non-ground point cloud points. For example, the reference frame is the 1 st frame, then the search frame is the 2 nd frame. After labeling, the largest cluster closest to the target obstacle in the reference frame may be determined from the cloud points of the second points labeled as the cloud points of the non-ground points, specifically, the position of the target obstacle in the reference frame may be determined according to the first target information, and the largest cluster closest to the position of the target obstacle in the reference frame may be determined from the cloud points of the non-ground points in the cloud points of the second points of the search frame by using the euclidean distance clustering algorithm. Then, the maximum cluster can be used as the point cloud corresponding to the target obstacle in the search frame.
After the point cloud corresponding to the target obstacle is determined from the search frame, a target bounding box of the point cloud corresponding to the target obstacle can be further determined, after the target bounding box is determined, two pieces of course information can also be determined, and then one piece of course information is finally determined from the two pieces of course information to serve as a target course corresponding to the target obstacle in the search frame according to the target course of the target obstacle in the first target information. And then, the length, width, height and other dimensions, position information and the like corresponding to the target bounding box can be obtained, an area label corresponding to the target obstacle in the search frame is determined according to the information and the target course, in addition, other information can be obtained, and a plurality of information including the area label form second target information. And finally, marking the point cloud corresponding to the target obstacle in the search frame by using second target information, and further obtaining a second sample point cloud corresponding to the search frame. Here, each first sample point cloud and each second sample point cloud constitute a final sample point cloud.
Because the types of the target obstacles to be supplemented in the mining area are pickup, minibus and other types of target obstacles and the outlines of the target obstacles are clearer, the target bounding box can be calculated by adopting an L-shape fitting method. The fitting method comprises the following procedures: 1) Traversing all possible directions of the rectangle, and finding a rectangle which points to the direction and contains all scanning points during each iteration; 2) Obtaining the distances from all points to four sides of the rectangle, dividing the points into p and q according to the distances, and calculating corresponding square errors as a target function; 3) After iterating all directions and obtaining all corresponding square errors, finding the optimal direction of the minimum square error, and adjusting the rectangle according to the direction. When selecting a suitable target bounding box rectangle, the point-to-edge squared error minimization is used as the evaluation criterion. And after the target bounding box is obtained, calculating the center of the target bounding box. When the length, the width and the height of the target bounding box are determined, firstly, the course is determined, the slopes of two edges of a rectangle of an XY plane (parallel to the ground plane) of the target bounding box are calculated to obtain four corresponding orientation angles (each edge corresponds to two opposite orientation angles), and the direction with the minimum angle difference with the target course corresponding to the target obstacle in the reference frame is selected as the target course corresponding to the target obstacle in the search frame. The reason for this is that the time interval between adjacent frames is small, the relative motion speed is in a low speed state, and thus the heading of the target object changes little. After the target heading is determined, the side parallel to the target heading is determined as a long side, the side vertical to the target heading is determined as a wide side, and the height is determined by the maximum height difference of the point cloud points. And finally obtaining the three-dimensional target bounding box.
In this embodiment of the present application, optionally before the "sorting the second point cloud frame", the method further includes: and performing down-sampling processing on the second point cloud frame, and removing the second point cloud frame which does not contain the target obstacle after the down-sampling processing to obtain an updated second point cloud frame.
In this embodiment, the original second point cloud frame has a higher acquisition frequency, and the significance of extracting all sample point clouds from the second point cloud frames of the continuous frame is not great, because the point clouds corresponding to the target obstacle in the second point cloud frames of the continuous frame have a smaller change, the original second point cloud frame can be subjected to downsampling processing. For example, the sampling frequency of the original second point cloud frame is 10Hz, and the original second point cloud frame may be sampled at equal intervals during the down-sampling process, where the interval is 1 frame/s, that is, the down-sampling is 1Hz. Subsequently, the second point cloud frame after the down-sampling processing can be screened, the second point cloud frame which does not contain the target obstacle is removed, and then the updated second point cloud frame is obtained. According to the method and the device, the original second point cloud frame is subjected to downsampling processing, the second point cloud frame which does not contain the target obstacle is removed, the position and the pose of the target obstacle contained in the updated different second point cloud frames can be changed, and the subsequent sample point cloud extraction effect is better.
In this embodiment of the application, optionally, after the "obtaining the second sample point cloud when the reference frame corresponds to a plurality of search frames", the method further includes: updating the search frame corresponding to the second sample point cloud into a reference frame, and determining the second sample point cloud corresponding to the next search frame by using the updated reference frame until the number of the second sample point clouds is consistent with the number of the plurality of search frames.
In this embodiment, if each reference frame corresponds to a plurality of search frames, after one search frame finishes extracting the second sample point cloud each time, the next search frame can extract the second sample point cloud according to the second target information of the extracted search frame until the number of the second sample point clouds is consistent with the number of the plurality of search frames, which indicates that the search frames corresponding to the reference frames completely extract the second sample point cloud, and at this time, the process can be finished. According to the embodiment of the application, a part of reference frames are determined from the second point cloud frame, after the reference frames are marked by using the first target information, the corresponding second target information can be automatically obtained and marked by the subsequent search frame corresponding to the reference frames, the workload of manual marking is greatly reduced, and the target course of the automatically marked target obstacle can be accurately judged by using the marking result of the adjacent marking frames.
Further, as a specific implementation of the method in fig. 1, an embodiment of the present application provides a point cloud data enhancement device constructed for an automatic driving data set in a mining area, as shown in fig. 4, the device includes:
the device comprises a first determining module, a second determining module and a third determining module, wherein the first determining module is used for determining a point cloud frame to be enhanced from a plurality of basic point cloud frames, and the basic point cloud frames are obtained based on a conventional mining area environment;
the second determining module is used for determining at least one target point cloud from a plurality of sample point clouds, each sample point cloud corresponds to a target obstacle and is provided with an area label, and the target point cloud is obtained based on a preset environment;
and the point cloud overlapping module is used for overlapping the at least one target point cloud to the point cloud frame to be enhanced based on the area label of the target point cloud to obtain an enhanced point cloud frame.
Optionally, the point cloud overlay module includes:
the position determining unit is used for determining a target placement position of any one target point cloud in the point cloud frame to be enhanced based on the area label of the target point cloud;
the judging unit is used for judging whether non-ground point cloud points exist in the ground placement area of any one target point cloud in the point cloud frame to be enhanced or not based on the target placement position;
and the superposition unit is used for superposing any target point cloud to the target placement position in the point cloud frame to be enhanced when the non-ground point cloud points do not exist in the ground placement area.
Optionally, the apparatus further comprises:
and the marking module is used for determining a target attribute corresponding to each first point cloud point in the point cloud frame to be enhanced based on a preset ground fitting algorithm before judging whether any non-ground point cloud point exists in the ground placement area of the target point cloud in the point cloud frame to be enhanced, and marking the first point cloud points one by using the target attribute, wherein the target attribute comprises ground point cloud points and non-ground point cloud points.
Optionally, the apparatus further comprises:
the quantity judging module is used for superposing any one target point cloud to the target placement position in the point cloud frame to be enhanced when the non-ground point cloud points exist in the ground placement area and the quantity of the non-ground point cloud points is less than or equal to a preset quantity threshold value; and if the number of the non-ground point cloud points is larger than the preset number threshold, rotating the target placement position based on a preset angle, and judging the number of the non-ground point cloud points in the corresponding ground placement area again according to the rotated target placement position until the number of the non-ground point cloud points in the ground placement area is smaller than or equal to the preset number threshold or the number of rotation times exceeds the preset number threshold.
Optionally, the apparatus further comprises:
and the deleting module is used for determining a target shielding area corresponding to any one target point cloud according to the target placement position of any one target point cloud after the any one target point cloud is superposed to the target placement position in the point cloud frame to be enhanced, determining a target point cloud point corresponding to the target shielding area from the first point cloud point, and deleting the target point cloud point from the point cloud frame to be enhanced.
Optionally, the apparatus further comprises:
the first acquisition module is used for acquiring a first point cloud frame acquired by the mining area automatic driving equipment in a conventional mining area environment before the point cloud frame to be enhanced is determined from a plurality of basic point cloud frames, generating a label file corresponding to the first point cloud frame, and acquiring the basic point cloud frame based on the first point cloud frame and the label file;
and the second acquisition module is used for acquiring a second point cloud frame acquired by the mining area automatic driving equipment in a preset environment, and determining the sample point cloud based on the second point cloud frame, wherein the preset environment comprises the target obstacle.
Optionally, the second obtaining module is configured to:
sorting the second point cloud frames based on the acquisition time of the second point cloud frames, and determining reference frames and search frames corresponding to the reference frames according to the sorted second point cloud frames; identifying the target obstacle in the reference frame, and labeling the point cloud corresponding to the target obstacle by using first target information to obtain a first sample point cloud, wherein the first target information comprises the area tag; determining the target attribute corresponding to each second point cloud point in the search frame based on the preset ground fitting algorithm, determining a maximum cluster closest to the target obstacle in the reference frame through an Euclidean distance clustering algorithm according to non-ground point cloud points in the second point cloud points and the first target information, and taking the maximum cluster as the point cloud corresponding to the target obstacle in the search frame; determining a target bounding box according to the point cloud corresponding to the target obstacle in the search frame, determining second target information corresponding to the target obstacle based on the target bounding box and the first target information, and marking the point cloud corresponding to the target obstacle in the search frame by using the second target information to obtain a second sample point cloud.
Optionally, the apparatus further comprises:
and the updating module is used for updating the search frame corresponding to the second sample point cloud into the reference frame after the second sample point cloud is obtained when the reference frame corresponds to a plurality of search frames, and determining the second sample point cloud corresponding to the next search frame by using the updated reference frame until the number of the second sample point clouds is consistent with that of the search frames.
It should be noted that other corresponding descriptions of the functional units related to the point cloud data enhancement device built for the mine automatic driving data set according to the embodiment of the present application may refer to the corresponding descriptions in the methods of fig. 1 to fig. 3, and are not described herein again.
Based on the above method shown in fig. 1 to 3, correspondingly, an embodiment of the present application further provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for enhancing point cloud data built by an automatic driving dataset facing a mining area as shown in fig. 1 to 3 is implemented.
Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.), and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the implementation scenarios of the present application.
Based on the foregoing methods shown in fig. 1 to fig. 3 and the virtual device embodiment shown in fig. 4, in order to achieve the foregoing object, an embodiment of the present application further provides a computer device, which may specifically be a personal computer, a server, a network device, and the like, where the computer device includes a storage medium and a processor; a storage medium for storing a computer program; and the processor is used for executing a computer program to realize the point cloud data enhancement method constructed by the mining area-oriented automatic driving data set shown in the figures 1 to 3.
Optionally, the computer device may also include a user interface, a network interface, a camera, radio Frequency (RF) circuitry, sensors, audio circuitry, a WI-FI module, and so forth. The user interface may include a Display screen (Display), an input unit such as a keypad (Keyboard), etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (e.g., a bluetooth interface, a WI-FI interface), etc.
It will be appreciated by those skilled in the art that the present embodiment provides a computer device architecture that is not limiting of the computer device, and that may include more or fewer components, or some components in combination, or a different arrangement of components.
The storage medium may further include an operating system and a network communication module. An operating system is a program that manages and maintains the hardware and software resources of a computer device, supporting the operation of information handling programs, as well as other software and/or programs. The network communication module is used for realizing communication among components in the storage medium and other hardware and software in the entity device.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present application can be implemented by software plus a necessary general hardware platform, and can also be implemented by hardware. First, a point cloud frame to be enhanced can be found from a plurality of base point cloud frames. One or more target point clouds may then be determined from the plurality of sample point clouds. Here, each sample point cloud may carry an area label. After one or more target point clouds are determined, the corresponding target point clouds can be superposed in the point cloud frame to be enhanced on the basis of the area label corresponding to each target point cloud, and finally the enhanced point cloud frame is obtained. According to the method and the device, at least one target point cloud corresponding to the target barrier is superposed to the point cloud frame to be enhanced, the enhanced point cloud frame is obtained, sample richness can be greatly improved, and meanwhile operability is high.
Those skilled in the art will appreciate that the figures are merely schematic representations of one preferred implementation scenario and that the blocks or flow diagrams in the figures are not necessarily required to practice the present application. Those skilled in the art can understand that the modules in the device in the implementation scenario may be distributed in the device in the implementation scenario according to the implementation scenario description, and may also be located in one or more devices different from the present implementation scenario with corresponding changes. The modules of the implementation scenario may be combined into one module, or may be further split into a plurality of sub-modules.
The above application serial numbers are for description purposes only and do not represent the superiority or inferiority of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any variations that can be considered by those skilled in the art are intended to fall within the scope of the present application.
Claims (10)
1. A point cloud data enhancement method for mining area automatic driving data set construction is characterized by comprising the following steps:
determining a point cloud frame to be enhanced from a plurality of basic point cloud frames, wherein the basic point cloud frames are obtained based on a conventional mining area environment;
determining at least one target point cloud from a plurality of sample point clouds, wherein each sample point cloud corresponds to a target obstacle and is provided with an area label, and the target point cloud is obtained based on a preset environment;
and overlapping the at least one target point cloud to the point cloud frame to be enhanced based on the area label of the target point cloud to obtain an enhanced point cloud frame.
2. The method of claim 1, wherein the superimposing the at least one target point cloud into the frame of point clouds to be enhanced based on the area label of the target point cloud comprises:
determining a target placement position of any one target point cloud in the point cloud frame to be enhanced based on a region label of the any one target point cloud;
judging whether non-ground point cloud points exist in a ground placement area of any one target point cloud in the point cloud frame to be enhanced or not based on the target placement position;
when the non-ground point cloud points do not exist in the ground placement area, the target point cloud is superposed to the target placement position in the point cloud frame to be enhanced.
3. The method according to claim 2, wherein before the determining whether there are non-ground point cloud points in the ground placement area of any of the target point clouds in the point cloud frame to be enhanced, the method further comprises:
and determining target attributes corresponding to each first point cloud point in the point cloud frame to be enhanced based on a preset ground fitting algorithm, and marking the first point cloud points one by utilizing the target attributes, wherein the target attributes comprise ground point cloud points and non-ground point cloud points.
4. The method of claim 2 or 3, wherein when the non-ground point cloud point is present within the ground placement area, the method further comprises:
if the number of the non-ground point cloud points is less than or equal to a preset number threshold, overlapping any one target point cloud to the target placement position in the point cloud frame to be enhanced;
and if the number of the non-ground point cloud points is larger than the preset number threshold, rotating the target placement position based on a preset angle, and judging the number of the non-ground point cloud points in the corresponding ground placement area again according to the rotated target placement position until the number of the non-ground point cloud points in the ground placement area is smaller than or equal to the preset number threshold or the rotation times exceed the preset number threshold.
5. The method of claim 2, wherein after the overlaying of the any one of the target point clouds to the target placement location in the frame of point cloud to be enhanced, the method further comprises:
and determining a target shielding area corresponding to any one target point cloud according to the target placement position of any one target point cloud, determining a target point cloud point corresponding to the target shielding area from the first point cloud points, and deleting the target point cloud point from the point cloud frame to be enhanced.
6. The method of claim 1, wherein prior to determining the point cloud frame to be enhanced from the plurality of base point cloud frames, the method further comprises:
acquiring a first point cloud frame acquired by the mining area automatic driving equipment in a conventional mining area environment, generating a label file corresponding to the first point cloud frame, and acquiring a basic point cloud frame based on the first point cloud frame and the label file;
and acquiring a second point cloud frame acquired by the mining area automatic driving equipment in a preset environment, and determining the sample point cloud based on the second point cloud frame, wherein the preset environment comprises the target barrier.
7. The method of claim 6, wherein determining the sample point cloud based on the second point cloud frame comprises:
sorting the second point cloud frames based on the acquisition time of the second point cloud frames, and determining reference frames and search frames corresponding to the reference frames according to the sorted second point cloud frames;
identifying the target obstacle in the reference frame, and labeling the point cloud corresponding to the target obstacle by using first target information to obtain a first sample point cloud, wherein the first target information comprises the area label;
determining the target attribute corresponding to each second point cloud point in the search frame based on the preset ground fitting algorithm, determining a maximum cluster closest to the target obstacle in the reference frame through an Euclidean distance clustering algorithm according to non-ground point cloud points in the second point cloud points and the first target information, and taking the maximum cluster as the point cloud corresponding to the target obstacle in the search frame;
and determining a target bounding box according to the point cloud corresponding to the target obstacle in the search frame, determining second target information corresponding to the target obstacle based on the target bounding box and the first target information, and labeling the point cloud corresponding to the target obstacle in the search frame by using the second target information to obtain a second sample point cloud.
8. The utility model provides a point cloud data reinforcing means towards mining area autopilot data set is built which characterized in that includes:
the device comprises a first determining module, a second determining module and a third determining module, wherein the first determining module is used for determining a point cloud frame to be enhanced from a plurality of basic point cloud frames, and the basic point cloud frames are obtained based on a conventional mining area environment;
the second determining module is used for determining at least one target point cloud from a plurality of sample point clouds, each sample point cloud corresponds to a target obstacle and is provided with an area label, and the target point cloud is obtained based on a preset environment;
and the point cloud overlapping module is used for overlapping the at least one target point cloud into the point cloud frame to be enhanced based on the area label of the target point cloud to obtain an enhanced point cloud frame.
9. A storage medium on which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 7.
10. A computer device comprising a storage medium, a processor and a computer program stored on the storage medium and executable on the processor, characterized in that the processor implements the method of any one of claims 1 to 7 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211005162.5A CN115330630A (en) | 2022-08-22 | 2022-08-22 | Point cloud data enhancement method and device for automatic driving data set construction in mining area |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211005162.5A CN115330630A (en) | 2022-08-22 | 2022-08-22 | Point cloud data enhancement method and device for automatic driving data set construction in mining area |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115330630A true CN115330630A (en) | 2022-11-11 |
Family
ID=83926748
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211005162.5A Pending CN115330630A (en) | 2022-08-22 | 2022-08-22 | Point cloud data enhancement method and device for automatic driving data set construction in mining area |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115330630A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117689908A (en) * | 2023-12-11 | 2024-03-12 | 深圳技术大学 | Stair point cloud data enhancement method and device, intelligent terminal and storage medium |
-
2022
- 2022-08-22 CN CN202211005162.5A patent/CN115330630A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117689908A (en) * | 2023-12-11 | 2024-03-12 | 深圳技术大学 | Stair point cloud data enhancement method and device, intelligent terminal and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10049492B2 (en) | Method and apparatus for rendering facades of objects of interest from three-dimensional point clouds | |
| US10198632B2 (en) | Survey data processing device, survey data processing method, and survey data processing program | |
| US9846946B2 (en) | Objection recognition in a 3D scene | |
| JP6259928B2 (en) | Lane data processing method, apparatus, storage medium and equipment | |
| Sohn et al. | Using a binary space partitioning tree for reconstructing polyhedral building models from airborne lidar data | |
| EP2874097A2 (en) | Automatic scene parsing | |
| EP2120009B1 (en) | Measuring device and measuring method | |
| JP5822255B2 (en) | Object identification device and program | |
| CN108052624A (en) | Processing Method of Point-clouds, device and computer readable storage medium | |
| CN110502983B (en) | Method and device for detecting obstacles in expressway and computer equipment | |
| CN115049700A (en) | Target detection method and device | |
| CN113593017A (en) | Method, device and equipment for constructing surface three-dimensional model of strip mine and storage medium | |
| CN115330630A (en) | Point cloud data enhancement method and device for automatic driving data set construction in mining area | |
| Dickenson et al. | Rotated rectangles for symbolized building footprint extraction | |
| CN116643291A (en) | SLAM method for removing dynamic targets by combining vision and laser radar | |
| JP2021179839A (en) | Classification system of features, classification method and program thereof | |
| CN113763438B (en) | Point cloud registration method, device, equipment and storage medium | |
| WO2024027587A1 (en) | Dynamic obstacle removing method and apparatus for laser point cloud, and electronic device | |
| CN115683109B (en) | Visual dynamic obstacle detection method based on CUDA and three-dimensional grid map | |
| CN114627395B (en) | Multi-rotor unmanned aerial vehicle angle analysis method, system and terminal based on nested targets | |
| CN115565072A (en) | Road garbage recognition and positioning method and device, electronic equipment and medium | |
| CN114998855A (en) | Road edge line generation method and device, storage medium and computer equipment | |
| CN114511582A (en) | Automatic ancient city battlement extraction method | |
| CN113375656A (en) | Positioning method and device | |
| CN111860084A (en) | Image feature matching and positioning method and device and positioning system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |