CN117553686B - Laser radar point cloud-based carriage bulk cargo overrun detection method - Google Patents
Laser radar point cloud-based carriage bulk cargo overrun detection method Download PDFInfo
- Publication number
- CN117553686B CN117553686B CN202410049227.9A CN202410049227A CN117553686B CN 117553686 B CN117553686 B CN 117553686B CN 202410049227 A CN202410049227 A CN 202410049227A CN 117553686 B CN117553686 B CN 117553686B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- axis
- point
- carriage
- rotation matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 53
- 239000011159 matrix material Substances 0.000 claims abstract description 75
- 238000000034 method Methods 0.000 claims abstract description 58
- 238000001914 filtration Methods 0.000 claims abstract description 28
- 230000009466 transformation Effects 0.000 claims abstract description 22
- 230000008569 process Effects 0.000 claims abstract description 20
- 239000000463 material Substances 0.000 claims abstract description 17
- 239000013598 vector Substances 0.000 claims description 75
- 239000013590 bulk material Substances 0.000 claims description 39
- 238000000605 extraction Methods 0.000 claims description 19
- 238000004364 calculation method Methods 0.000 claims description 18
- 238000012937 correction Methods 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 8
- 230000000007 visual effect Effects 0.000 claims description 8
- 230000002349 favourable effect Effects 0.000 claims description 7
- SAZUGELZHZOXHB-UHFFFAOYSA-N acecarbromal Chemical compound CCC(Br)(CC)C(=O)NC(=O)NC(C)=O SAZUGELZHZOXHB-UHFFFAOYSA-N 0.000 claims 1
- 239000000284 extract Substances 0.000 claims 1
- 238000011897 real-time detection Methods 0.000 abstract description 3
- 230000000903 blocking effect Effects 0.000 abstract 1
- 238000012545 processing Methods 0.000 description 4
- 241000764238 Isis Species 0.000 description 2
- 230000007797 corrosion Effects 0.000 description 2
- 238000005260 corrosion Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000007599 discharging Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- Radar, Positioning & Navigation (AREA)
- Electromagnetism (AREA)
- Computer Networks & Wireless Communication (AREA)
- Artificial Intelligence (AREA)
- Remote Sensing (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a carriage bulk cargo overrun detection method based on laser radar point cloud, which comprises the following two steps: the method comprises the steps that after vehicle point cloud data comprising vehicles and a feed opening are collected through a laser radar, redundant large blocks of miscellaneous points in the environment are removed through direct filtering, then point clouds are segmented through an European clustering algorithm, a rotation matrix is obtained, pose transformation is carried out on all the segmented point clouds, and finally information of the vehicles and the feed opening is extracted and recorded, so that the problem of carriage information loss caused by blocking a carriage by bulk materials in the loading process is avoided; the other step is that the blanking is controlled to start to acquire detection point cloud data with bulk cargo in a carriage, the rotation matrix is used for carrying out pose transformation on the detection point cloud, the blanking port point cloud is removed, finally, the direct filtering is used for extracting the bulk cargo point cloud from the processed detection point cloud, and whether the bulk cargo point cloud is out of limit is judged, so that the real-time detection of the loading capacity of the bulk cargo is facilitated, and the delay and the hysteresis of the data are reduced.
Description
Technical Field
The invention relates to a point cloud detection technology, in particular to a carriage bulk cargo overrun detection method based on laser radar point cloud.
Background
The existing bulk loading height depends on manual command at a loading site, after a vehicle runs to a loading site, whether the vehicle reaches a discharging position is judged through line drawing marks on the site, then a command control room is commanded to perform discharging operation, the loading site is often seriously polluted by dust, and workers observe and command the site to greatly damage human bodies; after the bulk cargo is loaded for a period of time, the vehicle needs to be moved to continue loading, and a worker is also required to judge whether the bulk cargo fills the carriage of the section and command the vehicle to move, so that the dependency on the worker is large, and the degree of automation is low.
The Chinese patent with publication number of CN115953400A discloses an automatic detection method of corrosion pits based on the surface of a three-dimensional point cloud object, wherein the method automatically identifies the corrosion pits on the surface of the object by automatically matching the point cloud on the surface of the photographed object with the point cloud on the surface of the ideal object, and provides a thinking for detecting the object based on the point cloud, but can not be directly applied to the specific scene of detecting the overrun of bulk cargo in a carriage.
Disclosure of Invention
Aiming at the problems, the invention provides a carriage bulk cargo overrun detection method based on laser radar point cloud.
The technical scheme adopted is that the carriage bulk cargo overrun detection method based on the laser radar point cloud comprises the following steps:
extracting information of a vehicle and a feed opening, and acquiring a rotation matrix;
extracting the point cloud of bulk materials and judging whether the bulk materials exceed the limit;
Wherein:
The method for extracting the information of the vehicle and the feed opening and obtaining the rotation matrix comprises the following steps:
s1, acquiring point cloud data PCD comprising a vehicle and a feed opening by using a laser radar;
s2, removing the miscellaneous points in the environment through direct filtering;
S3, dividing the point cloud data into an auxiliary Ping Miandian cloud, a carriage point cloud and a feed opening point cloud through an European clustering algorithm;
s4, performing plane fitting on the auxiliary Ping Miandian cloud;
s5, extracting Ping Miandian cloud at the bottom of the carriage through direct filtering and fitting a plane;
s6, checking the position and the posture of the point cloud after the posture correction in a visual window, and judging whether the information extraction is facilitated;
the method for extracting the point cloud of the bulk material and judging whether the point cloud of the bulk material exceeds the limit comprises the following steps:
A1. Inputting a bulk material height limit value h, and acquiring cloud data of detection points with bulk materials in a carriage;
A2. Performing pose transformation on the cloud data of the detection points;
A3. extracting a bulk material point cloud from the detection point cloud data through direct filtering, and removing floating miscellaneous points in the bulk material point cloud by using European cluster segmentation;
A4. And judging whether the maximum value of the bulk material point cloud in the vertical direction is smaller than the maximum value of the vehicle compartment in the vertical direction minus the bulk material height limit value.
Further, in S1, whether the vehicle and the blanking port in the point cloud data reach the designated position is checked through the visualization window, and if the vehicle and the blanking port do not reach the designated position, the vehicle or the blanking port needs to be moved until the designated position is reached.
Optionally, in S2, the three-dimensional filtering direction vector is set to be (a, B, C), and two planes parallel to each other and not coincident with each other are established perpendicular to the three-dimensional filtering direction vector:
Ax+By+Cz+D1=0；
Ax+By+Cz+D2=0；
Meanwhile, large blocks of miscellaneous points are removed based on the following formula:
；
；
wherein x, y and z are three axes of a Cartesian three-dimensional rectangular coordinate system respectively, PCD is point cloud data which are acquired by using a laser radar and comprise a vehicle and a feed opening, A, B, C is components of three-dimensional filtering direction vectors in the x, y and z directions respectively, and D1 and D2 are constant numbers of two planes respectively.
Further, in S3, in the euclidean clustering algorithm, a distance threshold is set, a distance between points is calculated, if the distance is smaller than the distance threshold, the two points are divided into the same point set, otherwise, the two points are divided into different point sets; and simultaneously setting a minimum clustering point number C min and removing point sets with the point number smaller than C min.
Optionally, in S4, in performing plane fitting on the auxiliary Ping Miandian cloud, calculating an included angle α and a rotation axis vector K1 between a normal vector N1 and a y axis of the fitting plane, then calculating a rotation matrix R1 through the included angle α and the rotation axis vector K1, and finally performing pose transformation on all the split point clouds by using the rotation matrix R1 as the y axis rotation matrix, where a calculation formula of the included angle α is as follows:
；
wherein alpha is the included angle between the normal vector N1 of the fitting plane and the y axis, The components of the normal vector N1 in the x, y and z directions are respectively;
The calculation formula of the rotation axis vector K1 is:
；
In the method, in the process of the invention, Is a vector of the rotation axis,The components of the normal vector N1 in the x, y and z directions are respectively;
The calculation formula of the rotation matrix R1 is:
；
Wherein R1 is a rotation matrix, E is an identity matrix, alpha is an included angle between a normal vector N1 of a fitting plane and a y axis, Is the product of the transposed vectors of vectors K1 and K1;
。
Further, in S5, the cloud of the bottom Ping Miandian of the carriage is extracted through filtering and fitted to the plane, and the included angle between the normal vector N2 of the fitted plane and the x-axis is calculated And an axis of rotation vector K2, then using the included angle/>And a rotation axis vector K2 is used for calculating a rotation matrix R2, and finally the rotation matrix R2 is used as an x-axis rotation matrix to perform pose transformation on all the segmented point clouds, wherein the included angle/>The calculation formula of (2) is as follows:
；
In the method, in the process of the invention, To fit the angle between the normal vector N2 to the plane and the x-axis,、、Components of the vector N2 in the x, y and z directions respectively;
the calculation formula of the rotation axis vector K2 is:
In the method, in the process of the invention, Is a vector of the rotation axis,、、The components of the normal vector N2 in the x, y and z directions are respectively;
The calculation formula of the rotation matrix R2 is:
；
Wherein R2 is a rotation matrix, E is an identity matrix, To fit the angle between the normal vector N2 to the plane and the x-axis,Is the product of the transposed vectors of vectors K2 and K2;
。
Optionally, in S6, whether the position and the posture of the point cloud after the pose correction are favorable for information extraction is checked in the visual window, if not, the vehicle or the feed opening is required to be moved, and the operations from S1 to S5 are re-executed, if the pose is favorable for information extraction, the maximum value and the minimum value of the y-axis rotation matrix R1, the x-axis rotation matrix R2, the feed opening point cloud, the vehicle compartment in the x-axis, and the maximum value and the minimum value of the vehicle compartment in the y-axis are extracted and recorded, wherein if the two are parallel, the information extraction is facilitated by observing whether the plane point cloud of the side surface of the compartment is parallel to the yoz plane and the plane point cloud of the bottom of the compartment is parallel to the xoz plane.
Further, in A2, the pose of the detection point cloud is transformed by using the y-axis rotation matrix R1 and the x-axis rotation matrix R2 to obtain the point cloudThe formula is as follows:
；
In the method, in the process of the invention, Is the detection point cloud before pose transformation,/>The method is characterized in that the method is a detection point cloud after pose transformation, R1 is a y-axis rotation matrix, and R2 is an x-axis rotation matrix;
Then carrying out KdTree radius search on the detection point cloud through the blanking port point cloud, and removing the searched point cloud from the detection point cloud, wherein the removed point cloud is The expression is:
；
In the method, in the process of the invention, Is a removed point cloud,/>For the cloud of the blanking port point extracted in S3,/>Is the detection point cloud after pose transformation,/>Search radius of KdTree,/>For/>Points and/>Distance between the points in (a) andThe expression can be represented by the following formula:
；
；
In the method, in the process of the invention, For/>Is a dot in (2);
For/> Is a dot in (2);
i is point cloud Point number of (n) >, n isIs the number of points of (3);
j is the point cloud Point number m is/>Is the number of points of (3);
Respectively, are dot/> Components in the x, y, z directions;
Respectively, are dot/> Components in x, y, z directions.
Optionally, in A3, the removing floating miscellaneous points in the bulk material point cloud by the european style clustering segmentation sets a minimum clustering point number C min, and excludes point sets with a point number less than C min from the algorithm result.
Further, in A4, extracting the maximum value of the bulk material point cloud in the y axis, judging whether the maximum value of the bulk material point cloud in the y axis is smaller than the maximum value of a carriage of the vehicle in the y axis minus the bulk material height limit value h, if so, sending out an overrun warning, otherwise, continuing blanking, and detecting the cloud data of the detection point of the next frame.
The beneficial effects of the invention at least comprise one of the following;
1. the carriage bulk cargo overrun detection method based on the laser radar point cloud is characterized in that after a vehicle and a feed opening reach a specified position, information of the vehicle carriage and the feed opening is acquired before bulk cargo is loaded, and the method is used for the subsequent processing process of the bulk cargo point cloud, so that the problem of carriage information deletion caused by the shielding of the carriage by bulk cargo in the loading process is avoided, and the influence of the feed opening on bulk cargo information extraction is avoided.
2. When the material is not filled, the y-axis rotation matrix, the x-axis rotation matrix, the material outlet point cloud, the maximum and minimum values of the vehicle carriage in the x-axis, and the maximum and minimum values of the vehicle carriage in the y-axis are extracted and recorded, and the information is used in each subsequent frame of point cloud, so that the subsequent point cloud processing speed can be improved, the real-time detection of the bulk material loading capacity is facilitated, and the delay and the hysteresis of data are reduced.
Drawings
FIG. 1 is a flow chart of vehicle and feed opening information extraction and rotation matrix acquisition;
FIG. 2 is a flow chart of bulk point cloud extraction;
FIG. 3 is a detection flow chart;
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.
It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complicated.
Example 1
A carriage bulk cargo overrun detection method based on laser radar point cloud comprises the following steps:
extracting information of a vehicle and a feed opening, and acquiring a rotation matrix;
extracting the point cloud of bulk materials and judging whether the bulk materials exceed the limit;
Wherein:
The method for extracting the information of the vehicle and the feed opening and obtaining the rotation matrix comprises the following steps:
s1, acquiring point cloud data PCD comprising a vehicle and a feed opening by using a laser radar;
s2, removing the miscellaneous points in the environment through direct filtering;
S3, dividing the point cloud data into an auxiliary Ping Miandian cloud, a carriage point cloud and a feed opening point cloud through an European clustering algorithm;
s4, performing plane fitting on the auxiliary Ping Miandian cloud;
s5, extracting Ping Miandian cloud at the bottom of the carriage through direct filtering and fitting a plane;
s6, checking the position and the posture of the point cloud after the posture correction in a visual window, and judging whether the information extraction is facilitated;
the method for extracting the point cloud of the bulk material and judging whether the point cloud of the bulk material exceeds the limit comprises the following steps:
A1. Inputting a bulk material height limit value h, and acquiring cloud data of detection points with bulk materials in a carriage;
A2. Performing pose transformation on the cloud data of the detection points;
A3. extracting a bulk material point cloud from the detection point cloud data through direct filtering, and removing floating miscellaneous points in the bulk material point cloud by using European cluster segmentation;
A4. And judging whether the maximum value of the bulk material point cloud in the vertical direction is smaller than the maximum value of the vehicle compartment in the vertical direction minus the bulk material height limit value.
The purpose of the design is to provide a carriage bulk cargo overrun detection method based on laser radar point cloud, after a vehicle and a feed opening reach a specified position, information of the vehicle carriage and the feed opening is acquired before bulk cargo is loaded, the method is used for the subsequent processing process of the bulk cargo point cloud, the problem of carriage information deletion caused by the shielding of the carriage by bulk cargo in the loading process is avoided, and the influence of the feed opening on bulk cargo information extraction is avoided. When the material is not filled, the y-axis rotation matrix, the x-axis rotation matrix, the material outlet point cloud, the maximum and minimum values of the vehicle carriage in the x-axis, and the maximum and minimum values of the vehicle carriage in the y-axis are extracted and recorded, and the information is used in each subsequent frame of point cloud, so that the subsequent point cloud processing speed can be improved, the real-time detection of the bulk material loading capacity is facilitated, and the delay and the hysteresis of data are reduced.
Example 2
A carriage bulk cargo overrun detection method based on laser radar point cloud comprises the following steps:
extracting information of a vehicle and a feed opening, and acquiring a rotation matrix;
extracting the point cloud of bulk materials and judging whether the bulk materials exceed the limit;
Wherein:
The method for extracting the information of the vehicle and the feed opening and obtaining the rotation matrix comprises the following steps:
s1, acquiring point cloud data PCD comprising a vehicle and a feed opening by using a laser radar;
s2, removing the miscellaneous points in the environment through direct filtering;
S3, dividing the point cloud data into an auxiliary Ping Miandian cloud, a carriage point cloud and a feed opening point cloud through an European clustering algorithm;
s4, performing plane fitting on the auxiliary Ping Miandian cloud;
s5, extracting Ping Miandian cloud at the bottom of the carriage through direct filtering and fitting a plane;
s6, checking the position and the posture of the point cloud after the posture correction in a visual window, and judging whether the information extraction is facilitated;
the method for extracting the point cloud of the bulk material and judging whether the point cloud of the bulk material exceeds the limit comprises the following steps:
A1. Inputting a bulk material height limit value h, and acquiring cloud data of detection points with bulk materials in a carriage;
A2. Performing pose transformation on the cloud data of the detection points;
A3. extracting a bulk material point cloud from the detection point cloud data through direct filtering, and removing floating miscellaneous points in the bulk material point cloud by using European cluster segmentation;
A4. And judging whether the maximum value of the bulk material point cloud in the vertical direction is smaller than the maximum value of the vehicle compartment in the vertical direction minus the bulk material height limit value.
As shown in fig. 1 and 3, in S1, whether the vehicle and the blanking port in the point cloud data reach the designated position is checked through the visualization window, and if the designated position is not reached, the vehicle or the blanking port needs to be moved until the designated position is reached.
In S2, a three-dimensional filtering direction vector is set as (a, B, C), and two planes parallel to each other and not coincident with each other are established perpendicular to the three-dimensional filtering direction vector:
Ax+By+Cz+D1=0；
Ax+By+Cz+D2=0；
Meanwhile, large blocks of miscellaneous points are removed based on the following formula:
；
；
wherein x, y and z are three axes of a Cartesian three-dimensional rectangular coordinate system respectively, PCD is point cloud data which are acquired by using a laser radar and comprise a vehicle and a feed opening, A, B, C is components of three-dimensional filtering direction vectors in the x, y and z directions respectively, and D1 and D2 are constant numbers of two planes respectively.
In S3, in the euro-type clustering algorithm, a distance threshold is set, the distance between the points is calculated, if the distance is smaller than the distance threshold, the two points are divided into the same point set, otherwise, the two points are divided into different point sets; and simultaneously setting a minimum clustering point number C min and removing point sets with the point number smaller than C min.
In S4, in the step S4 of performing plane fitting on the auxiliary Ping Miandian cloud, firstly calculating an included angle α and a rotation axis vector K1 between a normal vector N1 and a y axis of a fitting plane, then calculating a rotation matrix R1 through the included angle α and the rotation axis vector K1, and finally performing pose transformation on all the divided point clouds by taking the rotation matrix R1 as the y axis rotation matrix, wherein the calculation formula of the included angle α is as follows:
；
wherein alpha is the included angle between the normal vector N1 of the fitting plane and the y axis, The components of the normal vector N1 in the x, y and z directions are respectively;
The calculation formula of the rotation axis vector K1 is:
；
In the method, in the process of the invention, Is a vector of the rotation axis,The components of the normal vector N1 in the x, y and z directions are respectively;
The calculation formula of the rotation matrix R1 is:
；
Wherein R1 is a rotation matrix, E is an identity matrix, alpha is an included angle between a normal vector N1 of a fitting plane and a y axis, Is the product of the transposed vectors of vectors K1 and K1;
。
in S5, the cloud of the bottom Ping Miandian of the carriage is extracted through direct filtering and is fitted to a plane, and the included angle between the normal vector N2 of the fitted plane and the x axis is calculated And an axis of rotation vector K2, then using the included angle/>And a rotation axis vector K2 is used for calculating a rotation matrix R2, and finally the rotation matrix R2 is used as an x-axis rotation matrix to perform pose transformation on all the segmented point clouds, wherein the included angle/>The calculation formula of (2) is as follows:
；
In the method, in the process of the invention, To fit the angle between the normal vector N2 to the plane and the x-axis,、、Components of the vector N2 in the x, y and z directions respectively;
the calculation formula of the rotation axis vector K2 is:
In the method, in the process of the invention, Is a vector of the rotation axis,、、The components of the normal vector N2 in the x, y and z directions are respectively;
The calculation formula of the rotation matrix R2 is:
；
Wherein R2 is a rotation matrix, E is an identity matrix, To fit the angle between the normal vector N2 to the plane and the x-axis,Is the product of the transposed vectors of vectors K2 and K2;
。
In S6, checking whether the position and the posture of the point cloud after the pose correction are favorable for information extraction in a visual window, if not, moving a vehicle or a feed opening, re-executing the operations from S1 to S5, if the pose is favorable for information extraction, extracting and recording the maximum value and the minimum value of a y-axis rotation matrix R1, an x-axis rotation matrix R2, the feed opening point cloud, a vehicle carriage on the x-axis and the maximum value and the minimum value of the vehicle carriage on the y-axis, wherein the information extraction is facilitated by observing whether the plane point cloud on the side surface of the carriage is parallel to a yoz plane, and whether the plane point cloud on the bottom of the carriage is parallel to a xoz plane or not, and if so, the two are parallel.
As shown in fig. 2 and 3, in A2, the pose of the detection point cloud is transformed by using the y-axis rotation matrix R1 and the x-axis rotation matrix R2 to obtain the point cloudThe formula is as follows:
；
In the method, in the process of the invention, Is the detection point cloud before pose transformation,/>The method is characterized in that the method is a detection point cloud after pose transformation, R1 is a y-axis rotation matrix, and R2 is an x-axis rotation matrix;
Then carrying out KdTree radius search on the detection point cloud through the blanking port point cloud, and removing the searched point cloud from the detection point cloud, so as to avoid the influence of the blanking port on the extraction of the bulk material point cloud, wherein the removed point cloud is The expression is:
；
In the method, in the process of the invention, Is a removed point cloud,/>For the cloud of the blanking port point extracted in S3,/>Is the detection point cloud after pose transformation,/>Search radius of KdTree,/>For/>Points and/>Distance between the points in (a) andThe expression can be represented by the following formula:
；
；
In the method, in the process of the invention, For/>Is a dot in (2);
For/> Is a dot in (2);
i is point cloud Point number of (n) >, n isIs the number of points of (3);
j is the point cloud Point number m is/>Is the number of points of (3);
Respectively, are dot/> Components in the x, y, z directions;
Respectively, are dot/> Components in x, y, z directions.
In A3, floating miscellaneous points in the bulk material point cloud are removed by European cluster segmentation, wherein the minimum cluster point number C min is set, and a point set with the point number less than C min is excluded from the algorithm result.
And in A4, extracting the maximum value of the bulk material point cloud in the y axis, judging whether the maximum value of the bulk material point cloud in the y axis is smaller than the maximum value of a carriage of the vehicle in the y axis minus the bulk material height limit value h, if so, sending out an overrun warning, otherwise, continuing to feed, and detecting the cloud data of the detection point of the next frame.
It should be noted that, in a partial use scenario, the rotation matrix is also called as a correction matrix, which is the same in substance, only represents the difference between habits of the technicians in the field, and meanwhile, the technical scheme provided by the application mainly can be summarized into two large steps, one step is that after the laser radar collects the vehicle point cloud data comprising the vehicle and the feed opening, redundant massive miscellaneous points in the environment are removed by using the direct filtering, then the European cluster segmentation is used for segmenting the point cloud, the y-axis rotation matrix and the x-axis rotation matrix are obtained, pose transformation is carried out on all the segmented point clouds, and finally the information of the vehicle and the feed opening is extracted and recorded;
the other step is to control blanking, obtain detection point cloud data with bulk cargo in a carriage, respectively use a y-axis rotation matrix and an x-axis rotation matrix to perform pose transformation on the detection point cloud, remove the blanking port point cloud, finally use straight-through filtering to extract the bulk cargo point cloud from the processed detection point cloud, and judge whether the bulk cargo point cloud is out of limit.
Finally, it should be noted that: the foregoing description of the preferred embodiments of the present invention is not intended to be limiting, but rather, it will be apparent to those skilled in the art that the foregoing description of the preferred embodiments of the present invention can be modified or equivalents can be substituted for some of the features thereof, and any modification, equivalent substitution, improvement or the like that is within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A carriage bulk cargo overrun detection method based on laser radar point cloud is characterized by comprising the following steps:
extracting information of a vehicle and a feed opening, and acquiring a rotation matrix;
extracting the point cloud of bulk materials and judging whether the bulk materials exceed the limit;
Wherein:
The method for extracting the information of the vehicle and the feed opening and obtaining the rotation matrix comprises the following steps:
s1, acquiring point cloud data PCD comprising a vehicle and a feed opening by using a laser radar;
s2, removing the miscellaneous points in the environment through direct filtering;
S3, dividing the point cloud data into an auxiliary Ping Miandian cloud, a carriage point cloud and a feed opening point cloud through an European clustering algorithm;
S4, carrying out plane fitting on auxiliary Ping Miandian clouds, wherein in plane fitting on auxiliary Ping Miandian clouds, calculating an included angle alpha and a rotation axis vector K1 between a normal vector N1 and a y axis of a fitting plane, calculating a rotation matrix R1 through the included angle alpha and the rotation axis vector K1, and finally carrying out pose transformation on all divided point clouds by taking the rotation matrix R1 as the y axis rotation matrix, wherein the calculation formula of the included angle alpha is as follows:
wherein alpha is the included angle between the normal vector N1 of the fitting plane and the y axis, The components of the normal vector N1 in the x, y and z directions are respectively;
The calculation formula of the rotation axis vector K1 is:
In the method, in the process of the invention, Is the rotation axis vector,/>The components of the normal vector N1 in the x, y and z directions are respectively;
The calculation formula of the rotation matrix R1 is:
wherein R1 is a y-axis rotation matrix, E is an identity matrix, alpha is an included angle between a normal vector N1 of a fitting plane and a y-axis, Is the product of the transposed vectors of vectors K1 and K1;
S5, extracting a carriage bottom Ping Miandian cloud through direct filtering and fitting a plane, wherein the direct filtering extracts a carriage bottom Ping Miandian cloud and fits the plane, calculating an included angle beta and a rotation axis vector K2 between a normal vector N2 and an x axis of the fitting plane, then calculating a rotation matrix R2 by using the included angle beta and the rotation axis vector K2, and finally carrying out pose transformation on all the separated point clouds by taking the rotation matrix R2 as the x axis rotation matrix, wherein the calculation formula of the included angle beta is as follows:
wherein beta is the included angle between the normal vector N2 of the fitting plane and the x axis, Components of the vector N2 in the x, y and z directions respectively;
the calculation formula of the rotation axis vector K2 is:
In the method, in the process of the invention, Is the rotation axis vector,/>The components of the normal vector N2 in the x, y and z directions are respectively;
The calculation formula of the rotation matrix R2 is:
Wherein R2 is an x-axis rotation matrix, E is an identity matrix, beta is an included angle between a normal vector N2 of a fitting plane and an x-axis, Is the product of the transposed vectors of vectors K2 and K2;
s6, checking the position and the posture of the point cloud after the posture correction in a visual window, and judging whether the information extraction is facilitated;
the method for extracting the point cloud of the bulk material and judging whether the point cloud of the bulk material exceeds the limit comprises the following steps:
A1. Inputting a bulk material height limit value h, and acquiring cloud data of detection points with bulk materials in a carriage;
A2. Performing pose transformation on the cloud data of the detection points;
A3. extracting a bulk material point cloud from the detection point cloud data through direct filtering, and removing floating miscellaneous points in the bulk material point cloud by using European cluster segmentation;
A4. And judging whether the maximum value of the bulk material point cloud in the vertical direction is smaller than the maximum value of the vehicle compartment in the vertical direction minus the bulk material height limit value.
2. The method for detecting the overrun of bulk cargo in a carriage based on the laser radar point cloud according to claim 1, wherein in S1, whether the vehicle and the blanking port in the point cloud data reach the designated position or not is checked through a visual window, and if the designated position is not reached, the vehicle or the blanking port is required to be moved until the designated position is reached.
3. The method for detecting the overrun of bulk cargo in a car based on the laser radar point cloud as claimed in claim 2, wherein in S2, three-dimensional filtering direction vectors are set as (a, B, C), and two planes which are parallel to each other and do not coincide are established perpendicular to the three-dimensional filtering direction vectors:
Ax+By+Cz+D1＝0；
Ax+By+Cz+D2＝0；
Meanwhile, large blocks of miscellaneous points are removed based on the following formula:
{(x，y，z)∈PCD|Ax+By+Cz＜D1}∩{(x，y，z)∈PCD|Ax+By+Cz＞D2}；
(D1＞D2)；
Wherein x, y and z are three axes of a Cartesian three-dimensional rectangular coordinate system respectively, PCD is point cloud data which are acquired by using a laser radar and comprise a vehicle and a feed opening, A, B, C is components of three-dimensional filtering direction vectors in the x, y and z directions respectively, and D1 and D2 are constant numbers of two planes respectively.
4. The method for detecting the overrun of bulk cargo in a carriage based on the point cloud of the laser radar according to claim 3, wherein in the step S3, a distance threshold is set in the European clustering algorithm, the distance between the points is calculated, if the distance is smaller than the distance threshold, the two points are divided into the same point set, otherwise, the two points are divided into different point sets; and simultaneously setting a minimum clustering point number C min and removing point sets with the point number smaller than C min.
5. The method for detecting the overrun of bulk cargo in a carriage based on the laser radar point cloud according to claim 4, wherein in S6, whether the position and the posture of the point cloud after the pose correction are favorable for information extraction is checked in a visual window, if the position and the posture are unfavorable for information extraction, the operations of S1 to S5 are required to be carried out again by moving a vehicle or a feed opening, if the pose is favorable for information extraction, a y-axis rotation matrix R1, an x-axis rotation matrix R2, a feed opening point cloud, maximum and minimum values of the carriage of the vehicle in the x-axis and maximum and minimum values of the carriage of the vehicle in the y-axis are extracted and recorded, wherein by observing whether the plane point cloud of the side surface of the carriage is parallel to a yoz plane, and if the plane point cloud of the bottom of the carriage is parallel to a xoz plane, the information extraction is favorable.
6. The method for detecting the overrun of the bulk cargo in the carriage based on the laser radar point cloud according to claim 5, wherein in A2, the pose of the detection point cloud is transformed by using a y-axis rotation matrix R1 and an x-axis rotation matrix R2 to obtain a point cloud v r, and the formula is as follows:
vr＝R2·R1·v；
Wherein v is a detection point cloud before pose transformation, v r is a detection point cloud after pose transformation, R1 is a y-axis rotation matrix, and R2 is an x-axis rotation matrix;
Then carrying out KdTree radius search on the detection point cloud through the blanking port point cloud, and removing the searched point cloud in the detection point cloud, wherein the removed point cloud is B (v 0, d), and the expression is:
B(v0,d)＝{vr|ρ(vr,v0)＜d}；
Wherein B (v 0, d) is the removed point cloud, v 0 is the extracted feed opening point cloud in S3, v r is the pose-converted detection point cloud, d is the search radius of KdTree, ρ (v r,v0) is the distance between the point in v r and the point in v 0, and ρ (v r,v0) can be expressed by the following formula:
In the method, in the process of the invention,
V r [ i ] is the point in v r;
v 0 [ j ] is the point in v 0;
i is the point number of the point cloud v r, and n is the point number of v r;
j is the point number of the point cloud v 0, and m is the point number of v 0;
v r[i]x、vr[i]y、vr[i]z is the component of point v r [ i ] in the x, y, z directions, respectively;
v 0[j]x、v0[j]y、v0[j]z is the component of point v 0 [ j ] in the x, y, z directions, respectively.
7. The method for detecting the overrun of bulk cargo in a carriage based on the laser radar point cloud as claimed in claim 6, wherein in A3, floating miscellaneous points in the bulk cargo point cloud are removed by European clustering segmentation, wherein the minimum clustering point number C min is set, and point sets with the point number less than C min are excluded from algorithm results.
8. The method for detecting the overrun of the bulk cargo in the carriage based on the laser radar point cloud according to claim 7, wherein in A4, the maximum value of the bulk cargo point cloud in the y axis is extracted, whether the maximum value of the bulk cargo point cloud in the y axis is smaller than the maximum value of the bulk cargo in the carriage of the vehicle in the y axis minus a bulk cargo height limit value h is judged, if the maximum value of the bulk cargo point cloud is smaller than the maximum value of the bulk cargo height limit value in the carriage of the vehicle in the y axis, the overrun warning is sent out, otherwise, the blanking is continued, and the cloud data of the detection point of the next frame is detected.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410049227.9A CN117553686B (en) | 2024-01-12 | 2024-01-12 | Laser radar point cloud-based carriage bulk cargo overrun detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410049227.9A CN117553686B (en) | 2024-01-12 | 2024-01-12 | Laser radar point cloud-based carriage bulk cargo overrun detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117553686A CN117553686A (en) | 2024-02-13 |
| CN117553686B true CN117553686B (en) | 2024-05-07 |
Family
ID=89821953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410049227.9A Active CN117553686B (en) | 2024-01-12 | 2024-01-12 | Laser radar point cloud-based carriage bulk cargo overrun detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117553686B (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE1506092A1 (en) * | 1965-02-10 | 1970-07-16 | Thomson Inf Et Visualisation | Arrangement for simulating the response signals from radar targets |
| WO2015135015A1 (en) * | 2014-03-11 | 2015-09-17 | 3D Image Automation Pty Ltd | Train wagon 3d profiler |
| EP3094807A1 (en) * | 2014-01-14 | 2016-11-23 | Sandvik Mining and Construction Oy | Mine control system |
| CN110358895A (en) * | 2019-07-16 | 2019-10-22 | 江阴市恒润环锻有限公司 | A kind of superhigh pressure turbine alloy ring forging and its manufacturing method |
| CN110781827A (en) * | 2019-10-25 | 2020-02-11 | 中山大学 | Road edge detection system and method based on laser radar and fan-shaped space division |
| CN112299048A (en) * | 2019-07-30 | 2021-02-02 | 上海宝信软件股份有限公司 | Train loading and distributing system and method based on unmanned grab bucket running mode |
| SE1951064A1 (en) * | 2019-09-20 | 2021-03-21 | Metso Sweden Ab | Mining Equipment Inspection System, Mining Equipment Inspection Method, and Mining Equipment Inspection Device |
| CN112735135A (en) * | 2020-12-31 | 2021-04-30 | 东来智慧交通科技(深圳)有限公司 | High-speed moving vehicle overrun detection method |
| CN113625299A (en) * | 2021-07-26 | 2021-11-09 | 北京理工大学 | Three-dimensional laser radar-based method and device for detecting height and unbalance loading of loading material |
| CN114612665A (en) * | 2022-03-15 | 2022-06-10 | 北京航空航天大学 | Pose estimation and dynamic vehicle detection method based on normal vector histogram features |
| CN114782700A (en) * | 2022-03-03 | 2022-07-22 | 中冶赛迪重庆信息技术有限公司 | Ship type feature acquisition method and system, electronic device and readable storage medium |
| CN116424899A (en) * | 2023-04-17 | 2023-07-14 | 天津港远航国际矿石码头有限公司 | Four-point loading device and method for bulk materials of railway open wagon and storage medium |
| CN116424900A (en) * | 2023-05-31 | 2023-07-14 | 中煤科工智能储装技术有限公司 | Full-volume unmanned loading method based on laser radar point cloud identification |
| CN116630411A (en) * | 2023-07-26 | 2023-08-22 | 湖南睿图智能科技有限公司 | Mining electric shovel material surface identification method, device and system based on fusion perception |
| CN116645348A (en) * | 2023-05-29 | 2023-08-25 | 南京林业大学 | High-speed loop linear fitting method based on point cloud random sampling |
| CN116934832A (en) * | 2023-07-19 | 2023-10-24 | 齐鲁工业大学(山东省科学院) | Real-time calibration method of three-dimensional point cloud based on field reference object |
| CN117329971A (en) * | 2023-12-01 | 2024-01-02 | 海博泰科技(青岛)有限公司 | Compartment balance detection method and system based on three-dimensional laser radar |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9305364B2 (en) * | 2013-02-19 | 2016-04-05 | Caterpillar Inc. | Motion estimation systems and methods |
-
2024
- 2024-01-12 CN CN202410049227.9A patent/CN117553686B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE1506092A1 (en) * | 1965-02-10 | 1970-07-16 | Thomson Inf Et Visualisation | Arrangement for simulating the response signals from radar targets |
| EP3094807A1 (en) * | 2014-01-14 | 2016-11-23 | Sandvik Mining and Construction Oy | Mine control system |
| WO2015135015A1 (en) * | 2014-03-11 | 2015-09-17 | 3D Image Automation Pty Ltd | Train wagon 3d profiler |
| CN110358895A (en) * | 2019-07-16 | 2019-10-22 | 江阴市恒润环锻有限公司 | A kind of superhigh pressure turbine alloy ring forging and its manufacturing method |
| CN112299048A (en) * | 2019-07-30 | 2021-02-02 | 上海宝信软件股份有限公司 | Train loading and distributing system and method based on unmanned grab bucket running mode |
| SE1951064A1 (en) * | 2019-09-20 | 2021-03-21 | Metso Sweden Ab | Mining Equipment Inspection System, Mining Equipment Inspection Method, and Mining Equipment Inspection Device |
| CN110781827A (en) * | 2019-10-25 | 2020-02-11 | 中山大学 | Road edge detection system and method based on laser radar and fan-shaped space division |
| CN112735135A (en) * | 2020-12-31 | 2021-04-30 | 东来智慧交通科技(深圳)有限公司 | High-speed moving vehicle overrun detection method |
| CN113625299A (en) * | 2021-07-26 | 2021-11-09 | 北京理工大学 | Three-dimensional laser radar-based method and device for detecting height and unbalance loading of loading material |
| CN114782700A (en) * | 2022-03-03 | 2022-07-22 | 中冶赛迪重庆信息技术有限公司 | Ship type feature acquisition method and system, electronic device and readable storage medium |
| CN114612665A (en) * | 2022-03-15 | 2022-06-10 | 北京航空航天大学 | Pose estimation and dynamic vehicle detection method based on normal vector histogram features |
| CN116424899A (en) * | 2023-04-17 | 2023-07-14 | 天津港远航国际矿石码头有限公司 | Four-point loading device and method for bulk materials of railway open wagon and storage medium |
| CN116645348A (en) * | 2023-05-29 | 2023-08-25 | 南京林业大学 | High-speed loop linear fitting method based on point cloud random sampling |
| CN116424900A (en) * | 2023-05-31 | 2023-07-14 | 中煤科工智能储装技术有限公司 | Full-volume unmanned loading method based on laser radar point cloud identification |
| CN116934832A (en) * | 2023-07-19 | 2023-10-24 | 齐鲁工业大学(山东省科学院) | Real-time calibration method of three-dimensional point cloud based on field reference object |
| CN116630411A (en) * | 2023-07-26 | 2023-08-22 | 湖南睿图智能科技有限公司 | Mining electric shovel material surface identification method, device and system based on fusion perception |
| CN117329971A (en) * | 2023-12-01 | 2024-01-02 | 海博泰科技(青岛)有限公司 | Compartment balance detection method and system based on three-dimensional laser radar |
Non-Patent Citations (3)
| Title |
|---|
| 提升装车系统自动化程度实现精美定量装车;刘海锋;;神华科技;20180126(第01期);全文 * |
| 火电机组抓斗卸船机全自动系统开发及应用;余俐敏;孙浩童;蔡国强;黄益民;葛镇江;;能源研究与管理;20200915(第03期);全文 * |
| 珠海横琴国际金融中心大厦项目智慧工地集成化应用;周永明;苏章;苏前广;;土木建筑工程信息技术;20180815(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117553686A (en) | 2024-02-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111680542A (en) | Steel coil point cloud identification and classification method based on multi-scale feature extraction and Pointernet neural network | |
| CN111598946B (en) | Object pose measuring method and device and storage medium | |
| EP0142990A2 (en) | Inspecting articles | |
| CN109978911B (en) | Image feature point tracking method and camera | |
| CN111496789B (en) | Offline complex curved surface spraying track planning system and control method | |
| CN108663026B (en) | Vibration measuring method | |
| CN110866894B (en) | Cross-granularity sheet metal part identification system and method based on machine vision technology | |
| CN108009494A (en) | A kind of intersection wireless vehicle tracking based on unmanned plane | |
| CN113034600A (en) | Non-texture planar structure industrial part identification and 6D pose estimation method based on template matching | |
| CN109146817B (en) | Noise processing method for non-iterative single object scattered point cloud data | |
| CN111815706A (en) | Visual identification method, device, equipment and medium for single-article unstacking | |
| CN110838145B (en) | Visual positioning and mapping method for indoor dynamic scene | |
| CN109708627B (en) | Method for rapidly detecting space dynamic point target under moving platform | |
| CN109409182B (en) | Embryo automatic identification method based on image processing | |
| CN117553686B (en) | Laser radar point cloud-based carriage bulk cargo overrun detection method | |
| CN114627151A (en) | Method for tracking motion state of aerial target by unmanned aerial vehicle cluster | |
| CN110889362B (en) | Obstacle detection method using grid map height information | |
| CN116309371A (en) | Train part abnormality detection method and device, electronic equipment and medium | |
| CN113963129A (en) | Point cloud-based ship small component template matching and online identification method | |
| CN114529559A (en) | Ground segmentation method based on laser point cloud | |
| CN111445401B (en) | Visual identification method, device, equipment and medium for out-of-order sorting of cylindrical bars | |
| CN114137984A (en) | Modular transmission platform and control method and path planning method thereof | |
| CN117745719B (en) | Extraction method of robot weld milling track | |
| Li et al. | Research on Key Technologies for Container Ship Loading Test Based on 3D Laser Scanning | |
| CN113688706B (en) | Vehicle detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |