CN117727003A - Distribution detection method, device and equipment for irregular objects and storage medium - Google Patents

Abstract

The invention relates to the technical field of target detection, and discloses a distribution detection method, device and equipment for irregular objects and a storage medium, wherein the method comprises the following steps: acquiring point cloud data of a working scene, and performing ground segmentation on the point cloud data to obtain a ground point cloud and a non-ground point cloud; calculating a ground height value based on the ground point cloud; filtering the non-ground point cloud to obtain a target point cloud representing an irregular object; constructing a ground grid graph of the operation scene, and projecting a target point cloud into the ground grid graph; and determining the height distribution information of the irregular object on the ground of the operation scene by using the point cloud height data and the ground height value corresponding to each grid in the ground grid chart. The invention solves the problem of inaccurate detection of the distribution information of the irregular object.

Description

Distribution detection method, device and equipment for irregular objects and storage medium

Technical Field

The present invention relates to the field of target detection technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting distribution of an irregular object.

Background

In the field of passenger cars, unmanned technology has been developed at a high speed and has received more and more attention, but in the unmanned field of engineering machine tool, face the operational scenario such as dregs transportation, harbour transportation, construction, mining, rescue relief work, etc., the landing of engineering machine tool unmanned technology is in bad working condition, complicated changeable, road unstructured, special operational scenario has problems such as all-weather operation requirement, especially, to irregular operation object (for example dregs, heap ore), need carry out shovel loading and transport task, thereby carry out accurate target detection and discernment to irregular operation object and be the vital work, but current technical means mainly concentrate on the external profile detection of irregular object, for example, chinese patent CN115407322a and CN115407323a, can detect the profile boundary that the heap is projected on ground respectively, but when actual engineering project falls to the ground, not only need general target detection method output information such as target size, position and confidence, still need the spatial distribution information of target (distribution information refers to the irregular target of height, need high-low profile information to be required when the target is projected to the two-dimensional plane of ground, in this way, the low profile information is required to carry out the accurate profile information of the sediment heap ore, can be carried out, nevertheless, can be carried out to the accurate and the concrete pile, the following profile can be accurately and the concrete pile is not carried out, the following profile, the present has been described, can be carried out, the following profile information is planned and has been described, and has been completely, and has been difficult to provide to control to the following profile information.

Disclosure of Invention

In view of the above, the present invention provides a method, apparatus, device and storage medium for detecting distribution of irregular objects, so as to solve the problem of inaccurate detection of distribution information of irregular objects.

In a first aspect, the present invention provides a method for detecting distribution of irregular objects, where the method includes: acquiring point cloud data of a working scene, and performing ground segmentation on the point cloud data to obtain a ground point cloud and a non-ground point cloud; calculating a ground height value based on the ground point cloud; filtering the non-ground point cloud to obtain a target point cloud representing an irregular object; constructing a ground grid graph of the operation scene, and projecting a target point cloud into the ground grid graph; and determining the height distribution information of the irregular object on the ground of the operation scene by using the point cloud height data and the ground height value corresponding to each grid in the ground grid chart.

In an alternative embodiment, filtering the non-ground point cloud to obtain a target point cloud representing the irregular object includes: performing self-vehicle point cloud filtering, static obstacle filtering and dynamic obstacle filtering on the non-ground point cloud to obtain a residual point cloud; and filtering the space impurities of the residual point cloud to obtain a target point cloud.

In an alternative embodiment, the self-propelled point cloud filtering, static obstacle filtering and dynamic obstacle filtering are performed on the non-ground point cloud, and the method comprises the following steps: acquiring a vehicle body contour range; removing point cloud data in a vehicle body contour range from the non-ground point cloud; acquiring static obstacle region position information and vehicle positioning information preset in a working scene; determining a static obstacle point cloud from the non-ground point cloud according to the relative position relation between the vehicle positioning information and the static obstacle region position information, and removing the static obstacle point cloud from the non-ground point cloud; acquiring a dynamic target detection result of a working scene; and removing the dynamic obstacle point cloud covered by the dynamic target detection result from the non-ground point cloud.

In an alternative embodiment, spatial impurity filtering is performed on the remaining point cloud, including: semantic segmentation is carried out on each angle image of the operation scene, and external contours of irregular objects in each angle image are obtained; projecting the residual point clouds into each angle image of the operation scene respectively; comparing the external contour in each angle image with the corresponding point cloud projection to determine non-target point cloud projections outside the external contour; reversely mapping the non-target point cloud projections in the images of each angle to the residual point cloud, and determining the non-target point cloud in the residual point cloud; and eliminating non-target point clouds representing spatial impurities from the residual point clouds.

In an alternative embodiment, after spatial impurity filtering the remaining point cloud, the method further comprises: clustering the second residual point cloud after the spatial impurity filtering to obtain a plurality of clustering targets; and removing the abnormal targets from the clustered targets according to preset abnormal filtering indexes, wherein the abnormal filtering indexes at least comprise one of the limit of the number of point clouds in the clustered targets, the limit of projection areas of the clustered targets in multiple directions and the limit of the side lengths of the circumscribed rectangles of the clustered targets.

In an alternative embodiment, after removing the outlier target from the clustered targets, the method further comprises: tracking each residual clustering target in a preset time; respectively obtaining the maximum historical continuous frame number of each residual clustering target in preset time, wherein the maximum historical continuous frame number represents the maximum time frame number of the residual clustering targets which are continuously and successfully tracked in the preset time; and extracting reliable targets from the residual clustering targets, wherein the reliable targets are residual clustering targets with the maximum historical continuous frame number larger than a preset frame number threshold value.

In an alternative embodiment, determining height distribution information of an irregular object on the ground of a work scene by using point cloud height data and ground height values corresponding to each grid in a ground grid map includes: acquiring a point cloud height maximum value corresponding to each grid in a ground grid chart, and calculating a difference value between each point cloud height maximum value and the ground height value; and filling each calculated difference value into a corresponding grid to obtain the height distribution information of the irregular object.

In a second aspect, the present invention provides a distribution detecting apparatus for irregular objects, the apparatus comprising: the point cloud data acquisition module is used for acquiring point cloud data of a working scene, and carrying out ground segmentation on the point cloud data to obtain ground point cloud and non-ground point cloud; the ground height determining module is used for calculating a ground height value based on the ground point cloud; the irregular target identification module is used for filtering the non-ground point cloud to obtain a target point cloud representing an irregular object; the point cloud projection module is used for constructing a ground grid graph of the operation scene and projecting a target point cloud into the ground grid graph; and the distribution information calculation module is used for determining the height distribution information of the irregular object on the ground of the operation scene by utilizing the point cloud height data and the ground height value corresponding to each grid in the ground grid chart.

In a third aspect, the present invention provides a computer device comprising: the memory and the processor are in communication connection, the memory stores computer instructions, and the processor executes the computer instructions to perform the method of the first aspect or any implementation manner corresponding to the first aspect.

In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of the first aspect or any of its corresponding embodiments.

The technical scheme provided by the invention has the following advantages:

according to the embodiment of the invention, the ground segmentation is carried out on the point cloud data of the operation scene to obtain the ground point cloud and the non-ground point cloud, and then the ground height value is calculated based on the ground point cloud; meanwhile, non-ground point clouds are filtered, only target point clouds capable of representing irregular objects are reserved, and other interference point clouds are all removed; further constructing a ground grid graph of the operation scene and projecting a target point cloud into the ground grid graph; therefore, only the position of the target point cloud exists in the ground grid diagram, and finally, the height distribution information of the irregular object on the ground of the operation scene can be accurately calculated by utilizing the point cloud height data and the ground height value corresponding to each grid in the ground grid diagram. And reliable data support is provided for decisions such as path planning, obstacle avoidance, shovel loading and the like of engineering machinery.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for detecting distribution of irregular objects according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of distribution information of irregular objects according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of the effect of filtering non-target point clouds according to an embodiment of the invention;

FIG. 4 is another flow chart of a method for detecting distribution of irregular objects according to an embodiment of the present invention;

fig. 5 is a schematic structural view of a distribution detecting apparatus for irregular objects according to an embodiment of the present invention;

fig. 6 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In the unmanned field of engineering machinery, the unmanned engineering machinery is faced with the operation scenes such as muck transportation, port transportation, building construction, mining, rescue and relief work, and the like, the unmanned engineering machinery has the problems of severe landing working conditions, complexity and changeability, unstructured roads, all-weather operation requirements in special operation scenes, and the like, and especially the unmanned engineering machinery is required to carry out shoveling and transportation tasks for irregular operation objects such as muck, piled ore, and the like. For irregular operation objects such as slag and the like, it is important to describe the distribution information of the height fluctuation of the irregular operation objects when the targets are projected to a two-dimensional plane of the ground, so that data support can be provided for subsequent planning, obstacle avoidance, shoveling and other decisions, for example, the distribution information can determine which side of the slag is shoveled by engineering machinery, and the shoveling efficiency is highest and the safety is highest. However, the related technology mainly focuses on the external contour detection of the irregular object, only the contour boundary of the pile projected on the ground can be detected, and the distribution information of the irregular operation object can not be accurately identified. For example, the related technology projects point cloud data corresponding to a scene to be detected into a plurality of two-dimensional grids on the ground to obtain a first height value set of all point clouds in each two-dimensional grid; then, for each two-dimensional grid, determining the height range of all points in the two-dimensional grid according to the values in the first height value set; then, estimating the point cloud height range of irregular objects such as slag and the like, and identifying a height sub-range meeting the preset material pile height distribution condition in the estimated height range for each two-dimensional grid; the point cloud data in the height sub-range is then used as the point cloud data of a stockpile such as slag, so that the outline boundary of the stockpile projected onto the ground can be determined according to the specific two-dimensional grid position occupied when the data are projected onto the ground. However, the method can only rapidly estimate the occupied position of the irregular object on the ground, and cannot accurately calculate different heights corresponding to the different positions of the irregular object on the ground, and the heights of the irregular object are completely estimated by experience such as manpower, machinery and the like, so that accurate distribution information of the irregular object is difficult to obtain.

According to an embodiment of the present invention, there is provided an embodiment of a distribution detection method of irregular objects, it should be noted that the steps shown in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order different from that herein.

In this embodiment, a method for detecting distribution of irregular objects is provided, which may be used in the computer device described above, and fig. 1 is a flowchart of a method for detecting distribution of irregular objects according to an embodiment of the present invention, where the flowchart includes the following steps:

step S101, acquiring point cloud data of a working scene, and performing ground segmentation on the point cloud data to obtain a ground point cloud and a non-ground point cloud.

Step S102, calculating a ground height value based on the ground point cloud.

Specifically, the embodiment of the invention firstly acquires the point cloud data of the operation scene, performs voxel filtering on the input point cloud data, and can effectively remove noise in the point cloud data, smooth the point cloud, reduce the data quantity and extract the characteristics by dividing the point cloud data into a plurality of voxel grids and reserving only one representative point in each grid. Then, the point cloud data is subjected to ground segmentation to obtain a ground point cloud and a non-ground point cloud, and the adopted segmentation methods include, but are not limited to, a planar grid method, a point cloud normal vector method, a model fitting method and a surface element grid method, so that the specific process of ground segmentation is not repeated in this embodiment. And calculating the average height of all the point clouds by using the segmented ground point clouds to obtain a ground height value, and providing preparation data for measuring different height distribution of the irregular object at different positions in the subsequent step.

Step S103, filtering the non-ground point cloud to obtain a target point cloud representing the irregular object.

Specifically, the embodiment of the invention filters non-ground point clouds, and generally, the non-ground point clouds include irregular objects such as slag and other non-operation barriers such as pedestrians, vehicles, buildings, trees and the like, and the embodiment associates the interference point clouds corresponding to the non-operation barriers in the non-ground point clouds by combining an image target segmentation result of an operation scene, so that the interference point clouds are filtered out in the non-ground point clouds, and only target point clouds which can represent the irregular objects waiting for engineering machinery operation such as slag are reserved.

Step S104, constructing a ground grid graph of the operation scene, and projecting a target point cloud into the ground grid graph.

Step S105, determining the height distribution information of the irregular object on the ground of the operation scene by using the point cloud height data and the ground height value corresponding to each grid in the ground grid chart.

Specifically, as shown in fig. 2, in the embodiment of the present invention, by constructing a ground grid map of a working scene and projecting a target point cloud into the ground grid map, only the position where the target point cloud is located in the ground grid map has the height information of the point cloud, and finally, the height distribution information of an irregular object on the ground of the working scene is accurately calculated by using the point cloud height data and the ground height value corresponding to each grid in the ground grid map, for example, the average point cloud height and the ground height value corresponding to each grid are used as a difference value, so as to determine the stacker height value corresponding to each grid, and provide reliable data support for path planning, obstacle avoidance, spading and other decisions of engineering machinery.

In some alternative embodiments, the step S103 includes:

step a1, performing self-vehicle point cloud filtering, static obstacle filtering and dynamic obstacle filtering on non-ground point clouds to obtain residual point clouds;

and a2, filtering space impurities of the residual point cloud to obtain a target point cloud.

Specifically, the embodiment of the invention mainly performs self-vehicle point cloud filtering, static obstacle filtering, dynamic obstacle filtering and space impurity filtering on non-ground point cloud, wherein the self-vehicle point cloud filtering is to consider that a laser radar is usually arranged on a vehicle roof, so that a part of parts of a self-vehicle can be scanned when the vehicle scans point cloud data of a working scene, a part of the point cloud data represents a self-vehicle body, and the accuracy of the target point cloud can be improved by filtering the part of self-vehicle point cloud. Static obstacle filtering and dynamic obstacle filtering refer to filtering out point clouds scanned by a laser radar by static obstacles such as trees, buildings, signboards and the like in an operation scene and point clouds scanned by the laser radar and identified by dynamic obstacles such as pedestrians, animals, vehicles and the like. For the self-vehicle point cloud filtering, static obstacle filtering and dynamic obstacle filtering, the embodiment of the invention can perform target detection processing on the camera image, the point cloud data and/or the point cloud image through a machine learning algorithm, so that the specific positions of the self-vehicle, the static obstacle and the dynamic obstacle in the non-ground point cloud are obtained, and then the point cloud at the corresponding positions is removed, so that the self-vehicle point cloud filtering, the static obstacle filtering and the dynamic obstacle filtering are realized.

In addition, in the embodiment of the invention, the condition that false detection is easy to occur in extreme weather scenes such as sand dust, heavy rain and the like is also considered, so that a plurality of point clouds in the point cloud data are substances suspended in the air such as dust and rainwater, and the embodiment of the invention also performs space impurity filtering on the residual point clouds filtered in the previous step, thereby further ensuring that the obtained target point clouds can accurately represent irregular objects such as slag and the like, and improving the accuracy of the follow-up calculation distribution information. Specifically, for the step of filtering the spatial impurities, the embodiment may first draw a 3D contour of the remaining point cloud, and then reduce the 3D contour by a certain length based on the 3D contour by using a preset contour threshold, thereby filtering impurities such as dust and rainwater near the 3D contour, and improving the accuracy of the target point cloud.

In some alternative embodiments, step a1 above includes:

and a step a11, acquiring a vehicle body contour range.

And a12, eliminating point cloud data in the vehicle body contour range from the non-ground point cloud.

Specifically, according to the embodiment, aiming at the vehicle types of various engineering machines, the length, the width and the height of the vehicle are known, the positions of the bucket, the digging arm or the digging bucket of the engineering machine are known, and even if the angle of the bucket or the digging arm changes, the position of the bucket or the digging arm relative to the vehicle body can be obtained in advance through additionally installing an angle sensor, so that the vehicle body contour range which can be scanned by the laser radar according to the vehicle body structure data is obtained in advance, and further, the point cloud about the vehicle in the specified range in the point cloud data is filtered out through setting the vehicle body range, and the efficiency and the accuracy of filtering the point cloud of the vehicle are improved.

Step a13, acquiring static obstacle area position information and vehicle positioning information preset in a working scene.

And a step a14, determining a static obstacle point cloud from the non-ground point cloud according to the relative position relation between the vehicle positioning information and the static obstacle region position information, and eliminating the static obstacle point cloud from the non-ground point cloud.

Specifically, for filtering static obstacles, the embodiment of the invention obtains the static obstacle region position information of the static obstacles in the operation scene through a global positioning module such as a vector map, for example, the global coordinates of static objects such as buildings, signboards and the like in the operation scene. And then acquiring the self-vehicle positioning information of the self-vehicle in the operation scene by using positioning technologies such as GPS, beidou positioning and the like, namely the coordinates of the self-vehicle in the operation scene. According to the relative position relation between the positioning information of the vehicle and the position information of the static obstacle region, the angle, the distance and the range of each static obstacle relative to the vehicle under the coordinate system of the vehicle body can be identified, and then the point cloud in the corresponding angle, the distance and the range is directly removed from the non-ground point cloud as the static obstacle point cloud, so that the static obstacle point cloud filtering scheme with high efficiency and high accuracy is realized.

Step a15, acquiring a dynamic target detection result of a job scene;

and a step a16 of removing the dynamic obstacle point cloud covered by the dynamic target detection result from the non-ground point cloud.

Specifically, for filtering a dynamic obstacle, the target detection module of the vehicle system provided by the embodiment of the invention obtains the position, the size and the angle information of the dynamic obstacle through an image and a point cloud (laser radar) detection method, and obtains the detected dynamic obstacle position, so that the dynamic obstacle position can cover a part of the point cloud in a point cloud space. Specifically, the embodiment of the invention can perform reasoning detection based on a neural network algorithm on the input image and the point cloud data to obtain the information such as the position, the size, the angle and the like of the dynamic obstacle. For example: the algorithms for detecting the image 3d commonly used include DETR3D, MONOFLEX, BEV and the algorithms for detecting the point cloud target commonly used include VoxelNet, pointPillars, voteNet, and the embodiment of the present invention is not limited to this, and the specific algorithm process may refer to the related technology and will not be described herein.

In some alternative embodiments, step a2 above comprises:

step a21, carrying out semantic segmentation on each angle image of the operation scene to obtain the external contour of the irregular object in each angle image;

step a22, projecting the residual point clouds into each angle image of the operation scene respectively;

step a23, comparing the external contour in each angle image with the corresponding point cloud projection to determine non-target point cloud projections outside the external contour;

step a24, the non-target point cloud projections in the images of all angles are reversely mapped into the residual point cloud, and the non-target point cloud in the residual point cloud is determined;

and a step a25 of eliminating non-target point clouds representing spatial impurities from the residual point clouds.

Specifically, the embodiment of the invention provides a spatial impurity filtering scheme combining an image and point cloud fusion so as to further improve the accuracy of spatial impurity filtering in point cloud data. Firstly, the embodiment of the invention performs joint calibration (only one calibration when a vehicle is started) by utilizing the laser radar and the camera in advance to obtain a space conversion relation between the laser radar and the camera, namely a rotation matrix R and a translation vector T for mutually converting the coordinates of the point cloud and the coordinates of the image, so that each point cloud can be mapped into the image.

Then, the present embodiment performs image acquisition on the operation scene at a plurality of angles, so as to consider the spatial impurity distribution of 3D as much as possible from the 2D image. And then carrying out semantic segmentation on each angle image to obtain the external contour of the irregular object in each angle image (the semantic segmentation is an image-based target classification method, and classifying each pixel in the image through a deep learning semantic segmentation algorithm so as to divide the image into a plurality of areas containing different types of information, wherein the targets in the image can be marked as vehicles, pedestrians, irregular operation objects, other types and the like during data marking). Because the semantic segmentation can not segment the miscellaneous points such as dust and rainwater, the embodiment combines the result of the joint calibration to project the residual point cloud into each angle image of the operation scene, and because each angle image has drawn the outline of the irregular object, the point cloud projection projected onto the image has a part falling into the outline and a part falling outside the outline, as shown in fig. 3, the point cloud projection falling outside the outline can be regarded as the non-target point cloud projection corresponding to the space impurity point cloud such as dust and raindrop.

Because the point cloud projection outside the outline can only represent a plane in the 2D image, no depth information exists, and the 3D point cloud representing dust needs to be filtered in the rest point clouds, the embodiment also reversely maps the non-target point cloud projections in the images of all angles to the rest point clouds, and can directly reject the point clouds with all depths on the observation angles in the rest point clouds as the non-target point clouds from the observation angles corresponding to the non-target point cloud projections, thereby effectively rejecting the non-target point clouds representing space impurities in the 360-degree directions in the rest point clouds, and remarkably improving the filtering efficiency and the filtering accuracy of the non-target point clouds.

In some optional implementations, after the step a2, the technical solution provided by the embodiment of the present invention further includes:

step a3, clustering the second residual point cloud after the space impurity filtering to obtain a plurality of clustering targets;

and a4, removing the abnormal targets from the clustered targets according to a preset abnormal filtering index, wherein the abnormal filtering index at least comprises one of the limit of the number of point clouds in the clustered targets, the limit of projection areas of the clustered targets in a plurality of directions and the limit of the side length of the circumscribed rectangle of the clustered targets.

Specifically, the embodiment of the invention also performs point cloud target clustering on the second residual point cloud after the space impurity filtering, so that the point cloud of the point cloud data set is divided into different point cloud clusters according to preset clustering conditions (such as distance and density among the point clouds), and each point cloud cluster represents a clustering target. And then, filtering abnormal obstacles (filtering clusters with too large or too small point cloud number, clusters with too large or too small area, and the like, such as noise points with only a few points for clustering the targets, unfiltered clean ground points and the like) by the obtained clustering targets according to the point cloud number, the projection area, the length, width, height and other thresholds (the thresholds can comprise an upper limit threshold and/or a lower limit threshold) of each target, thereby further improving the accuracy of the target point cloud representing the irregular object and ensuring the accuracy of calculating the height distribution information of the irregular object on the ground of the operation scene in the subsequent steps.

In some optional implementations, after the step a4, the technical solution provided by the embodiment of the present invention further includes:

step a5, tracking each residual clustering target in a preset time;

step a6, respectively obtaining the maximum historical continuous frame number of each residual clustering target in the preset time, wherein the maximum historical continuous frame number represents the maximum time frame number of the residual clustering targets which are continuously and successfully tracked in the preset time;

And a step a5, extracting reliable targets from all the residual clustering targets, wherein the reliable targets are residual clustering targets with the maximum historical continuous frame number larger than a preset frame number threshold value.

Specifically, for irregular objects such as slag, although the objects are in a standing state on the ground, the scanning of the lidar is a continuous process, and a small position change may be generated for the point cloud of the lidar of each frame of the same slag, so that only one or two frames of target point clouds may occasionally appear in a certain place, and the occasionally appearing and disappearing targets are basically false detection targets caused by the error of the lidar, and theoretically, the targets for stable detection should be real slag. Based on the above, the embodiment of the invention tracks a plurality of residual clustered targets filtered in the last step, calculates similarity scores, shape scores, motion scores, overlapping degree scores and the like between a newly detected target and a historical tracked target of each frame of the laser radar through a multi-target tracking algorithm, and then performs score weighting calculation to serve as a threshold judgment standard of target association between adjacent frames. If the association of the residual clustering targets between the frames is successful, adding the newly detected target into the historical tracking target, and if the association is failed, taking the target detected by the latest frame as a new target. The historical continuous frame number of each residual clustering target is obtained through multi-target tracking, and each residual clustering target corresponds to a maximum historical continuous frame number and is used for representing the maximum time frame number of continuous successful tracking of each residual clustering target in preset time, for example, target 1 is continuously tracked for 10 frames in preset time, and target 2 is continuously tracked for 5 frames in preset time. And finally, filtering the non-continuous frame targets, namely outputting each residual clustering target when the maximum historical continuous frame number of the target is larger than a preset frame number threshold value, and considering the target as a reliable target, or else, occasionally generating a false alarm of one frame and two frames to be an unreliable target. By the technical scheme provided by the embodiment of the invention, the accuracy of the target point cloud of the irregular object is further improved, so that the accuracy of the subsequent calculation of the distribution information is improved.

In some alternative embodiments, step S105 includes:

step b1, obtaining a point cloud height maximum value corresponding to each grid in a ground grid chart, and calculating a difference value between each point cloud height maximum value and the ground height value;

and b2, filling the calculated difference values into corresponding grids to obtain the height distribution information of the irregular object.

Specifically, the filtered target point cloud is stored in a corresponding ground grid, and the ground height value calculated in step S102 is subtracted from the maximum value of the point cloud height in each grid to obtain a work object height distribution grid diagram, wherein the value of each grid represents the height of the work object, and 0 represents no work object. Therefore, by the technical scheme provided by the embodiment of the invention, the accurate detection method for the height distribution of the irregular operation object is realized.

The technical scheme provided by the embodiment of the invention has the following beneficial effects:

the technical scheme provided by the embodiment of the invention provides a distribution detection method of an irregular operation object based on point cloud, which can output information such as the size, the position and the like of the irregular object such as slag and the like and can also output the spatial distribution information of the object. The technical scheme provided by the embodiment of the invention does not limit the operation scene and the region, does not need a preset height range, is more universal and has stronger robustness. According to the technical scheme provided by the embodiment of the invention, the non-operation point cloud is filtered, and only the point cloud distribution of the specific operation object is output. The technical scheme provided by the embodiment of the invention can also process extreme weather scenes such as sand dust, heavy rain and the like, and further improves the accuracy of point cloud filtering. The technical scheme provided by the embodiment of the invention can also process occasional false detection by utilizing the target tracking technology, and further improves the accuracy of point cloud filtering.

In a specific application scenario embodiment, as shown in fig. 4, the technical scheme provided by the invention comprises the following complete steps:

1. the engineering machinery scans and shoots the operation scene of the slag through the laser radar and the camera respectively, and inputs point cloud data and image data.

2. And executing joint calibration of the laser radar and the camera, and preparing for matching of subsequent point cloud data and image data.

3. And filtering the point cloud voxels, and performing downsampling and smoothing.

4. And (3) filtering the self-vehicle point cloud, static obstacle and dynamic obstacle.

5. And projecting the residual point cloud into the image, and filtering non-target point clouds such as dust, raindrops and the like.

6. And performing point cloud clustering, and filtering abnormal targets according to the clustered targets.

7. And (3) tracking targets, and filtering false alarms of which one frame and two frames occur occasionally for clustered targets with the maximum historical continuous frame number less than or equal to a preset frame number threshold.

8. And constructing a ground grid graph of the operation scene, and projecting the target point cloud remained after all filtering steps into the ground grid graph.

9. And calculating a difference value by utilizing the maximum value of the point cloud height and the ground height corresponding to each grid in the ground grid graph, filling the calculated difference value into the corresponding grid, and determining the height distribution information of the irregular object on the ground of the operation scene to obtain the slag height distribution grid graph.

The embodiment also provides a device for detecting the distribution of the irregular object, which is used for realizing the above embodiment and the preferred implementation manner, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

The present embodiment provides a distribution detecting apparatus for irregular objects, as shown in fig. 5, including:

the point cloud data acquisition module 501 is configured to acquire point cloud data of a working scene, and perform ground segmentation on the point cloud data to obtain a ground point cloud and a non-ground point cloud;

a ground height determination module 502 for calculating a ground height value based on the ground point cloud;

an irregular target recognition module 503, configured to filter the non-ground point cloud to obtain a target point cloud representing an irregular object;

the point cloud projection module 504 is configured to construct a ground raster pattern of the operation scene, and project a target point cloud into the ground raster pattern;

the distribution information calculating module 505 is configured to determine height distribution information of the irregular object on the ground of the operation scene by using the point cloud height data and the ground height value corresponding to each grid in the ground grid map.

In some alternative embodiments, irregular object recognition module 503 includes:

the obstacle filtering unit is used for performing self-vehicle point cloud filtering, static obstacle filtering and dynamic obstacle filtering on the non-ground point cloud to obtain residual point cloud;

and the impurity filtering unit is used for filtering the space impurities of the residual point cloud to obtain a target point cloud.

In some alternative embodiments, the obstacle filtering unit includes:

the contour range determining unit is used for obtaining the contour range of the vehicle body;

the self-vehicle filtering unit is used for removing point cloud data in the vehicle body contour range from non-ground point clouds;

the positioning information acquisition unit is used for acquiring static obstacle area position information and vehicle positioning information preset in the operation scene;

the static obstacle filtering unit is used for determining a static obstacle point cloud from the non-ground point cloud according to the relative position relation between the vehicle positioning information and the static obstacle region position information, and removing the static obstacle point cloud from the non-ground point cloud;

the target detection acquisition unit is used for acquiring a dynamic target detection result of the operation scene;

and the dynamic obstacle filtering unit is used for removing the dynamic obstacle point cloud covered by the dynamic target detection result from the non-ground point cloud.

In some alternative embodiments, the impurity filtering unit includes:

the semantic segmentation unit is used for carrying out semantic segmentation on each angle image of the operation scene to obtain the external contour of the irregular object in each angle image;

the point cloud projection unit is used for respectively projecting the residual point clouds into each angle image of the operation scene;

the judging unit is used for comparing the external contour in each angle image with the corresponding point cloud projection to determine non-target point cloud projections outside the external contour;

the back projection unit is used for carrying out back mapping on the non-target point cloud projections in the images of all angles to the residual point cloud, and determining the non-target point cloud in the residual point cloud;

and a non-target point filtering unit for removing a non-target point cloud representing the spatial impurity from the remaining point cloud.

In some alternative embodiments, the impurity filtering unit further comprises, thereafter:

the clustering unit is used for clustering the second residual point cloud after the spatial impurity filtering to obtain a plurality of clustering targets;

the abnormal target eliminating unit is used for eliminating abnormal targets from the clustered targets according to a preset abnormal filtering index, wherein the abnormal filtering index at least comprises one of the limit of the number of point clouds in the clustered targets, the limit of projection areas of the clustered targets in multiple directions and the limit of the side length of the circumscribed rectangle of the clustered targets.

In some alternative embodiments, after the abnormal target eliminating unit, the method further includes:

tracking each residual clustering target in a preset time;

the tracking unit is used for respectively acquiring the maximum historical continuous frame number of each residual clustering target in the preset time, wherein the maximum historical continuous frame number represents the maximum time frame number of the residual clustering targets which are continuously and successfully tracked in the preset time;

and the false alarm rejection unit is used for extracting reliable targets from all the residual clustered targets, wherein the reliable targets are residual clustered targets with the maximum historical continuous frame number greater than a preset frame number threshold.

In some alternative embodiments, the distribution information calculation module 505 includes:

the maximum height point cloud computing unit is used for obtaining the maximum point cloud height value corresponding to each grid in the ground grid graph and computing the difference value between the maximum point cloud height value and the ground height value;

and the distribution calculation unit is used for filling the calculated difference values into the corresponding grids to obtain the height distribution information of the irregular object.

Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.

The irregular object distribution detecting device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit ) circuit, a processor and a memory executing one or more software or fixed programs, and/or other devices that can provide the above functions.

The embodiment of the invention also provides computer equipment, which is provided with the distribution detection device of the irregular object shown in the figure 5.

Referring to fig. 6, fig. 6 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, as shown in fig. 6, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 6.

The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.

Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform a method for implementing the embodiments described above.

The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.

The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.

The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.

Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.

Claims (10)

1. A method for detecting a distribution of irregular objects, the method comprising:

acquiring point cloud data of a working scene, and performing ground segmentation on the point cloud data to obtain a ground point cloud and a non-ground point cloud;

calculating a ground height value based on the ground point cloud;

filtering the non-ground point cloud to obtain a target point cloud representing an irregular object;

constructing a ground grid graph of the operation scene, and projecting the target point cloud into the ground grid graph;

and determining the height distribution information of the irregular object on the ground of the operation scene by utilizing the point cloud height data corresponding to each grid in the ground grid diagram and the ground height value.

2. The method of claim 1, wherein filtering the non-ground point cloud to obtain a target point cloud representing an irregular object comprises:

Performing self-vehicle point cloud filtering, static obstacle filtering and dynamic obstacle filtering on the non-ground point cloud to obtain a residual point cloud;

and filtering the space impurities of the residual point cloud to obtain the target point cloud.

3. The method of claim 2, wherein said performing self-vehicle point cloud filtering, static obstacle filtering, and dynamic obstacle filtering on said non-ground point cloud comprises:

acquiring a vehicle body contour range;

removing point cloud data in the vehicle body contour range from the non-ground point cloud;

acquiring static obstacle region position information and vehicle positioning information preset in the operation scene;

determining a static obstacle point cloud from the non-ground point cloud according to the relative position relation between the vehicle positioning information and the static obstacle region position information, and removing the static obstacle point cloud from the non-ground point cloud;

acquiring a dynamic target detection result of a working scene;

and removing the dynamic obstacle point cloud covered by the dynamic target detection result from the non-ground point cloud.

4. The method of claim 2, wherein the spatially filtering the remaining point cloud comprises:

Semantic segmentation is carried out on each angle image of the operation scene, and external contours of irregular objects in each angle image are obtained;

projecting the residual point clouds into each angle image of the operation scene respectively;

comparing the external contour in each angle image with the corresponding point cloud projection to determine non-target point cloud projections outside the external contour;

reversely mapping non-target point cloud projections in the images of all angles into the residual point cloud, and determining the non-target point cloud in the residual point cloud;

and eliminating the non-target point cloud representing the space impurity from the residual point cloud.

5. The method of claim 2, wherein after spatially filtering the remaining point cloud, the method further comprises:

clustering the second residual point cloud after the spatial impurity filtering to obtain a plurality of clustering targets;

and removing the abnormal targets from the clustering targets according to preset abnormal filtering indexes, wherein the abnormal filtering indexes at least comprise one of the limit of the number of point clouds in the clustering targets, the limit of projection areas of the clustering targets in multiple directions and the limit of the side lengths of the circumscribed rectangles of the clustering targets.

6. The method of claim 5, wherein after eliminating outliers from the clustered objects, the method further comprises:

tracking each residual clustering target in a preset time;

respectively obtaining the maximum historical continuous frame number of each residual clustering target in preset time, wherein the maximum historical continuous frame number represents the maximum time frame number of the residual clustering targets continuously and successfully tracked in the preset time;

and extracting reliable targets from the residual clustering targets, wherein the reliable targets are residual clustering targets with the maximum historical continuous frame number larger than a preset frame number threshold.

7. The method according to claim 1, wherein determining the height distribution information of the irregular object on the ground of the operation scene by using the point cloud height data corresponding to each grid in the ground grid map and the ground height value comprises:

acquiring a point cloud height maximum value corresponding to each grid in the ground grid graph, and calculating a difference value between each point cloud height maximum value and the ground height value;

and filling each calculated difference value into a corresponding grid to obtain the height distribution information of the irregular object.

8. A distribution detection apparatus for irregular objects, the apparatus comprising:

the point cloud data acquisition module is used for acquiring point cloud data of a working scene, and carrying out ground segmentation on the point cloud data to obtain a ground point cloud and a non-ground point cloud;

a ground height determination module for calculating a ground height value based on the ground point cloud;

the irregular target identification module is used for filtering the non-ground point cloud to obtain a target point cloud representing an irregular object;

the point cloud projection module is used for constructing a ground grid graph of the operation scene and projecting the target point cloud into the ground grid graph;

and the distribution information calculation module is used for determining the height distribution information of the irregular object on the ground of the operation scene by utilizing the point cloud height data corresponding to each grid in the ground grid diagram and the ground height value.

9. A computer device, comprising:

a memory and a processor in communication with each other, the memory having stored therein computer instructions which, upon execution, cause the processor to perform the method of any of claims 1 to 7.

10. A computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1 to 7.