CN116258832A - Shovel loading volume acquisition method and system based on three-dimensional reconstruction of material stacks before and after shovel loading - Google Patents
Shovel loading volume acquisition method and system based on three-dimensional reconstruction of material stacks before and after shovel loading Download PDFInfo
- Publication number
- CN116258832A CN116258832A CN202211594238.2A CN202211594238A CN116258832A CN 116258832 A CN116258832 A CN 116258832A CN 202211594238 A CN202211594238 A CN 202211594238A CN 116258832 A CN116258832 A CN 116258832A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- material pile
- shovel
- shovel loading
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000000463 material Substances 0.000 title claims abstract description 161
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000013507 mapping Methods 0.000 claims abstract description 28
- 238000001514 detection method Methods 0.000 claims abstract description 22
- 230000011218 segmentation Effects 0.000 claims abstract description 21
- 238000000746 purification Methods 0.000 claims abstract description 12
- 239000011159 matrix material Substances 0.000 claims description 15
- 229920001651 Cyanoacrylate Polymers 0.000 claims description 13
- 239000004830 Super Glue Substances 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000003384 imaging method Methods 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 13
- 239000013598 vector Substances 0.000 claims description 9
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 5
- 230000003287 optical effect Effects 0.000 claims description 5
- 238000010586 diagram Methods 0.000 claims description 4
- 230000001131 transforming effect Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000012937 correction Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 abstract description 3
- 238000005259 measurement Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000003702 image correction Methods 0.000 description 2
- 238000000691 measurement method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000013590 bulk material Substances 0.000 description 1
- 239000003818 cinder Substances 0.000 description 1
- 239000003245 coal Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Graphics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Processing (AREA)
Abstract
The method comprises the following steps: collecting binocular images of the surface of a material pile before shoveling; performing feature point detection, description, matching and purification on the binocular image, calculating to obtain sparse matching feature points and parallax, and obtaining triangular mapping parameters; based on the sparse feature points and parallax, constructing to obtain a sparse three-dimensional point cloud; based on sparse three-dimensional point cloud and triangular mapping parameters, constructing and obtaining a dense three-dimensional point cloud model on the surface of the material pile before shovel loading; carrying out binocular image acquisition on the surface of the material pile after the shoveling of the engineering machinery, and constructing to obtain a dense three-dimensional point cloud model of the surface of the material pile after the shoveling; performing point cloud downsampling and point cloud registration on a dense three-dimensional point cloud model on the surfaces of the material piles before and after the shovel loading; and carrying out fine segmentation on the registered point cloud model according to the shovel region boundary to obtain an actual shovel region point cloud model and obtain the shovel volume. The invention can rapidly and accurately estimate the shovel material volume of the engineering machinery, and has important significance for real-time evaluation of engineering progress and operation efficiency of operators.
Description
Technical Field
The invention relates to the technical field of binocular stereoscopic vision three-dimensional reconstruction and engineering machinery shovel loading volume estimation, in particular to an engineering machinery shovel loading volume acquisition method and system based on three-dimensional reconstruction of material piles before and after shovel loading.
Background
The earthwork of engineering machinery is used as a mechanized operation process, and productivity planning, engineering progress evaluation and engineering labor fund distribution are required. The earth work volume measurement can be performed by integral measurement or single shovel loading measurement of the material pile. The total station robot scanner and the photogrammetry technology based on unmanned aerial vehicle aerial photography are used as an effective three-dimensional modeling method to be widely applied to the integral measurement of earthwork, however, the integral measurement method is suitable for planning the integral productivity of earthwork, but the real-time operation efficiency in the operation process is difficult to evaluate, so that single shovel loading measurement is needed. For single shovel loading measurement, the prior art mainly focuses on calculating the volume of the materials in the shovel through constructing a three-dimensional point cloud model of the materials in the shovel and the shovel, and accordingly, the shovel loading rate and the shovel loading operation efficiency are evaluated. However, the bucket of different earthworks has different specifications and sizes, and the bucket three-dimensional point cloud model-based bucket loading volume estimation method has the defects of strong dependence on the bucket model and poor universality. Therefore, the method for measuring the shovel loading volume of the engineering machinery based on the three-dimensional reconstruction of the material pile is studied, so that the method has important significance in evaluating the continuous engineering progress and the working efficiency of operators.
Methods for obtaining object volume information by non-contact measurement typically employ a three-dimensional laser scanning system and a vision sensor for three-dimensional reconstruction of the object surface. Because the data volume of the point cloud generated when the laser scanning system is used for scanning the object is far greater than that of the vision sensor, the post-processing and the three-dimensional reconstruction of the three-dimensional point cloud are more time-consuming, and the requirement of real-time update of the object volume information perception is difficult to meet. In addition, the three-dimensional laser scanning system is sensitive to the shape and the size, cannot acquire color information, is high in price, and limits the application of the three-dimensional laser scanning system in earthwork to a certain extent. Binocular stereo vision is to utilize two cameras to simulate human eyes to shoot a measured object from different positions, find corresponding matching points of the same spatial point on left and right camera images according to image features, and calculate and solve spatial coordinate points by utilizing a parallax principle and a spatial corresponding relation. The method can quickly obtain dense depth information, has low cost and light weight, and is suitable for three-dimensional reconstruction of the earth work material pile.
The Chinese patent 202010538172.X provides a method for calculating the earthwork of a foundation pit based on a digital informatization technology, and the method is used for realizing accurate and rapid calculation of the earthwork of the foundation pit based on an unmanned aerial vehicle aerial inclined photogrammetry technology. Chinese patent 202010134566.9 proposes a method and a system for automatic extraction and volume measurement of a material pile based on three-dimensional point cloud, wherein the method uses a laser to scan to obtain three-dimensional point cloud data of a material pile scene, and calculates the volume of the whole material pile. The invention patent 201910972066.X of China proposes an irregular stacking volume measurement method based on a binocular camera, the method adopts a semi-global stereo matching algorithm (SGBM) to obtain a parallax image of an integral stacking image, the SGBM determines cost values between corresponding matching points by comparing pixel intensities, and the descriptor description performance based on pixel blocks is relatively weak, so that the accuracy of volume estimation is influenced, and the calculation is time-consuming. The method is suitable for integral measurement of the earthwork, but is difficult to be suitable for estimation of the single shovel loading volume of the engineering machinery and real-time estimation of the working efficiency. The Chinese patent 201911240082.6 discloses a real-time measuring method and measuring device for the earthwork volume of an excavator bucket based on structured light, and the Chinese patent 202111578028.X discloses a bulk material volume measuring method based on image characteristics and a three-dimensional point cloud technology, wherein the two methods belong to volume estimation methods based on three-dimensional modeling of materials in the bucket, and are difficult to be suitable for measuring the volumes of materials in the bucket with different specifications and sizes.
Disclosure of Invention
In order to solve the problems, the invention provides a shovel loading volume acquisition method and system based on three-dimensional reconstruction of a material stack before and after shovel loading, which can rapidly and accurately estimate the shovel loading material volume of engineering machinery and has important significance for real-time evaluation of engineering progress and operation efficiency of operators.
On the one hand, the shovel loading volume acquisition method based on three-dimensional reconstruction of the material stacks before and after shovel loading comprises the following steps:
step (1), binocular image acquisition is carried out on the surface of a material pile before spading;
step (2), carrying out feature point detection and feature point description on a binocular image on the surface of a material pile before spading by adopting a SuperPoint algorithm, carrying out feature point matching and purification by adopting a SuperGlue algorithm and a random sampling consistency algorithm, calculating to obtain sparse matching feature points and parallax, and obtaining corresponding triangular mapping parameters by Delaunay triangulation; according to the sparse feature points and parallax obtained by matching, constructing and obtaining a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle; constructing a maximum posterior probability model to estimate the optimal parallax value of the residual pixel points according to the obtained sparse three-dimensional point cloud and the triangular mapping parameters, and constructing to obtain a dense three-dimensional point cloud model on the surface of the material pile before spading;
step (3), binocular image acquisition is carried out on the surface of the material pile after the construction machinery is shoveled;
step (4), carrying out feature point detection and feature point description on the binocular image of the surface of the material pile after the shoveling, carrying out feature point matching and purification by adopting a SuperPoint algorithm and a random sampling consistency algorithm, calculating to obtain sparse matching feature points and parallax, and obtaining corresponding triangular mapping parameters by Delaunay triangulation; according to the sparse feature points and parallax obtained by matching, constructing and obtaining a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle; constructing a maximum posterior probability model to estimate the optimal parallax value of the residual pixel points according to the obtained sparse three-dimensional point cloud and the triangular mapping parameters, and constructing to obtain a dense three-dimensional point cloud model on the surface of the material pile after spading;
step (5), adopting a voxelized grid method to respectively perform point cloud downsampling on the dense three-dimensional point cloud model of the surface of the material pile before the shoveling and the dense three-dimensional point cloud model of the surface of the material pile after the shoveling; coarsely dividing the down-sampled point cloud at a position which is expanded outwards by a certain distance according to the boundary of the shovel area, and carrying out point cloud registration on the point cloud model before shovel loading and the point cloud model after shovel loading after coarse division by adopting an ICP algorithm; and carrying out fine segmentation on the registered point cloud model according to the shovel region boundary to obtain an actual shovel region point cloud model, and estimating by adopting an Alpha shape algorithm of Delaunay triangulation to obtain the shovel volume.
Preferably, before the binocular image acquisition is performed on the surface of the material pile before the shovel loading, the method further comprises:
and calibrating and correcting images of the binocular stereo camera arranged on the engineering machinery.
Preferably, the calibration of the binocular stereo camera specifically comprises:
the internal parameters and the external parameters of the camera are obtained by a Zhengyou calibration method, and the internal parameters of the camera comprise the projection position coordinates (u 0 ,v 0 ) And the camera focal length f, wherein the external parameters of the camera comprise a rotation matrix R and a translation matrix T, and the conversion between the image pixel coordinate system and the world coordinate system is realized through the internal parameters and the external parameters of the camera.
Preferably, the image correction is performed on the binocular stereo camera, specifically including:
so that a point in the left image can find a corresponding point in the right image along the same horizontal line.
Preferably, the feature point detection and feature point description are performed by using a SuperPoint algorithm, the feature point matching and the feature point purification are performed by using a SuperGlue algorithm and a random sampling consistency algorithm, sparse matching feature points and parallax are obtained by calculation, and corresponding triangular mapping parameters are obtained by Delaunay triangulation, and the method specifically comprises the following steps:
step (2.1), taking a pre-labeled aggregate shape dataset as supervision data, taking a VGG 16-like detector as a basic detector for pre-training to obtain a basic detection network, respectively extracting corner points of a left image and a right image, and labeling key points; constructing a key point loss function and a descriptor loss function, and performing combined training to obtain a SuperPoint detection network;
step (2.2), respectively obtaining descriptors and coordinate positions of corresponding feature points of a left image and a right image of the material pile according to the SuperPoint detection network, and parallelly obtaining feature description vectors D of the left image and the right image (l) and D(r) The method comprises the steps of carrying out a first treatment on the surface of the Inputting two groups of vectors into a SuperGlue attention graph neural network and an optimal matching layer, and iteratively solving a distribution optimization problem through a Sinkhorn algorithm to obtain an optimal distribution matrix; according to the characteristic point pairs corresponding to the horizontal and vertical coordinates of each column of the maximum value of the distribution matrix as matching point pairs, adopting a random sampling consistency algorithm to obtain an affine transformation matrix, carrying out corresponding affine transformation on the right image of the material pile, and completing matching and purifying of the left image and the right image to obtain sparse matching points;
step (2.3), obtaining corresponding triangular mapping parameters through Delaunay triangulation according to the sparse matching points
The calculation is as follows:
wherein n represents a packetContaining pixels of the image on the left
Triangle sequence number of (a); solving the linear equation for the three vertices of each triangle can obtain triangle plane parameters (a i ,b i ,c i )；u n Representing the image pixel abscissa; v n Representing the ordinate of the image pixels.
Preferably, according to the sparse feature points and parallax obtained by matching, a sparse three-dimensional point cloud is constructed based on a binocular camera stereoscopic imaging principle; according to the obtained sparse three-dimensional point cloud and triangular mapping parameters, constructing a maximum posterior probability model to estimate the optimal parallax value of the rest pixel points, and constructing a dense three-dimensional point cloud model on the surface of the material pile before shovel loading or a dense three-dimensional point cloud model on the surface of the material pile after shovel loading, wherein the method specifically comprises the following steps:
step (3.1), constructing and obtaining a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle, and calculating by the following formula:
wherein ,(Xc ,Y c ,Z c ) Is a point in the three-dimensional scene, and (u, v) is a pixel point on the two-dimensional image; f (f) x Representing transforming the focal length f of the camera into a pixel metric in the x-direction; f (f) y Representing transforming the focal length f of the camera into a pixel metric in the y-direction; b represents binocular camera baseline length; u (u) (l) Representing left image pixel abscissa; u (u) (r) Representing the right image pixel abscissa; u (u) 0 Representing the abscissa of the projection position of the optical axis of the camera lens in a pixel coordinate system; v 0 Representing the ordinate of the projection position of the optical axis of the camera lens in a pixel coordinate system;
in the step (3.2) of constructing the method for estimating the optimal parallax value of the residual pixel point by using the maximum posterior probability model, the constructed probability estimation model is shown as the formula (3):
wherein ,
representing an estimated probability value; argmax represents->
Is a parameter set function of (a);
(|) represents a conditional probability; s= (S) 1 ,s 2 ,……,s M ) Represents the previously constructed sparse three-dimensional point clouds, each having three-dimensional coordinates (X c ,Y c ,Z c ) And each point is denoted as s m ＝(u m ,v m ,d m )，d m Is the point (u) m ,v m ) A corresponding parallax; u (u) m Representing the abscissa in each point corresponding to an image pixel; v m Representing the ordinate in each point corresponding to an image pixel;
representation and->
All pixels in the right-hand graph having the same horizontal line; a certain point in the left diagram +.>
Parallax d of (2) n Considering the random variable to be solved, the posterior probability is expressed as the product of the prior probability and the likelihood probability, as shown in formula (4):
assuming that the prior probability is proportional to the gaussian distribution, as shown in equation (5):
wherein ,
representing a pixel-containing +.>
Is mapped to the triangle of (a); oc represents proportional; exp represents an exponential function based on a natural constant e; sigma represents standard deviation; mu represents the mean; assuming that likelihood probabilities can be expressed as a laplace distribution, as shown in equation (6):
wherein ,
and />
The feature description vectors respectively represent the nth pixel point of the left image and the nth pixel point of the right image; II represents a norming function; since the correction of the left and right images obtained by the binocular camera is completed in advance, the corresponding points of the left and right images necessarily appear on the same horizontal polar line, and therefore this constraint condition is ensured by the if condition in formula (6); further deriving a likelihood probability model for equation (6) is shown in equation (7):
and (3.3) obtaining parallax values of all pixel points in the image, and then calculating according to the formula (2) to obtain a dense three-dimensional point cloud model on the surface of the material pile before shovel loading or a dense three-dimensional point cloud model on the surface of the material pile after shovel loading.
Preferably, the step (5) specifically includes:
step (5.1), firstly adopting a voxelized grid method to simultaneously downsample three-dimensional point clouds on the surfaces of the material piles before and after shoveling; after the downsampling is finished, a certain area is outwards expanded on the boundary of the shoveling area, and meanwhile, rough segmentation is respectively carried out on the three-dimensional point cloud of the surface of the material pile before shoveling and the three-dimensional point cloud of the surface of the material pile after shoveling;
step (5.2), taking the roughly-segmented three-dimensional point cloud on the surface of the material pile before shovel loading as a reference model, taking the roughly-segmented three-dimensional point cloud on the surface of the material pile after shovel loading as a registration model, and registering by adopting an ICP algorithm to ensure that the pose of the three-dimensional point cloud on the surface of the material pile before shovel loading and the pose of the three-dimensional point cloud on the surface of the material pile after shovel loading are consistent;
step (5.3), carrying out fine segmentation on three-dimensional point clouds on the surfaces of the material piles before and after the shovel loading after registration according to the boundary limit of the shovel loading area, so as to obtain a point cloud model of an actual shovel loading area;
and (5.4) performing Delaunay triangulation processing on the point cloud model according to the obtained point cloud model of the actual shovel region, fitting the point cloud parameterization to obtain a point cloud contour envelope, and estimating by adopting an Alpha shape algorithm to obtain the shovel volume.
Preferably, the surface of the material pile before the shoveling is the surface of the material pile before the first shoveling or the single shoveling is carried out on the material pile, and the surface profile information of the material pile is included; the surface of the material pile after the shoveling is the surface of the material pile after the single shoveling of the material pile, and the surface profile information of the material pile is included.
In another aspect, a shovel load volume acquisition system based on three-dimensional reconstruction of a material stack before and after shovel load, includes:
the material pile image acquisition module is used for respectively carrying out binocular image acquisition on the surface of the material pile before the shoveling and the surface of the material pile after the shoveling;
the triangular mapping parameter acquisition module is used for carrying out feature point detection and feature point description on the binocular image of the surface of the material pile before the shoveling and the binocular image of the surface of the material pile after the shoveling respectively by adopting a SuperPoint algorithm, carrying out feature point matching and purification by adopting a SuperGlue algorithm and a random sampling consistency algorithm, calculating to obtain sparse matching feature points and parallax, and obtaining corresponding triangular mapping parameters by Delaunay triangulation;
the dense three-dimensional point cloud model construction module is used for constructing a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle according to the sparse feature points and parallax obtained by matching; according to the obtained sparse three-dimensional point cloud and triangular mapping parameters, constructing a maximum posterior probability model to estimate the optimal parallax value of the residual pixel points, and respectively constructing a dense three-dimensional point cloud model on the surface of the material pile before shovel loading and a dense three-dimensional point cloud model on the surface of the material pile after shovel loading;
the shoveling volume acquisition module is used for respectively carrying out point cloud downsampling on the dense three-dimensional point cloud model of the surface of the material pile before shoveling and the dense three-dimensional point cloud model of the surface of the material pile after shoveling by adopting a voxelized grid method; coarsely dividing the down-sampled point cloud at a position which is expanded outwards by a certain distance according to the boundary of the shovel area, and carrying out point cloud registration on the point cloud model before shovel loading and the point cloud model after shovel loading after coarse division by adopting an ICP algorithm; and carrying out fine segmentation on the registered point cloud model according to the shovel region boundary to obtain an actual shovel region point cloud model, and estimating by adopting an Alpha shape algorithm of Delaunay triangulation to obtain the shovel volume.
Compared with the prior art, the invention has the following innovation points and remarkable advantages:
(1) According to the invention, the image characteristic point detection algorithm SuperPoint and the characteristic point description algorithm SuperGlue based on the deep neural network are adopted to accurately and rapidly extract rich material pile characteristic information;
(2) According to the method, a probability model is constructed based on Bayesian estimation to obtain a dense parallax map, sparse point cloud and parallax obtained through matching are used as priori information, the searching range of parallax of the residual pixel points is reduced, the computing efficiency is improved, and meanwhile sufficient matching precision is ensured;
(3) The invention provides a volume estimation method based on three-dimensional point clouds on the surfaces of material piles before and after spading, which comprises the following steps: the method has the advantages that the method reduces the calculation cost of the point cloud model and has higher calculation accuracy at the same time in five stages of point cloud downsampling, point cloud rough segmentation, point cloud registration, point cloud precise segmentation and volume calculation.
Drawings
FIG. 1 is a flow chart of a method for acquiring the shovel loading volume based on three-dimensional reconstruction of a material pile before and after shovel loading;
FIG. 2 is a flow chart of dense three-dimensional point cloud reconstruction of the surface of a material pile before and after spading;
FIG. 3 is a schematic diagram of point cloud registration and segmentation taking shovel boundary limitations into consideration in the present invention;
fig. 4 is a block diagram of a shovel loader volume acquisition system based on three-dimensional reconstruction of a material stack before and after shovel loader according to the present invention.
Detailed Description
The present invention is further described below with reference to the drawings and examples.
Referring to fig. 1, the shovel loading volume acquisition method based on three-dimensional reconstruction of a material stack before and after shovel loading comprises the following steps:
and (1) calibrating and correcting the binocular stereo camera, installing the binocular camera at the high position of the engineering machinery, and collecting binocular images on the surface of the material pile before spading.
In this embodiment, calibrating the binocular stereo camera means obtaining the internal parameters and the external parameters of the camera by Zhang Zhengyou calibration method, wherein the internal parameters of the camera are related to the optical characteristics of the camera, such as the projection position coordinates (u 0 ,v 0 ) And the camera focal length f, the external parameters of the camera comprise a rotation matrix R and a translation matrix T, and the conversion between the image pixel coordinate system and the world coordinate system can be realized through the internal and external parameters.
The binocular stereo camera is subjected to image correction, so that one point in the left image can find a corresponding point in the right image along the same horizontal line.
Specifically, the binocular camera is arranged at the high position of the engineering machinery, so that the binocular camera can shoot the complete binocular image on the surface of the material stack in the whole shovel loading operation process and is not blocked by a task object.
In this embodiment, the surface of the material pile before the material pile is shoveled is the surface of the material pile before the material pile is shoveled for the first time or the single shoveling, and the surface profile information of the material pile is included.
The material pile refers to a material pile of earth and stone of common operation objects of engineering machinery, such as fine sand, coal cinder, raw soil, ore and the like.
Step (2), performing feature point detection and feature point description on the obtained binocular image of the surface of the material pile before loading by a SuperPoint algorithm, performing feature point matching and purification by a SuperGlue algorithm and a random sampling consistency (RANSAC) algorithm, calculating to obtain sparse matching feature points and parallax, and performing Delaunay triangulation processing to obtain corresponding triangular mapping parameters
Specifically, a pre-labeled set shape dataset is used as supervision data, a VGG 16-like basic detector is used as a basic detector for pre-training to obtain a basic detection network, angular points of left and right images (namely a left image and a right image) are respectively extracted, and key point labeling is carried out. And constructing a key point loss function and a descriptor loss function, and performing combined training to obtain the SuperPoint detection network.
Specifically, descriptors and coordinate positions of corresponding key points of left and right images of a material pile are respectively obtained according to a SuperPoint detection network, and a left and right image feature description vector D is obtained in parallel (l) and D(r) . Inputting two groups of vectors into a SuperGlue attention-seeking neural network and an optimal matching layer, and iteratively solving a distribution optimization problem through a sink horn algorithm to obtain an optimal distribution matrix. And then, according to characteristic point pairs corresponding to the horizontal and vertical coordinates of each column of the maximum value of the distribution matrix, obtaining an affine transformation matrix by adopting a random sampling consistency algorithm (RANSAC), and carrying out corresponding affine transformation on the right image of the material pile to complete matching and purification of the left image and the right image, thereby obtaining sparse matching points.
Specifically, according to the sparse matching points, corresponding triangular mapping parameters are obtained through Delaunay triangulation processing
The calculation is as follows:
wherein n represents a pixel including a left image
Is a triangle sequence number of (c). Solving the linear equation for the three vertices of each triangle can obtain triangle plane parameters (a i ,b i ,c i )。
And (3), referring to fig. 2, constructing and obtaining a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle according to the sparse feature points and parallax obtained by matching. And estimating the optimal parallax value of the residual pixel points by constructing a maximum posterior probability model according to the obtained sparse three-dimensional point cloud and the triangular mapping parameters, and constructing to obtain a dense three-dimensional point cloud model on the surface of the material pile before spading. The specific steps are as follows:
step (3.1), constructing and obtaining a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle, and calculating by the following formula:
wherein ,(Xc ,Y c ,Z c ) Is a point in the three-dimensional scene, (u, v) is a pixel point on the two-dimensional image, f x ＝f/d x ，f y ＝f/d y The focal length f of the camera is transformed into a pixel metric in the x, y direction, respectively. B represents the binocular camera baseline length. u (u) (l) 、u (r) Respectively representing left and right image pixel abscissa.
And (3.2) in the method for estimating the optimal parallax value of the residual pixel point by constructing the maximum posterior probability model, the constructed probability estimation model is shown in the formula (3).
Wherein s= (S) 1 ,s 2 ,……,s M ) Representing the previously constructed sparse three-dimensional point cloud, each point represented as s m ＝(u m ,v m ,d m )，d m Is the point (u) m ,v m ) A corresponding parallax.
Representation and->
All pixels in the right image having the same horizontal line. A certain point in the left graph +.>
Parallax d of (2) n Considering the random variable to be solved, the posterior probability may be expressed as a product of the prior probability and the likelihood probability, as shown in equation (4).
It is assumed that the prior probability is proportional to the gaussian distribution as shown in equation (5).
wherein ,
representing a pixel-containing +.>
Is defined by a triangle map of the set of (a). It is assumed that likelihood probabilities can be expressed as a laplace distribution as shown in equation (6).
wherein ,
and />
The feature description vectors of the nth pixel point of the left image and the nth pixel point of the right image are respectively represented. Since the correction of the left and right images obtained by the binocular camera is completed in advance, the corresponding points of the left and right images necessarily appear on the same horizontal polar line, and therefore this constraint condition is ensured by the if condition in the formula (6). Therefore, the likelihood probability model is further derived for the expression (6) as shown in the expression (7).
And (3.3) obtaining parallax values of all pixel points in the image, and then calculating according to the formula (2) to obtain the dense three-dimensional point cloud model on the surface of the material pile before shovel loading.
And (4) after the construction machinery finishes the shoveling, carrying out binocular image acquisition on the surface of the shoveled material pile, and referring to the step (2) and the step (3), constructing and obtaining a dense three-dimensional point cloud model on the surface of the shoveled material pile.
Specifically, the surface of the material pile after the material pile is shoveled is the surface of the material pile after the material pile is shoveled for a single time, and the surface profile information of the material pile is included.
And (5) respectively adopting a voxelized grid method to perform point cloud downsampling on the dense three-dimensional point cloud model of the surface of the material pile before the shovel loading and the dense three-dimensional point cloud model of the surface of the material pile after the shovel loading. And performing rough segmentation on the down-sampled point cloud at a position which is expanded outwards by a certain distance according to the boundary of the shovel loading area, and performing point cloud registration on the point cloud model before and after shovel loading after rough segmentation by adopting an ICP algorithm. And (3) finely dividing the registered point cloud model according to the shovel region boundary to obtain an actual shovel region point cloud model, and estimating the shovel volume by adopting an Alpha shape algorithm of Delaunay triangulation, wherein the principle is shown in figure 3.
Specifically, the method comprises the following steps:
and (5.1), firstly adopting a voxelized grid method to simultaneously downsample three-dimensional point clouds on the surfaces of the material piles before and after the shovel loading. After the downsampling is finished, the boundary of the shovel loading area is outwards expanded to a certain distance area, and meanwhile, the three-dimensional point cloud on the surface of the material pile before and after shovel loading is roughly segmented. Specifically, the certain distance area needs to ensure that the point cloud model after rough segmentation can include all shovel areas, for example, the certain distance length can be set to be 0.1-0.5 times of the minimum distance between the boundary of the shovel area and the outer boundary.
And (5.2) taking the three-dimensional point cloud of the surface of the material pile before the shovel loading after the rough segmentation as a reference model, taking the three-dimensional point cloud of the surface of the material pile after the shovel loading after the rough segmentation as a registration model, and registering by adopting an ICP algorithm, so that the pose of the three-dimensional point cloud of the surface of the material pile before and after the shovel loading has higher consistency.
And (5.3) carrying out fine segmentation on three-dimensional point clouds on the surfaces of the material piles before and after the shovel loading after registration according to the boundary limit of the shovel loading area, and obtaining a point cloud model of an actual shovel loading area.
And (5.4) performing Delaunay triangulation processing on the point cloud model according to the obtained shovel region point cloud model, fitting the point cloud parameterization to obtain a point cloud contour envelope, and estimating the shovel volume by adopting an Alpha shape algorithm.
The shovel loading volume estimation method based on three-dimensional reconstruction of the material stacks before and after shovel loading has the following beneficial effects:
(1) The feature information of the rich material pile is accurately and rapidly extracted by adopting a deep neural network-based image feature point detection algorithm SuperPoint and a feature point description algorithm SuperGlue;
(2) The method comprises the steps of providing a dense parallax map obtained by constructing a probability model based on Bayesian estimation, taking sparse point cloud and parallax obtained by matching as priori information, reducing the searching range of parallax of the residual pixel points, improving the computing efficiency and ensuring enough matching precision;
(3) The volume estimation method based on three-dimensional point clouds on the surfaces of the material piles before and after spading is provided, and comprises the following steps: the method has the advantages that the method reduces the calculation cost of the point cloud model and has higher calculation accuracy at the same time in five stages of point cloud downsampling, point cloud rough segmentation, point cloud registration, point cloud precise segmentation and volume calculation.
Referring to fig. 4, according to another aspect of the present invention, there is also disclosed a shovel loading volume obtaining system based on three-dimensional reconstruction of a material stack before and after shovel loading, including:
the material pile image acquisition module 401 is used for respectively carrying out binocular image acquisition on the surface of the material pile before shoveling and the surface of the material pile after shoveling;
the triangular mapping parameter obtaining module 402 is configured to perform feature point detection and feature point description on a binocular image of a material pile surface before shoveling and a binocular image of a material pile surface after shoveling by using a SuperPoint algorithm, perform feature point matching and purification by using a SuperGlue algorithm and a random sampling consistency algorithm, calculate to obtain sparse matching feature points and parallax, and obtain corresponding triangular mapping parameters through Delaunay triangulation;
the dense three-dimensional point cloud model construction module 403 is configured to construct a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle according to the sparse feature points and parallax obtained by matching; according to the obtained sparse three-dimensional point cloud and triangular mapping parameters, constructing a maximum posterior probability model to estimate the optimal parallax value of the residual pixel points, and respectively constructing a dense three-dimensional point cloud model on the surface of the material pile before shovel loading and a dense three-dimensional point cloud model on the surface of the material pile after shovel loading;
the shovel loading volume acquisition module 404 is configured to perform point cloud downsampling on the dense three-dimensional point cloud model of the surface of the material pile before shovel loading and the dense three-dimensional point cloud model of the surface of the material pile after shovel loading by adopting a voxelized grid method; coarsely dividing the down-sampled point cloud at a position which is expanded outwards by a certain distance according to the boundary of the shovel area, and carrying out point cloud registration on the point cloud model before shovel loading and the point cloud model after shovel loading after coarse division by adopting an ICP algorithm; and carrying out fine segmentation on the registered point cloud model according to the shovel region boundary to obtain an actual shovel region point cloud model, and estimating by adopting an Alpha shape algorithm of Delaunay triangulation to obtain the shovel volume.
The specific implementation of the shovel loading volume acquisition system based on three-dimensional reconstruction of the material stacks before and after shovel loading is the same shovel loading volume acquisition method based on three-dimensional reconstruction of the material stacks before and after shovel loading, and the description of the embodiment is not repeated.
The above embodiments are provided to illustrate the technical concept and features of the present invention and are intended to enable those skilled in the art to understand the content of the present invention and implement the same, and are not intended to limit the scope of the present invention. All equivalent changes or modifications made in accordance with the spirit of the present invention should be construed to be included in the scope of the present invention.
Claims (9)
1. The shovel loading volume acquisition method based on three-dimensional reconstruction of the material stacks before and after shovel loading is characterized by comprising the following steps:
step (1), binocular image acquisition is carried out on the surface of a material pile before spading;
step (2), carrying out feature point detection and feature point description on a binocular image on the surface of a material pile before spading by adopting a SuperPoint algorithm, carrying out feature point matching and purification by adopting a SuperGlue algorithm and a random sampling consistency algorithm, calculating to obtain sparse matching feature points and parallax, and obtaining corresponding triangular mapping parameters by Delaunay triangulation; according to the sparse feature points and parallax obtained by matching, constructing and obtaining a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle; constructing a maximum posterior probability model to estimate the optimal parallax value of the residual pixel points according to the obtained sparse three-dimensional point cloud and the triangular mapping parameters, and constructing to obtain a dense three-dimensional point cloud model on the surface of the material pile before spading;
step (3), binocular image acquisition is carried out on the surface of the material pile after the construction machinery is shoveled;
step (4), carrying out feature point detection and feature point description on the binocular image of the surface of the material pile after the shoveling, carrying out feature point matching and purification by adopting a SuperPoint algorithm and a random sampling consistency algorithm, calculating to obtain sparse matching feature points and parallax, and obtaining corresponding triangular mapping parameters by Delaunay triangulation; according to the sparse feature points and parallax obtained by matching, constructing and obtaining a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle; constructing a maximum posterior probability model to estimate the optimal parallax value of the residual pixel points according to the obtained sparse three-dimensional point cloud and the triangular mapping parameters, and constructing to obtain a dense three-dimensional point cloud model on the surface of the material pile after spading;
step (5), adopting a voxelized grid method to respectively perform point cloud downsampling on the dense three-dimensional point cloud model of the surface of the material pile before the shoveling and the dense three-dimensional point cloud model of the surface of the material pile after the shoveling; coarsely dividing the down-sampled point cloud at a position which is expanded outwards by a certain distance according to the boundary of the shovel area, and carrying out point cloud registration on the point cloud model before shovel loading and the point cloud model after shovel loading after coarse division by adopting an ICP algorithm; and carrying out fine segmentation on the registered point cloud model according to the shovel region boundary to obtain an actual shovel region point cloud model, and estimating by adopting an Alpha shape algorithm of Delaunay triangulation to obtain the shovel volume.
2. The method for acquiring the shovel loading volume based on three-dimensional reconstruction of a material pile before and after shovel loading according to claim 1, wherein before the binocular image acquisition is performed on the surface of the material pile before shovel loading, the method further comprises:
and calibrating and correcting images of the binocular stereo camera arranged on the engineering machinery.
3. The method for acquiring the shovel loading volume based on three-dimensional reconstruction of the material stack before and after shovel loading according to claim 2, wherein the method for calibrating the binocular stereo camera specifically comprises the following steps:
the internal parameters and the external parameters of the camera are obtained by a Zhengyou calibration method, and the internal parameters of the camera comprise the projection position coordinates (u 0 ,v 0 ) And the camera focal length f, wherein the external parameters of the camera comprise a rotation matrix R and a translation matrix T, and the conversion between the image pixel coordinate system and the world coordinate system is realized through the internal parameters and the external parameters of the camera.
4. The method for acquiring the shovel loading volume based on three-dimensional reconstruction of a material stack before and after shovel loading according to claim 2, wherein the method for correcting the image of the binocular stereo camera specifically comprises the following steps:
so that a point in the left image can find a corresponding point in the right image along the same horizontal line.
5. The method for acquiring the shovel loading volume based on three-dimensional reconstruction of a material stack before and after shovel loading according to claim 1, wherein the method is characterized in that a SuperPoint algorithm is adopted for feature point detection and feature point description, a SuperGlue algorithm and a random sampling consistency algorithm are adopted for feature point matching and purification, sparse matching feature points and parallax are obtained through calculation, and corresponding triangular mapping parameters are obtained through Delaunay triangulation processing, and specifically comprises the following steps:
step (2.1), taking a pre-labeled aggregate shape dataset as supervision data, taking a VGG 16-like detector as a basic detector for pre-training to obtain a basic detection network, respectively extracting corner points of a left image and a right image, and labeling key points; constructing a key point loss function and a descriptor loss function, and performing combined training to obtain a SuperPoint detection network;
step (2.2), respectively obtaining descriptors and coordinate positions of corresponding feature points of a left image and a right image of the material pile according to the SuperPoint detection network, and parallelly obtaining feature description vectors D of the left image and the right image (l) and D(r) The method comprises the steps of carrying out a first treatment on the surface of the Inputting two groups of vectors into a SuperGlue attention graph neural network and an optimal matching layer, and iteratively solving a distribution optimization problem through a Sinkhorn algorithm to obtain an optimal distribution matrix; according to the characteristic point pairs corresponding to the horizontal and vertical coordinates of each column of the maximum value of the distribution matrix as matching point pairs, adopting a random sampling consistency algorithm to obtain an affine transformation matrix, carrying out corresponding affine transformation on the right image of the material pile, and completing matching and purifying of the left image and the right image to obtain sparse matching points;
step (2.3), obtaining corresponding triangular mapping parameters through Delaunay triangulation according to the sparse matching points
The calculation is as follows:
wherein n represents a pixel including a left image
Triangle sequence number of (a); solving the linear equation for the three vertices of each triangle can obtain triangle plane parameters (a i ,b i ,c i )；u n Representing the image pixel abscissa; v n Representing the ordinate of the image pixels.
6. The method for acquiring the shovel loading volume based on the three-dimensional reconstruction of the material stacks before and after shovel loading according to claim 5, which is characterized in that a sparse three-dimensional point cloud is constructed based on a binocular camera stereoscopic imaging principle according to sparse feature points and parallax obtained by matching; according to the obtained sparse three-dimensional point cloud and triangular mapping parameters, constructing a maximum posterior probability model to estimate the optimal parallax value of the rest pixel points, and constructing a dense three-dimensional point cloud model on the surface of the material pile before shovel loading or a dense three-dimensional point cloud model on the surface of the material pile after shovel loading, wherein the method specifically comprises the following steps:
step (3.1), constructing and obtaining a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle, and calculating by the following formula:
wherein ,(Xc ,Y c ,Z c ) Is a point in the three-dimensional scene, and (u, v) is a pixel point on the two-dimensional image; f (f) x Representing transforming the focal length f of the camera into a pixel metric in the x-direction; f (f) y Representing transforming the focal length f of the camera into a pixel metric in the y-direction; b represents binocular camera baseline length; u (u) (l) Representing left image pixel abscissa; u (u) (r) Representing the right image pixel abscissa; u (u) 0 Representing projection of camera lens optical axis in pixel coordinate systemA position abscissa; v 0 Representing the ordinate of the projection position of the optical axis of the camera lens in a pixel coordinate system;
in the step (3.2) of constructing the method for estimating the optimal parallax value of the residual pixel point by using the maximum posterior probability model, the constructed probability estimation model is shown as the formula (3):
wherein ,
representing an estimated probability value; argmax represents->
Is a parameter set function of (a);
(|) represents a conditional probability; s= (S) 1 ,s 2 ,……,s M ) Represents the previously constructed sparse three-dimensional point clouds, each having three-dimensional coordinates (X c ,Y c ,Z c ) And each point is denoted as s m ＝(u m ,v m ,d m )，d m Is the point (u) m ,v m ) A corresponding parallax; u (u) m Representing the abscissa in each point corresponding to an image pixel; v m Representing the ordinate in each point corresponding to an image pixel;
representation and->
All pixels in the right-hand graph having the same horizontal line; a certain point in the left diagram +.>
Parallax d of (2) n Considering the random variable to be solved, the posterior probability is expressed as the product of the prior probability and the likelihood probability,as shown in formula (4): />
Assuming that the prior probability is proportional to the gaussian distribution, as shown in equation (5):
wherein ,
representing a pixel-containing +.>
Is mapped to the triangle of (a); oc represents proportional; exp represents an exponential function based on a natural constant e; sigma represents standard deviation; mu represents the mean; assuming that likelihood probabilities can be expressed as a laplace distribution, as shown in equation (6):
wherein ,
and />
The feature description vectors respectively represent the nth pixel point of the left image and the nth pixel point of the right image; II represents a norming function; since the correction of the left and right images obtained by the binocular camera is completed in advance, the corresponding points of the left and right images necessarily appear on the same horizontal polar line, and therefore this constraint condition is ensured by the if condition in formula (6); further deriving likelihood probability models for equation (6) such asFormula (7):
and (3.3) obtaining parallax values of all pixel points in the image, and then calculating according to the formula (2) to obtain a dense three-dimensional point cloud model on the surface of the material pile before shovel loading or a dense three-dimensional point cloud model on the surface of the material pile after shovel loading.
7. The method for acquiring the shovel loading volume based on three-dimensional reconstruction of a material pile before and after shovel loading according to claim 1, wherein the step (5) specifically comprises:
step (5.1), firstly adopting a voxelized grid method to simultaneously downsample three-dimensional point clouds on the surfaces of the material piles before and after shoveling; after the downsampling is finished, a certain area is outwards expanded on the boundary of the shoveling area, and meanwhile, rough segmentation is respectively carried out on the three-dimensional point cloud of the surface of the material pile before shoveling and the three-dimensional point cloud of the surface of the material pile after shoveling;
step (5.2), taking the roughly-segmented three-dimensional point cloud on the surface of the material pile before shovel loading as a reference model, taking the roughly-segmented three-dimensional point cloud on the surface of the material pile after shovel loading as a registration model, and registering by adopting an ICP algorithm to ensure that the pose of the three-dimensional point cloud on the surface of the material pile before shovel loading and the pose of the three-dimensional point cloud on the surface of the material pile after shovel loading are consistent;
step (5.3), carrying out fine segmentation on three-dimensional point clouds on the surfaces of the material piles before and after the shovel loading after registration according to the boundary limit of the shovel loading area, so as to obtain a point cloud model of an actual shovel loading area;
and (5.4) performing Delaunay triangulation processing on the point cloud model according to the obtained point cloud model of the actual shovel region, fitting the point cloud parameterization to obtain a point cloud contour envelope, and estimating by adopting an Alpha shape algorithm to obtain the shovel volume.
8. The method for acquiring the shovel loading volume based on three-dimensional reconstruction of a material pile before and after shovel loading according to claim 1, wherein the surface of the material pile before shovel loading refers to the surface of the material pile before the first shovel loading or the single shovel loading, and comprises surface profile information of the material pile; the surface of the material pile after the shoveling is the surface of the material pile after the single shoveling of the material pile, and the surface profile information of the material pile is included.
9. Shovel loading volume acquisition system based on three-dimensional reconstruction of material piles before and after shovel loading, which is characterized by comprising:
the material pile image acquisition module is used for respectively carrying out binocular image acquisition on the surface of the material pile before the shoveling and the surface of the material pile after the shoveling;
the triangular mapping parameter acquisition module is used for carrying out feature point detection and feature point description on the binocular image of the surface of the material pile before the shoveling and the binocular image of the surface of the material pile after the shoveling respectively by adopting a SuperPoint algorithm, carrying out feature point matching and purification by adopting a SuperGlue algorithm and a random sampling consistency algorithm, calculating to obtain sparse matching feature points and parallax, and obtaining corresponding triangular mapping parameters by Delaunay triangulation;
the dense three-dimensional point cloud model construction module is used for constructing a sparse three-dimensional point cloud based on a binocular camera stereoscopic imaging principle according to the sparse feature points and parallax obtained by matching; according to the obtained sparse three-dimensional point cloud and triangular mapping parameters, constructing a maximum posterior probability model to estimate the optimal parallax value of the residual pixel points, and respectively constructing a dense three-dimensional point cloud model on the surface of the material pile before shovel loading and a dense three-dimensional point cloud model on the surface of the material pile after shovel loading;
the shoveling volume acquisition module is used for respectively carrying out point cloud downsampling on the dense three-dimensional point cloud model of the surface of the material pile before shoveling and the dense three-dimensional point cloud model of the surface of the material pile after shoveling by adopting a voxelized grid method; coarsely dividing the down-sampled point cloud at a position which is expanded outwards by a certain distance according to the boundary of the shovel area, and carrying out point cloud registration on the point cloud model before shovel loading and the point cloud model after shovel loading after coarse division by adopting an ICP algorithm; and carrying out fine segmentation on the registered point cloud model according to the shovel region boundary to obtain an actual shovel region point cloud model, and estimating by adopting an Alpha shape algorithm of Delaunay triangulation to obtain the shovel volume.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211594238.2A CN116258832A (en) | 2022-12-13 | 2022-12-13 | Shovel loading volume acquisition method and system based on three-dimensional reconstruction of material stacks before and after shovel loading |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211594238.2A CN116258832A (en) | 2022-12-13 | 2022-12-13 | Shovel loading volume acquisition method and system based on three-dimensional reconstruction of material stacks before and after shovel loading |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116258832A true CN116258832A (en) | 2023-06-13 |
Family
ID=86678315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211594238.2A Pending CN116258832A (en) | 2022-12-13 | 2022-12-13 | Shovel loading volume acquisition method and system based on three-dimensional reconstruction of material stacks before and after shovel loading |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116258832A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116665139A (en) * | 2023-08-02 | 2023-08-29 | 中建八局第一数字科技有限公司 | Method and device for identifying volume of piled materials, electronic equipment and storage medium |
-
2022
- 2022-12-13 CN CN202211594238.2A patent/CN116258832A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116665139A (en) * | 2023-08-02 | 2023-08-29 | 中建八局第一数字科技有限公司 | Method and device for identifying volume of piled materials, electronic equipment and storage medium |
| CN116665139B (en) * | 2023-08-02 | 2023-12-22 | 中建八局第一数字科技有限公司 | Method and device for identifying volume of piled materials, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112132972B (en) | Three-dimensional reconstruction method and system for fusing laser and image data | |
| CN109945853B (en) | Geographic coordinate positioning system and method based on 3D point cloud aerial image | |
| CN111563921B (en) | Underwater point cloud acquisition method based on binocular camera | |
| CN110634161B (en) | Rapid high-precision estimation method and device for workpiece pose based on point cloud data | |
| KR101533182B1 (en) | 3d streets | |
| WO2014024579A1 (en) | Optical data processing device, optical data processing system, optical data processing method, and optical data processing-use program | |
| CN107767456A (en) | A kind of object dimensional method for reconstructing based on RGB D cameras | |
| CN111127613B (en) | Image sequence three-dimensional reconstruction method and system based on scanning electron microscope | |
| CN111998862B (en) | BNN-based dense binocular SLAM method | |
| CN110533774B (en) | Three-dimensional model reconstruction method based on smart phone | |
| CN113205604A (en) | Feasible region detection method based on camera and laser radar | |
| CN112465849B (en) | Registration method for laser point cloud and sequence image of unmanned aerial vehicle | |
| CN112270698A (en) | Non-rigid geometric registration method based on nearest curved surface | |
| CN113034678A (en) | Three-dimensional rapid modeling method for dam face of extra-high arch dam based on group intelligence | |
| CN112929626A (en) | Three-dimensional information extraction method based on smartphone image | |
| CN116258832A (en) | Shovel loading volume acquisition method and system based on three-dimensional reconstruction of material stacks before and after shovel loading | |
| Parmehr et al. | Automatic registration of optical imagery with 3d lidar data using local combined mutual information | |
| CN112017259B (en) | Indoor positioning and image building method based on depth camera and thermal imager | |
| Ebrahimikia et al. | True orthophoto generation based on unmanned aerial vehicle images using reconstructed edge points | |
| CN111611525B (en) | Remote sensing data elevation calculation method based on object space matching elevation deviation iterative correction | |
| CN112767459A (en) | Unmanned aerial vehicle laser point cloud and sequence image registration method based on 2D-3D conversion | |
| Hamzah et al. | Software Calibration for Stereo Camera on Stereo Vision Mobile Robot using Tsai's Method | |
| CN115457022A (en) | Three-dimensional deformation detection method based on real-scene three-dimensional model front-view image | |
| CN115222789A (en) | Training method, device and equipment for instance depth estimation model | |
| Previtali et al. | An automatic multi-image procedure for accurate 3D object reconstruction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |