CN115583510B - Automatic soil discharging control method and system based on laser scanner - Google Patents

Automatic soil discharging control method and system based on laser scanner Download PDF

Info

Publication number
CN115583510B
CN115583510B CN202211194861.9A CN202211194861A CN115583510B CN 115583510 B CN115583510 B CN 115583510B CN 202211194861 A CN202211194861 A CN 202211194861A CN 115583510 B CN115583510 B CN 115583510B
Authority
CN
China
Prior art keywords
model
training
vector
dimensional
statistical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211194861.9A
Other languages
Chinese (zh)
Other versions
CN115583510A (en
Inventor
袁金祥
赵耀忠
田宏哲
刘强
沈洋
马广玉
咸金龙
刘立丰
朱龙啸
王志元
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huaneng Yimin Coal and Electricity Co Ltd
Original Assignee
Huaneng Yimin Coal and Electricity Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huaneng Yimin Coal and Electricity Co Ltd filed Critical Huaneng Yimin Coal and Electricity Co Ltd
Priority to CN202211194861.9A priority Critical patent/CN115583510B/en
Publication of CN115583510A publication Critical patent/CN115583510A/en
Application granted granted Critical
Publication of CN115583510B publication Critical patent/CN115583510B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G65/00—Loading or unloading
    • B65G65/005—Control arrangements
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G65/00—Loading or unloading
    • B65G65/28—Piling or unpiling loose materials in bulk, e.g. coal, manure, timber, not otherwise provided for
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Geometry (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mechanical Engineering (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Biology (AREA)
  • Biophysics (AREA)
  • Computer Graphics (AREA)
  • Operations Research (AREA)
  • Probability & Statistics with Applications (AREA)
  • Remote Sensing (AREA)
  • Algebra (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The application discloses an automatic soil discharging control method and system based on a laser scanner, wherein a plurality of statistical parameters are firstly obtained from a three-dimensional model of a soil discharging belt collected by the laser scanner and are passed through a sequence encoder to obtain statistical feature vectors, then the three-dimensional model of the soil discharging belt is passed through an image encoder to obtain model feature vectors, then the encoding of the model feature vectors is optimized based on the statistical feature vectors to obtain an optimized model state matrix, and finally the optimized model state matrix is passed through a classifier to obtain a classification result for indicating that the rotation angle of a receiving arm of the soil discharging machine should be increased or decreased. In this way, large equipment such as a soil discharge machine can be remotely and intelligently operated.

Description

Automatic soil discharging control method and system based on laser scanner
Technical Field
The application relates to the technical field of intelligent control, in particular to an automatic soil discharging control method and system based on a laser scanner.
Background
The dumping machine is a key device for continuous mining and semi-continuous mining processes of the strip mine. In continuous mining systems, the dumping machine works in conjunction with a bucket wheel excavator, the bucket wheel excavator being at the head of the system and the dumping machine being at the very tail. The dumping machine is widely applied to strip mines in various countries of the world, and the equipment is slowly enlarged and tends to be in a stable state from the development trend in recent years. The industry is then focusing on improving the structure of the equipment, saving energy, increasing the degree of automation, improving the comfort of the working environment, and increasing environmental protection measures.
The waste discharging machine is self-propelled equipment for discharging waste stones through a belt conveyor on a rotary discharging arm. The waste rock is transported to a discharging arm from a receiving end of the receiving arm through an on-arm belt conveyor and then is discharged to a waste rock field through the on-arm belt conveyor. The rotation of the receiving arm is that the crawler travel mechanism is fixed through one side crawler, and the other side crawler rotates to adjust the direction.
The working environment of the operator of the dumping machine is clear and bad, and as the unmanned technology of bulk material loading and unloading equipment such as bucket wheel machines is more and more mature, the requirements of large-scale equipment such as the dumping machine and bucket wheel excavator on remote operation and intelligent operation are more and more increased.
However, as the mechanism of the track bridge type dumping machine, which needs to be positioned, comprises a discharging trolley, a receiving arm, a track travelling mechanism, a slewing mechanism and a pitching mechanism, the positioning of the discharging point of the track bridge type dumping machine is difficult.
Thus, an optimized automatic soil discharge control scheme is desired.
Disclosure of Invention
The present application has been made to solve the above-mentioned technical problems. The embodiment of the application provides an automatic soil discharge control method and system based on a laser scanner. Firstly, obtaining a plurality of statistical parameters from a three-dimensional model of a soil discharge belt acquired by a laser scanner, passing the three-dimensional model of the soil discharge belt through a sequence encoder to obtain a statistical feature vector, then, passing the three-dimensional model of the soil discharge belt through an image encoder to obtain a model feature vector, then, optimizing the coding of the model feature vector based on the statistical feature vector to obtain an optimized model state matrix, and finally, passing the optimized model state matrix through a classifier to obtain a classification result for indicating that the rotation angle of a receiving arm of the soil discharge machine should be increased or decreased. In this way, large equipment such as a soil discharge machine can be remotely and intelligently operated.
According to an aspect of the present application, there is provided an automatic soil discharging control method based on a laser scanner, comprising:
acquiring a three-dimensional model of the earth discharging belt acquired by a laser scanner;
obtaining a plurality of statistical parameters from the three-dimensional model of the dumping belt, wherein the plurality of statistical parameters comprise the width of the dumping belt stack, the height and the flatness of the dumping belt stack;
the three-dimensional model of the soil discharge belt is passed through an image encoder of the trained Clip model to obtain model feature vectors, wherein the image encoder encodes the three-dimensional model of the soil discharge belt by using a convolutional neural network model with a three-dimensional convolutional kernel;
the plurality of statistical parameters are passed through a sequence encoder of the Clip model which is completed by training to obtain statistical feature vectors;
optimizing the coding of the model feature vector based on the statistical feature vector to obtain an optimized model state matrix; and
and training the optimized model state matrix through a classifier to obtain a classification result, wherein the classification result is used for indicating that the rotation angle of a receiving arm of the dumping machine should be increased or decreased.
In the above automatic soil discharging control method based on a laser scanner, the training the three-dimensional model of the soil discharging belt by the image encoder of the Clip model to obtain the model feature vector includes:
Performing depth convolution coding on the three-dimensional model of the earth discharging belt by using the convolution neural network model with the three-dimensional convolution kernel to obtain a model feature map; and
and carrying out global average pooling treatment on each feature matrix of the model feature map along the channel dimension to obtain the model feature vector.
In the above automatic soil discharging control method based on a laser scanner, the performing depth convolution encoding on the three-dimensional model of the soil discharging belt by using the convolution neural network model with the three-dimensional convolution kernel to obtain a model feature map includes:
input data are respectively carried out in forward transfer of layers by using the convolutional neural network model with the three-dimensional convolutional kernel:
performing three-dimensional convolution processing on the input data based on the three-dimensional convolution check to obtain a convolution characteristic diagram;
carrying out mean pooling treatment on the convolution feature map to obtain a pooled feature map; and
non-linear activation is carried out on the pooled feature map so as to obtain an activated feature map;
the output of the last layer of the convolutional neural network model is the model feature map, and the input of the first layer of the convolutional neural network model is the three-dimensional model of the soil discharging belt.
In the above automatic soil discharging control method based on a laser scanner, the step of obtaining the statistical feature vector by passing the plurality of statistical parameters through a sequence encoder of the Clip model after the training includes:
arranging the plurality of statistical parameters into an input vector;
the full-concatenated layer of the sequence encoder using the trained Clip model is formulated as followsThe input vector is subjected to full-connection coding to extract high-dimensional implicit features of feature values of all positions in the input vector, wherein the formula is as follows:whereinIs the input vector to be used for the input of the input device,is the output vector of the vector,is a matrix of weights that are to be used,is the offset vector of the reference signal,representing a matrix multiplication;
and carrying out one-dimensional convolution coding on the input vector by using a one-dimensional convolution layer of the trained Clip model sequence encoder according to the following formula to extract high-dimensional implicit correlation features among feature values of each position in the input vector, wherein the formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,ais convolution kernel inxWidth in the direction,Is a convolution kernel parameter vector,For a local vector matrix that operates with a convolution kernel,wfor the size of the convolution kernel,representing the input vector.
In the above automatic soil discharging control method based on a laser scanner, the optimizing the coding of the model feature vector based on the statistical feature vector to obtain an optimized model state matrix includes: optimizing the coding of the model feature vector based on the statistical feature vector by the following formula to obtain an optimized model state matrix;
wherein, the formula is:
wherein the method comprises the steps ofThe feature vectors of the model are represented as such,a transpose vector representing the model feature vector,the statistical feature vector is represented as such,representing the state matrix of the optimization model,representing vector multiplication.
In the above automatic soil discharging control method based on a laser scanner, the training the optimized model state matrix by a classifier to obtain a classification result includes:
processing the optimized model state matrix using the classifier to generate the classification result with the following formula:whereinRepresenting the projection of the optimization model state matrix as a vector,is a weight matrix of the full connection layer,representing the bias matrix of the fully connected layer.
The automatic soil discharging control method based on the laser scanner further comprises training the Clip model and the classifier.
In the above automatic soil discharging control method based on a laser scanner, the training the Clip model and the classifier includes:
acquiring training data, wherein the training data comprises a training three-dimensional model of a soil discharge belt acquired by the laser scanner, a plurality of training statistical parameters obtained from the training three-dimensional model of the soil discharge belt, and a real value of which the rotation angle of a receiving arm of the soil discharge machine should be increased or decreased;
passing the training three-dimensional model through an image encoder of the Clip model to obtain a training model feature vector;
passing the plurality of training statistical parameters through a sequence encoder of the Clip model to obtain training statistical feature vectors;
optimizing the codes of the training model feature vectors based on the training statistical feature vectors to obtain a training optimization model state matrix;
the training optimization model state matrix passes through the classifier to obtain a classification loss function value;
calculating a coding mode resolved suppression loss function value of the training statistical feature vector and the training model feature vector, wherein the coding mode resolved suppression loss function value is related to the square of the two norms of the differential feature vector between the training statistical feature vector and the training model feature vector; and
Training the Clip model and the classifier with a weighted sum of the classification loss function value and the suppression loss function value of the coding mode resolution as a loss function value.
In the above automatic soil discharge control method based on a laser scanner, the calculating the suppression loss function value of the coding mode digestion of the training statistical feature vector and the training model feature vector includes:
calculating a suppression loss function value of the coding mode resolution of the training statistical feature vector and the training model feature vector according to the following formula;
wherein, the formula is:
wherein the method comprises the steps ofAndrepresenting the training model feature vector and the training statistical feature vector respectively,andrepresenting the weight matrix of the classifier for the training model feature vector and the training statistical feature vector respectively,representation ofF-norm of matrix, andrepresenting the square of the two norms of the vector,the logarithmic function value is represented with a base of 2,representing per-position subtraction.
According to another aspect of the present application, there is provided an automatic soil discharging control system based on a laser scanner, comprising:
the acquisition module is used for acquiring a three-dimensional model of the soil discharge belt acquired by the laser scanner;
The system comprises a plurality of statistical parameter calculation modules, a data acquisition module and a data processing module, wherein the plurality of statistical parameters are used for obtaining a plurality of statistical parameters from a three-dimensional model of the soil discharge belt, and the plurality of statistical parameters comprise the width of the soil discharge belt stack, the height of the soil discharge belt stack and the flatness;
the image coding module is used for enabling the three-dimensional model of the soil discharging belt to pass through an image coder of the trained Clip model to obtain a model feature vector, wherein the image coder is used for coding the three-dimensional model of the soil discharging belt by using a convolutional neural network model with a three-dimensional convolutional kernel;
the sequence coding module is used for enabling the plurality of statistical parameters to pass through a sequence coder of the Clip model after training so as to obtain statistical feature vectors;
the optimization module is used for optimizing the codes of the model feature vectors based on the statistical feature vectors so as to obtain an optimized model state matrix; and
the result generation module is used for training the optimized model state matrix through the classifier to obtain a classification result, wherein the classification result is used for indicating that the rotation angle of the receiving arm of the dumping machine should be increased or decreased.
Compared with the prior art, the automatic dumping control method and the system based on the laser scanner provided by the application have the advantages that firstly, a plurality of statistical parameters are obtained from a three-dimensional model of a dumping belt collected by the laser scanner and are transmitted through a sequence encoder to obtain statistical feature vectors, then, the three-dimensional model of the dumping belt is transmitted through an image encoder to obtain model feature vectors, then, the encoding of the model feature vectors is optimized based on the statistical feature vectors to obtain an optimized model state matrix, and finally, the optimized model state matrix is transmitted through a classifier to obtain a classification result for indicating that the rotation angle of a receiving arm of the dumping machine is increased or reduced. In this way, large equipment such as a soil discharge machine can be remotely and intelligently operated.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing embodiments of the present application in more detail with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of embodiments of the application and are incorporated in and constitute a part of this specification, illustrate the application and together with the embodiments of the application, and not constitute a limitation to the application. In the drawings, like reference numerals generally refer to like parts or steps.
Fig. 1 is a schematic view of a scenario of an automatic soil discharge control method based on a laser scanner according to an embodiment of the present application.
Fig. 2 is a flowchart of an automatic soil discharge control method based on a laser scanner according to an embodiment of the present application.
Fig. 3 is a schematic diagram of an architecture of an automatic soil discharge control method based on a laser scanner according to an embodiment of the present application.
Fig. 4 is a flowchart showing a sub-step of step S130 in the automatic soil discharging control method based on the laser scanner according to the embodiment of the present application.
Fig. 5 is a flowchart of the sub-step of training the Clip model and the classifier further included in the automatic soil discharge control method based on a laser scanner according to an embodiment of the present application.
Fig. 6 is a block diagram of an automatic soil discharge control system based on a laser scanner according to an embodiment of the present application.
Fig. 7 is a block diagram of an electronic device according to an embodiment of the application.
Detailed Description
Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
Scene overview
It should be understood that automatic dumping is a control system for researching automatic operation of a dumping machine, develops automatic identification technologies such as dumping belt pile height, flatness and coordinates by using mature technologies such as a laser scanner and high-precision positioning, forms real-time 3D modeling, constructs an equal-proportion virtual three-dimensional coordinate space, develops an automatic operation control system, dynamically adjusts actions such as walking and turning of the dumping machine, and realizes automatic dumping operation.
Correspondingly, as the waste stones are transferred to the discharging arm from the receiving end of the receiving arm through the belt conveyor on the arm, the rotation positioning of the receiving arm is determined by the crawler travelling mechanism through the fact that one side crawler is fixed and the other side crawler rotates to adjust the direction. Therefore, the accurate determination of the rotation angle of the receiving arm is an important guarantee for realizing the accurate positioning of the discharging point of the crawler bridge type dumping machine. Based on this, in the technical solution of the present application, it is desirable to accurately determine the rotation angle of the receiving arm based on the three-dimensional model of the discharging belt acquired by the laser scanner, so as to accurately position the discharging point of the discharging machine.
Specifically, in the technical scheme of the application, an artificial intelligent control technology based on deep learning is adopted to extract three-dimensional spatial characteristics of the dumping belt from a three-dimensional model of the dumping belt, and implicit association characteristics of statistical parameters such as dumping belt stack width, dumping belt stack height and flatness of the dumping belt in a high-dimensional characteristic space are utilized as optimization feature vectors to optimize model spatial characteristic extraction of the dumping belt, so that the rotation angle dynamic control of a receiving arm of the dumping machine is carried out. Like this, make the waste rock can transport the discharge arm from receiving the material arm accurately, and then to the discharge point of dumping machine carries out accurate location, and can also be adjusted dynamically actions such as dumping machine walking and gyration realize automatic discharging operation.
Specifically, in the technical scheme of the application, firstly, a three-dimensional model of the dumping belt is acquired through a laser scanner. And obtaining a plurality of statistical parameters from the three-dimensional model of the dumping belt, wherein the plurality of statistical parameters comprise the dumping belt pile width, the dumping belt pile height and the flatness. It should be understood that in the automatic operation of the dumping machine, the width, height, flatness and coordinates of the dumping belt stack are important elements for determining the rotation angle of the receiving arm with respect to the three-dimensional model of the dumping belt, so in the technical scheme of the application, the optimization of the three-dimensional spatial feature extraction of the dumping belt is performed by using the implicit associated feature information of the dumping belt stack width, the dumping belt stack height and the flatness, and obviously, the accuracy of the rotation angle control of the receiving arm can be improved.
And then, for the three-dimensional model of the dumping belt, encoding the three-dimensional model of the dumping belt in an image encoder of a Clip model to obtain a model feature vector. In particular, here, the image encoder encodes the three-dimensional model of the dumping belt using a convolutional neural network model having a three-dimensional convolutional kernel to extract three-dimensional spatial feature information of the dumping belt.
And then, coding the multiple statistical parameters of the width of the dumping belt stack and the height and the flatness of the dumping belt stack in a sequence coder of the Clip model to extract implicit correlation characteristics of the statistical parameters of the width of the dumping belt stack and the height and the flatness of the dumping belt stack so as to obtain a statistical characteristic vector. In particular, in one specific example of the present application, the sequence encoder is composed of a fully connected layer and a one-dimensional convolution layer that are alternately arranged, and extracts the correlation of the plurality of statistical parameters in the time sequence dimension through one-dimensional convolution encoding and extracts the high-dimensional implicit characteristics of the plurality of statistical parameters through fully connected encoding.
Further, based on the statistical feature vector, the encoding of the model feature vector is optimized to obtain an optimized model state matrix. In this way, the implicit correlation characteristics of the statistical parameters of the width of the dumping belt stack, the height of the dumping belt stack and the flatness in the high-dimensional characteristic space can be used as the optimized characteristic vector to optimize the model space characteristic extraction of the dumping belt, so as to perform classification treatment, and dynamically control the rotation angle of the receiving arm of the dumping machine to be increased or reduced.
In particular, in the technical solution of the present application, since the optimization model state matrix is obtained by optimizing the encoding of the model feature vector based on the statistical feature vector, in the training process, when the gradient counter propagates through the image encoder and the sequence encoder of the Clip model, the classification loss function of the classifier may affect the accuracy of the classification result of the optimization model state matrix due to the resolution of the image semantic association mode and the parameter context association mode of the encoding of the image encoder and the sequence encoder caused by the abnormal gradient divergence.
Thus, it is preferable to introduce feature vectors for the model, e.g. denoted asAnd the statistical feature vector, e.g. denoted asThe suppression loss function of the coding mode resolution of (2) is expressed as:
here the number of the elements is the number,andthe classifier is for the model feature vectorAnd the statistical feature vectorIs used for the weight matrix of the (c),represents the F norm of the matrix, anRepresenting the square of the two norms of the vector.
Specifically, the suppression loss function of the coding mode digestion ensures that the directional derivative of the gradient in the opposite propagation is regularized near the branching point of the gradient propagation by keeping the difference distribution of the weighting matrix of the classifier relative to different feature vectors consistent with the true feature difference distribution of the feature vectors in a cross entropy mode, namely, the image semantic association mode of the gradient for coding of the image coder and the parameter context association mode of the sequence coder are subjected to over-weighting, so that the digestion of the coding mode is suppressed, the expression capability of the model feature vector on the image semantic association feature and the expression capability of the statistical feature vector on the parameter context association feature are improved, and the accuracy of the classification result of the optimized model state matrix is correspondingly improved. Like this, can be to the gyration angle of the material receiving arm of the earth-moving machine carries out accurate control to make the waste rock can follow accurately the material receiving arm is transported to the material discharging arm, and then to the discharge point of the earth-moving machine carries out accurate location. Meanwhile, the actions such as walking and turning of the dumping machine can be dynamically adjusted, and automatic dumping operation is realized.
Based on the above, the application provides an automatic soil discharge control method based on a laser scanner, which specifically comprises the following steps: acquiring a three-dimensional model of the earth discharging belt acquired by a laser scanner; obtaining a plurality of statistical parameters from the three-dimensional model of the dumping belt, wherein the plurality of statistical parameters comprise the width of the dumping belt stack, the height and the flatness of the dumping belt stack; the three-dimensional model of the soil discharge belt is passed through an image encoder of the trained Clip model to obtain model feature vectors, wherein the image encoder encodes the three-dimensional model of the soil discharge belt by using a convolutional neural network model with a three-dimensional convolutional kernel; the plurality of statistical parameters are passed through a sequence encoder of the Clip model which is completed by training to obtain statistical feature vectors; optimizing the coding of the model feature vector based on the statistical feature vector to obtain an optimized model state matrix; and training the optimized model state matrix through a classifier to obtain a classification result, wherein the classification result is used for indicating that the rotation angle of a receiving arm of the soil discharging machine should be increased or decreased.
Fig. 1 illustrates an application scenario diagram of an automatic soil discharge control method based on a laser scanner according to an embodiment of the present application. As shown in fig. 1, in this application scenario, a three-dimensional model (e.g., D as shown in fig. 1) of a discharging belt (e.g., F as shown in fig. 1) acquired by a laser scanner (e.g., C as shown in fig. 1) is acquired, and then the three-dimensional model of the discharging belt is input into a server (e.g., S as shown in fig. 1) in which a laser scanner-based automatic discharging control algorithm is deployed, wherein the server is capable of generating a classification result for indicating that a rotation angle of a receiving arm of the discharging machine should be increased or should be decreased with the laser scanner-based automatic discharging control algorithm.
Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described in detail with reference to the accompanying drawings.
Exemplary method
Fig. 2 illustrates a flowchart of an automatic soil discharge control method based on a laser scanner according to an embodiment of the present application. As shown in fig. 2, the automatic soil discharging control method based on the laser scanner according to the embodiment of the application includes the steps of: s110, acquiring a three-dimensional model of a soil discharge belt acquired by a laser scanner; s120, obtaining a plurality of statistical parameters from the three-dimensional model of the dumping belt, wherein the plurality of statistical parameters comprise the dumping belt pile width, the dumping belt pile height and the flatness; s130, passing the three-dimensional model of the dumping belt through an image encoder of the trained Clip model to obtain a model feature vector, wherein the image encoder encodes the three-dimensional model of the dumping belt by using a convolutional neural network model with a three-dimensional convolutional kernel; s140, the plurality of statistical parameters are passed through a sequence encoder of the Clip model after training to obtain statistical feature vectors; s150, optimizing the codes of the model feature vectors based on the statistical feature vectors to obtain an optimized model state matrix; and S160, the optimized model state matrix is classified by training a finished classifier.
Fig. 3 illustrates an architecture diagram of an automatic soil discharge control method based on a laser scanner according to an embodiment of the present application. As shown in fig. 3, in the network architecture, first, a three-dimensional model of a waste tape acquired by a laser scanner is acquired; then, obtaining a plurality of statistical parameters from the three-dimensional model of the dumping belt, wherein the plurality of statistical parameters comprise the dumping belt pile width, the dumping belt pile height and the flatness; then, the three-dimensional model of the dumping belt is passed through an image encoder of the trained Clip model to obtain model feature vectors, wherein the image encoder encodes the three-dimensional model of the dumping belt by using a convolutional neural network model with a three-dimensional convolutional kernel; then, the plurality of statistical parameters pass through a sequence encoder of the Clip model after training to obtain statistical feature vectors; then, optimizing the codes of the model feature vectors based on the statistical feature vectors to obtain an optimized model state matrix; finally, the optimized model state matrix is trained by a classifier to obtain a classification result, wherein the classification result is used for indicating that the rotation angle of a receiving arm of the soil discharging machine should be increased or decreased.
More specifically, in step S110, a three-dimensional model of the earth-discharging belt acquired by the laser scanner is acquired. The automatic dumping is a control system for researching automatic operation of the dumping machine, and by using mature technologies such as a laser scanner, high-precision positioning and the like, the height, flatness and coordinates of the dumping belt stack are automatically identified, real-time 3D modeling is formed, an equal proportion virtual three-dimensional coordinate space is constructed, so that a three-dimensional model of the dumping belt is obtained, and the walking, rotation and other actions of the dumping machine are dynamically adjusted, so that automatic dumping operation is realized.
More specifically, in step S120, a plurality of statistical parameters including a dumping belt stack width, a dumping belt stack height, and a flatness are obtained from the three-dimensional model of the dumping belt. It should be understood that in the automatic operation of the dumping machine, the width, height, flatness and coordinates of the dumping belt stack are important elements for determining the rotation angle of the receiving arm with respect to the three-dimensional model of the dumping belt, so in the technical scheme of the application, the optimization of the three-dimensional spatial feature extraction of the dumping belt is performed by using the implicit associated feature information of the dumping belt stack width, the dumping belt stack height and the flatness, and obviously, the accuracy of the rotation angle control of the receiving arm can be improved.
More specifically, in step S130, the three-dimensional model of the discharging belt is passed through an image encoder of the trained Clip model to obtain a model feature vector, wherein the image encoder encodes the three-dimensional model of the discharging belt using a convolutional neural network model having a three-dimensional convolutional kernel. In particular, here, the image encoder encodes the three-dimensional model of the dumping belt using a convolutional neural network model having a three-dimensional convolutional kernel to extract three-dimensional spatial feature information of the dumping belt.
Accordingly, in one specific example, as shown in fig. 4, in the automatic soil discharging control method based on a laser scanner, the step of passing the three-dimensional model of the soil discharging belt through an image encoder of a Clip model after training to obtain model feature vectors includes the following sub-steps: s131, performing depth convolution coding on the three-dimensional model of the earth discharging belt by using the convolution neural network model with the three-dimensional convolution kernel to obtain a model feature map; and S132, carrying out global mean pooling processing on each feature matrix of the model feature map along the channel dimension to obtain the model feature vector.
Accordingly, in a specific example, in the automatic soil discharging control method based on a laser scanner, the performing depth convolution encoding on the three-dimensional model of the soil discharging belt by using the convolution neural network model with the three-dimensional convolution kernel to obtain a model feature map includes: input data are respectively carried out in forward transfer of layers by using the convolutional neural network model with the three-dimensional convolutional kernel: performing three-dimensional convolution processing on the input data based on the three-dimensional convolution check to obtain a convolution characteristic diagram; carrying out mean pooling treatment on the convolution feature map to obtain a pooled feature map; performing nonlinear activation on the pooled feature map to obtain an activated feature map; the output of the last layer of the convolutional neural network model is the model feature map, and the input of the first layer of the convolutional neural network model is the three-dimensional model of the soil discharging belt.
More specifically, in step S140, the plurality of statistical parameters are passed through a sequence encoder of the trained Clip model to obtain a statistical feature vector. And for the multiple statistical parameters of the width of the dumping belt stack, the height of the dumping belt stack and the flatness, carrying out coding processing on the multiple statistical parameters in a sequence coder of the Clip model to extract implicit correlation characteristics of the statistical parameters of the width of the dumping belt stack, the height of the dumping belt stack and the flatness, thereby obtaining a statistical characteristic vector. In particular, in one specific example of the present application, the sequence encoder is composed of a fully connected layer and a one-dimensional convolution layer that are alternately arranged, and extracts the correlation of the plurality of statistical parameters in the time sequence dimension through one-dimensional convolution encoding and extracts the high-dimensional implicit characteristics of the plurality of statistical parameters through fully connected encoding.
Accordingly, in a specific example, in the automatic soil discharging control method based on a laser scanner, the step of passing the plurality of statistical parameters through a sequence encoder of the Clip model after the training to obtain a statistical feature vector includes: integrating the multiple systemsThe counting parameters are arranged as input vectors; and performing full-connection coding on the input vector by using a full-connection layer of the sequence encoder of the Clip model after training to extract high-dimensional implicit characteristics of characteristic values of all positions in the input vector, wherein the formula is as follows:wherein the input vector is a vector of the input,is the output vector of the vector,is a matrix of weights that are to be used,is the offset vector of the reference signal,representing a matrix multiplication; and carrying out one-dimensional convolution coding on the input vector by using a one-dimensional convolution layer of the trained Clip model sequence encoder according to the following formula to extract high-dimensional implicit correlation features among feature values of each position in the input vector, wherein the formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,ais convolution kernel inxWidth in the direction,Is a convolution kernel parameter vector,For a local vector matrix that operates with a convolution kernel,wfor the size of the convolution kernel, Representing the input vector.
More specifically, in step S150, the encoding of the model feature vector is optimized based on the statistical feature vector to obtain an optimized model state matrix. In this way, the implicit correlation characteristics of the statistical parameters of the width of the dumping belt stack, the height of the dumping belt stack and the flatness in the high-dimensional characteristic space can be used as the optimized characteristic vector to optimize the model space characteristic extraction of the dumping belt, so as to perform classification treatment, and dynamically control the rotation angle of the receiving arm of the dumping machine to be increased or reduced.
Accordingly, in one specific example, in the automatic soil discharging control method based on a laser scanner, the optimizing the encoding of the model feature vector based on the statistical feature vector to obtain an optimized model state matrix includes: optimizing the coding of the model feature vector based on the statistical feature vector by the following formula to obtain an optimized model state matrix; wherein, the formula is:
wherein the method comprises the steps ofThe feature vectors of the model are represented as such,a transpose vector representing the model feature vector, The statistical feature vector is represented as such,representing the state matrix of the optimization model,representing vector multiplication.
More specifically, in step S160, the optimized model state matrix is passed through a trained classifier to obtain a classification.
Accordingly, in a specific example, in the automatic soil discharging control method based on a laser scanner, the training the classifier of the optimized model state matrix to obtain a classification result includes: processing the optimized model state matrix using the classifier to generate the classification result with the following formula:whereinRepresenting the projection of the optimization model state matrix as a vector,is a weight matrix of the full connection layer,representing the bias matrix of the fully connected layer.
More specifically, in one specific example, as shown in fig. 5, the automatic soil discharging control method based on a laser scanner further includes training the Clip model and the classifier.
Accordingly, in one specific example, in the automatic soil discharging control method based on a laser scanner, the training the Clip model and the classifier includes the steps of: s210, acquiring training data, wherein the training data comprises a training three-dimensional model of a soil discharge belt acquired by the laser scanner, a plurality of training statistical parameters obtained from the training three-dimensional model of the soil discharge belt, and a true value of the rotation angle of a receiving arm of the soil discharge machine to be increased or decreased; s220, enabling the training three-dimensional model to pass through an image encoder of the Clip model to obtain training model feature vectors; s230, passing the plurality of training statistical parameters through a sequence encoder of the Clip model to obtain training statistical feature vectors; s240, optimizing the codes of the training model feature vectors based on the training statistical feature vectors to obtain a training optimization model state matrix; s250, passing the training optimization model state matrix through the classifier to obtain a classification loss function value; s260, calculating a suppression loss function value of the coding mode digestion of the training statistical feature vector and the training model feature vector, wherein the suppression loss function value of the coding mode digestion is related to the square of the two norms of the differential feature vector between the training statistical feature vector and the training model feature vector; and S270, training the Clip model and the classifier by taking the weighted sum of the classified loss function value and the suppression loss function value resolved by the coding mode as the loss function value.
In particular, in the technical solution of the present application, since the optimization model state matrix is obtained by optimizing the encoding of the model feature vector based on the statistical feature vector, in the training process, when the gradient counter propagates through the image encoder and the sequence encoder of the Clip model, the classification loss function of the classifier may affect the accuracy of the classification result of the optimization model state matrix due to the resolution of the image semantic association mode and the parameter context association mode of the encoding of the image encoder and the sequence encoder caused by the abnormal gradient divergence.
Accordingly, in one specific example, in the automatic soil discharge control method based on a laser scanner, the calculating the suppression loss function value of the coding mode digestion of the training statistical feature vector and the training model feature vector includes: calculating a suppression loss function value of the coding mode resolution of the training statistical feature vector and the training model feature vector according to the following formula; wherein, the formula is:
wherein the method comprises the steps ofAndrepresenting the training model feature vector and the training statistical feature vector respectively, Andrepresenting the weight matrix of the classifier for the training model feature vector and the training statistical feature vector respectively,represents the F norm of the matrix, anRepresenting the square of the two norms of the vector,the logarithmic function value is represented with a base of 2,representing per-position subtraction.
Specifically, the suppression loss function of the coding mode digestion ensures that the directional derivative of the gradient in the opposite propagation is regularized near the branching point of the gradient propagation by keeping the difference distribution of the weighting matrix of the classifier relative to different feature vectors consistent with the true feature difference distribution of the feature vectors in a cross entropy mode, namely, the image semantic association mode of the gradient for coding of the image coder and the parameter context association mode of the sequence coder are subjected to over-weighting, so that the digestion of the coding mode is suppressed, the expression capability of the model feature vector on the image semantic association feature and the expression capability of the statistical feature vector on the parameter context association feature are improved, and the accuracy of the classification result of the optimized model state matrix is correspondingly improved. Like this, can be to the gyration angle of the material receiving arm of the earth-moving machine carries out accurate control to make the waste rock can follow accurately the material receiving arm is transported to the material discharging arm, and then to the discharge point of the earth-moving machine carries out accurate location. Meanwhile, the actions such as walking and turning of the dumping machine can be dynamically adjusted, and automatic dumping operation is realized.
In summary, according to the automatic soil discharging control method based on the laser scanner of the embodiment of the application, a plurality of statistical parameters are firstly obtained from a three-dimensional model of a soil discharging belt collected by the laser scanner and are passed through a sequence encoder to obtain statistical feature vectors, then the three-dimensional model of the soil discharging belt is passed through an image encoder to obtain model feature vectors, then the encoding of the model feature vectors is optimized based on the statistical feature vectors to obtain an optimized model state matrix, and finally the optimized model state matrix is passed through a classifier to obtain a classification result for indicating that the rotation angle of a material receiving arm of the soil discharging machine should be increased or decreased. In this way, large equipment such as a soil discharge machine can be remotely and intelligently operated.
Exemplary System
Fig. 6 illustrates a block diagram of a laser scanner based automatic dumping control system 100 in accordance with an embodiment of the application. As shown in fig. 6, the automatic soil discharging control system 100 based on a laser scanner according to an embodiment of the present application includes: an acquisition module 110 for acquiring a three-dimensional model of the earth-discharging belt acquired by the laser scanner; a plurality of statistical parameters calculation module 120, configured to obtain a plurality of statistical parameters from the three-dimensional model of the dumping belt, where the plurality of statistical parameters include a dumping belt stack width, a dumping belt stack height, and a flatness; an image encoding module 130, configured to pass the three-dimensional model of the dumping belt through an image encoder of the trained Clip model to obtain a model feature vector, where the image encoder encodes the three-dimensional model of the dumping belt using a convolutional neural network model with a three-dimensional convolutional kernel; a sequence encoding module 140, configured to pass the plurality of statistical parameters through the trained sequence encoder of the Clip model to obtain a statistical feature vector; an optimization module 150, configured to optimize the coding of the model feature vector based on the statistical feature vector to obtain an optimized model state matrix; and a result generating module 160, configured to train the optimized model state matrix through a classifier to obtain a classification result, where the classification result is used to indicate that the rotation angle of the receiving arm of the dumping machine should be increased or decreased.
In one example, in the automatic soil discharging control system 100 based on a laser scanner, the image encoding module 130 includes: the depth convolution coding unit is used for carrying out depth convolution coding on the three-dimensional model of the soil discharging belt by using the convolution neural network model with the three-dimensional convolution kernel so as to obtain a model feature map; and the pooling unit is used for carrying out global mean pooling processing on each feature matrix of the model feature graph along the channel dimension so as to obtain the model feature vector.
In one example, in the automatic soil discharging control system 100 based on a laser scanner, the depth convolution encoding unit is configured to use the convolution neural network model with three-dimensional convolution kernel to perform the following steps on the input data in forward transfer of the layer: performing three-dimensional convolution processing on the input data based on the three-dimensional convolution check to obtain a convolution characteristic diagram; carrying out mean pooling treatment on the convolution feature map to obtain a pooled feature map; performing nonlinear activation on the pooled feature map to obtain an activated feature map; the output of the last layer of the convolutional neural network model is the model feature map, and the input of the first layer of the convolutional neural network model is the three-dimensional model of the soil discharging belt.
In one example, the automatic soil discharge control system based on the laser scanner100, the sequence encoding module 140 includes: an input vector arrangement unit, configured to arrange the plurality of statistical parameters into an input vector; the full-connection unit is used for performing full-connection coding on the input vector by using a full-connection layer of the trained Clip model sequence encoder according to the following formula to extract high-dimensional implicit characteristics of characteristic values of all positions in the input vector, wherein the formula is as follows:whereinIs the input vector to be used for the input of the input device,is the output vector of the vector,is a matrix of weights that are to be used,is the offset vector of the reference signal,representing a matrix multiplication; and a one-dimensional convolution unit, configured to perform one-dimensional convolution encoding on the input vector by using a one-dimensional convolution layer of the trained Clip model to extract high-dimensional implicit correlation features between feature values of each position in the input vector, where the formula is:
wherein, the liquid crystal display device comprises a liquid crystal display device,ais convolution kernel inxWidth in the direction,Is a convolution kernel parameter vector,For a local vector matrix that operates with a convolution kernel,wfor the size of the convolution kernel,representing the input vector.
In one example, in the automatic soil discharging control system 100 based on a laser scanner, the optimizing module 150 is further configured to optimize the encoding of the model feature vector based on the statistical feature vector to obtain an optimized model state matrix according to the following formula; wherein, the formula is:
wherein the method comprises the steps ofThe feature vectors of the model are represented as such,a transpose vector representing the model feature vector,the statistical feature vector is represented as such,representing the state matrix of the optimization model,representing vector multiplication.
In one example, in the automatic soil discharging control system 100 based on a laser scanner, the result generating module 160 is further configured to process the optimized model state matrix with the classifier to generate the classification result according to the following formula:whereinRepresenting the projection of the optimization model state matrix as a vector,is a weight matrix of the full connection layer,representing the bias matrix of the fully connected layer.
In one example, the automatic soil discharging control system 100 based on a laser scanner further includes a training module for training the Clip model and the classifier.
In one example, in the automatic soil discharge control system 100 based on a laser scanner, the training module includes: the training data acquisition unit is used for acquiring training data, wherein the training data comprises a training three-dimensional model of the dumping belt acquired by the laser scanner, a plurality of training statistical parameters obtained from the training three-dimensional model of the dumping belt and a real value of which the rotation angle of a receiving arm of the dumping machine is increased or reduced; the training image coding unit is used for enabling the training three-dimensional model to pass through an image coder of the Clip model so as to obtain training model feature vectors; the training sequence coding unit is used for enabling the plurality of training statistical parameters to pass through a sequence coder of the Clip model so as to obtain training statistical feature vectors; the training optimization unit is used for optimizing the codes of the training model feature vectors based on the training statistical feature vectors so as to obtain a training optimization model state matrix; the classification loss function value calculation unit is used for enabling the training optimization model state matrix to pass through the classifier to obtain a classification loss function value; a suppression loss function value unit configured to calculate a suppression loss function value of a coding mode resolution of the training statistical feature vector and the training model feature vector, wherein the suppression loss function value of the coding mode resolution is related to a square of a two-norm of a differential feature vector between the training statistical feature vector and the training model feature vector; and a weighted training unit for training the Clip model and the classifier with a weighted sum of the classification loss function value and the suppression loss function value resolved by the coding mode as a loss function value.
In one example, in the automatic soil discharge control system 100 based on a laser scanner as described above, the suppression loss function value unit is configured to calculate the suppression loss function value of the coding mode resolution of the training statistical feature vector and the training model feature vector in the following formula; wherein, the formula is:
wherein the method comprises the steps ofAndrepresenting the training model feature vector and the training statistical feature vector respectively,andrepresenting the weight matrix of the classifier for the training model feature vector and the training statistical feature vector respectively,represents the F norm of the matrix, anRepresenting the square of the two norms of the vector,the logarithmic function value is represented with a base of 2,representing per-position subtraction.
Here, it will be understood by those skilled in the art that the specific functions and operations of the respective units and modules in the above-described laser scanner-based automatic soil discharge control system 100 have been described in detail in the above description of the laser scanner-based automatic soil discharge control method with reference to fig. 1 to 5, and thus, repetitive descriptions thereof will be omitted.
As described above, the laser scanner-based automatic soil discharge control system 100 according to the embodiment of the present application may be implemented in various wireless terminals, such as a server of the laser scanner-based automatic soil discharge control system, or the like. In one example, the laser scanner based automatic dumping control system 100 according to embodiments of the application may be integrated into the wireless terminal as a software module and/or hardware module. For example, the laser scanner based automatic dumping control system 100 may be a software module in the operating system of the wireless terminal, or may be an application developed for the wireless terminal; of course, the laser scanner based automatic dumping control system 100 could equally be one of many hardware modules of the wireless terminal.
Alternatively, in another example, the laser scanner based automatic dumping control system 100 may be a separate device from the wireless terminal, and the laser scanner based automatic dumping control system 100 may be connected to the wireless terminal via a wired and/or wireless network and communicate interactive information in accordance with a agreed data format.
Exemplary electronic device
Next, an electronic device according to an embodiment of the present application is described with reference to fig. 7.
Fig. 7 illustrates a block diagram of an electronic device according to an embodiment of the application.
As shown in fig. 7, the electronic device 10 includes one or more processors 11 and a memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM) and/or cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer readable storage medium that may be executed by the processor 11 to implement the laser scanner based automatic soil discharge control and/or other desired functions of the various embodiments of the present application described above. Various contents such as a three-dimensional model of the dumping belt may also be stored in the computer readable storage medium.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown).
The basic principles of the present application have been described above in connection with specific embodiments, however, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are merely examples and not intended to be limiting, and these advantages, benefits, effects, etc. are not to be considered as essential to the various embodiments of the present application. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the application is not necessarily limited to practice with the above described specific details.
The block diagrams of the devices, apparatuses, devices, systems referred to in the present application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.
It is also noted that in the apparatus, devices and methods of the present application, the components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered as equivalent aspects of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the application to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.

Claims (7)

1. An automatic soil discharge control method based on a laser scanner is characterized by comprising the following steps:
acquiring a three-dimensional model of the earth discharging belt acquired by a laser scanner;
Obtaining a plurality of statistical parameters from the three-dimensional model of the dumping belt, wherein the plurality of statistical parameters comprise the width of the dumping belt stack, the height and the flatness of the dumping belt stack;
the three-dimensional model of the soil discharge belt is passed through an image encoder of the trained Clip model to obtain model feature vectors, wherein the image encoder encodes the three-dimensional model of the soil discharge belt by using a convolutional neural network model with a three-dimensional convolutional kernel;
the plurality of statistical parameters are passed through a sequence encoder of the Clip model which is completed by training to obtain statistical feature vectors;
optimizing the coding of the model feature vector based on the statistical feature vector to obtain an optimized model state matrix; and
the optimized model state matrix is used for obtaining a classification result through a classifier which is completed through training, wherein the classification result is used for indicating that the rotation angle of a receiving arm of the soil discharging machine should be increased or decreased;
wherein, the step of obtaining the statistical feature vector by passing the plurality of statistical parameters through the trained sequence encoder of the Clip model comprises the following steps:
arranging the plurality of statistical parameters into an input vector;
and performing full-connection coding on the input vector by using a full-connection layer of the sequence encoder of the Clip model after training to extract high-dimensional implicit characteristics of characteristic values of all positions in the input vector, wherein the formula is as follows: Wherein X is the input vector, Y is the output vector, W is the weight matrix, B is the bias vector, < >>Representing a matrix multiplication;
and carrying out one-dimensional convolution coding on the input vector by using a one-dimensional convolution layer of the trained Clip model sequence encoder according to the following formula to extract high-dimensional implicit correlation features among feature values of each position in the input vector, wherein the formula is as follows:
wherein a is the width of the convolution kernel in the X direction, F (a) is a convolution kernel parameter vector, G (X-a) is a local vector matrix calculated by a convolution kernel function, w is the size of the convolution kernel, and X represents the input vector;
wherein, based on the statistical feature vector, optimizing the coding of the model feature vector to obtain an optimized model state matrix, including: optimizing the coding of the model feature vector based on the statistical feature vector by the following formula to obtain an optimized model state matrix;
wherein, the formula is:
wherein V is s Representing the model feature vector, V s T A transpose vector representing the model feature vector, V b Representing the statistical feature vector, M b Representing the state matrix of the optimization model,representing vector multiplication;
The step of training the optimized model state matrix through a classifier to obtain a classification result comprises the following steps:
processing the optimized model state matrix using the classifier to generate the classification result with the following formula: softmax { (M) c ,B c ) Project (F), where Project (F) represents projecting the optimization model state matrix as a vector, M c Weight matrix of full connection layer, B c Representing the bias of the fully connected layerAnd (5) matrix placement.
2. The automatic soil discharging control method based on the laser scanner according to claim 1, wherein said passing the three-dimensional model of the soil discharging belt through the image encoder of the Clip model completed by training to obtain the model feature vector comprises:
performing depth convolution coding on the three-dimensional model of the earth discharging belt by using the convolution neural network model with the three-dimensional convolution kernel to obtain a model feature map; and
and carrying out global average pooling treatment on each feature matrix of the model feature map along the channel dimension to obtain the model feature vector.
3. The automatic soil discharge control method based on a laser scanner according to claim 2, wherein the depth convolution encoding the three-dimensional model of the soil discharge zone using the convolution neural network model having the three-dimensional convolution kernel to obtain a model feature map comprises:
Input data are respectively carried out in forward transfer of layers by using the convolutional neural network model with the three-dimensional convolutional kernel:
performing three-dimensional convolution processing on the input data based on the three-dimensional convolution check to obtain a convolution characteristic diagram;
carrying out mean pooling treatment on the convolution feature map to obtain a pooled feature map; and
non-linear activation is carried out on the pooled feature map so as to obtain an activated feature map;
the output of the last layer of the convolutional neural network model is the model feature map, and the input of the first layer of the convolutional neural network model is the three-dimensional model of the soil discharging belt.
4. The automatic soil discharge control method based on a laser scanner according to claim 1, further comprising training the Clip model and the classifier.
5. The automatic soil discharge control method based on a laser scanner according to claim 4, wherein the training of the Clip model and the classifier comprises:
acquiring training data, wherein the training data comprises a training three-dimensional model of a soil discharge belt acquired by the laser scanner, a plurality of training statistical parameters obtained from the training three-dimensional model of the soil discharge belt, and a real value of which the rotation angle of a receiving arm of the soil discharge machine should be increased or decreased;
Passing the training three-dimensional model through an image encoder of the Clip model to obtain a training model feature vector;
passing the plurality of training statistical parameters through a sequence encoder of the Clip model to obtain training statistical feature vectors;
optimizing the codes of the training model feature vectors based on the training statistical feature vectors to obtain a training optimization model state matrix;
the training optimization model state matrix passes through the classifier to obtain a classification loss function value;
calculating a coding mode resolved suppression loss function value of the training statistical feature vector and the training model feature vector, wherein the coding mode resolved suppression loss function value is related to the square of the two norms of the differential feature vector between the training statistical feature vector and the training model feature vector; and
training the Clip model and the classifier with a weighted sum of the classification loss function value and the suppression loss function value of the coding mode resolution as a loss function value.
6. The automatic soil discharge control method based on a laser scanner according to claim 5, wherein the calculating the suppression loss function value of the coding mode resolution of the training statistical feature vector and the training model feature vector includes:
Calculating a suppression loss function value of the coding mode resolution of the training statistical feature vector and the training model feature vector according to the following formula;
wherein, the formula is:
wherein V is 1 And V 2 Representing the training model feature vector and the training statistical feature vector, M 1 And M 2 Respectively representing weight matrix of the classifier on the training model feature vector and the training statistical feature vector F Represents the F norm of the matrix, anRepresenting the square of the two norms of the vector, log representing the log function value based on 2, < ->Representing per-position subtraction.
7. An automatic soil discharge control system based on a laser scanner, comprising:
the acquisition module is used for acquiring a three-dimensional model of the soil discharge belt acquired by the laser scanner;
the system comprises a plurality of statistical parameter calculation modules, a data acquisition module and a data processing module, wherein the plurality of statistical parameters are used for obtaining a plurality of statistical parameters from a three-dimensional model of the soil discharge belt, and the plurality of statistical parameters comprise the width of the soil discharge belt stack, the height of the soil discharge belt stack and the flatness;
the image coding module is used for enabling the three-dimensional model of the soil discharging belt to pass through an image coder of the trained Clip model to obtain a model feature vector, wherein the image coder is used for coding the three-dimensional model of the soil discharging belt by using a convolutional neural network model with a three-dimensional convolutional kernel;
The sequence coding module is used for enabling the plurality of statistical parameters to pass through a sequence coder of the Clip model after training so as to obtain statistical feature vectors;
the optimization module is used for optimizing the codes of the model feature vectors based on the statistical feature vectors so as to obtain an optimized model state matrix; and
the result generation module is used for training the optimized model state matrix through a classifier to obtain a classification result, wherein the classification result is used for indicating that the rotation angle of a receiving arm of the soil discharging machine should be increased or decreased;
wherein the sequence encoding module comprises: an input vector arrangement unit, configured to arrange the plurality of statistical parameters into an input vector; the full-connection unit is used for performing full-connection coding on the input vector by using a full-connection layer of the trained Clip model sequence encoder according to the following formula to extract high-dimensional implicit characteristics of characteristic values of all positions in the input vector, wherein the formula is as follows:wherein X is the input vector, Y is the output vector, W is the weight matrix, B is the bias vector, < >>Representing a matrix multiplication; and a one-dimensional convolution unit, configured to perform one-dimensional convolution encoding on the input vector by using a one-dimensional convolution layer of the trained Clip model to extract high-dimensional implicit correlation features between feature values of each position in the input vector, where the formula is:
Wherein a is the width of the convolution kernel in the X direction, F (a) is a convolution kernel parameter vector, G (X-a) is a local vector matrix calculated by a convolution kernel function, w is the size of the convolution kernel, and X represents the input vector;
wherein, the optimization module is used for: optimizing the coding of the model feature vector based on the statistical feature vector by the following formula to obtain an optimized model state matrix; wherein, the formula is:
wherein V is s Representing the model feature vector, V s T A transpose vector representing the model feature vector, V b Representing the statistical feature vector, M b Representing the state matrix of the optimization model,representing vector multiplication;
wherein, the result generation module is used for: processing the optimized model state matrix using the classifier to generate the classification result with the following formula: softmax { (M) c ,B c ) Project (F), where Project (F) represents projecting the optimization model state matrix as a vector, M c Weight matrix of full connection layer, B c Representing the bias matrix of the fully connected layer.
CN202211194861.9A 2022-09-29 2022-09-29 Automatic soil discharging control method and system based on laser scanner Active CN115583510B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211194861.9A CN115583510B (en) 2022-09-29 2022-09-29 Automatic soil discharging control method and system based on laser scanner

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211194861.9A CN115583510B (en) 2022-09-29 2022-09-29 Automatic soil discharging control method and system based on laser scanner

Publications (2)

Publication Number Publication Date
CN115583510A CN115583510A (en) 2023-01-10
CN115583510B true CN115583510B (en) 2023-10-27

Family

ID=84773439

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211194861.9A Active CN115583510B (en) 2022-09-29 2022-09-29 Automatic soil discharging control method and system based on laser scanner

Country Status (1)

Country Link
CN (1) CN115583510B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109685066A (en) * 2018-12-24 2019-04-26 中国矿业大学(北京) A kind of mine object detection and recognition method based on depth convolutional neural networks
CN110194375A (en) * 2019-05-30 2019-09-03 北京中盛博方环保工程技术有限公司 It is a kind of for the automatic stockpiling of material field, feeding, material stacking and fetching method and system
CN111522364A (en) * 2020-06-03 2020-08-11 中国电建集团长春发电设备有限公司 Remote control system with automatic addressing function for dumping plough
CN112508041A (en) * 2020-12-21 2021-03-16 苏州锐希信息技术有限公司 Training method of neural network for spray control based on classification result label
WO2021068503A1 (en) * 2019-10-10 2021-04-15 中国矿业大学 Method for automatically controlling traction speed of coal cutter on the basis of coal wall caving degree
CN113255527A (en) * 2021-05-28 2021-08-13 汉谷云智(武汉)科技有限公司 Method and equipment for monitoring operation normative during concrete unloading process
CN114372530A (en) * 2022-01-11 2022-04-19 北京邮电大学 Abnormal flow detection method and system based on deep self-coding convolutional network

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109685066A (en) * 2018-12-24 2019-04-26 中国矿业大学(北京) A kind of mine object detection and recognition method based on depth convolutional neural networks
CN110194375A (en) * 2019-05-30 2019-09-03 北京中盛博方环保工程技术有限公司 It is a kind of for the automatic stockpiling of material field, feeding, material stacking and fetching method and system
WO2021068503A1 (en) * 2019-10-10 2021-04-15 中国矿业大学 Method for automatically controlling traction speed of coal cutter on the basis of coal wall caving degree
CN111522364A (en) * 2020-06-03 2020-08-11 中国电建集团长春发电设备有限公司 Remote control system with automatic addressing function for dumping plough
CN112508041A (en) * 2020-12-21 2021-03-16 苏州锐希信息技术有限公司 Training method of neural network for spray control based on classification result label
CN113255527A (en) * 2021-05-28 2021-08-13 汉谷云智(武汉)科技有限公司 Method and equipment for monitoring operation normative during concrete unloading process
CN114372530A (en) * 2022-01-11 2022-04-19 北京邮电大学 Abnormal flow detection method and system based on deep self-coding convolutional network

Also Published As

Publication number Publication date
CN115583510A (en) 2023-01-10

Similar Documents

Publication Publication Date Title
CN112001914B (en) Depth image complement method and device
Lyu et al. ChipNet: Real-time LiDAR processing for drivable region segmentation on an FPGA
You et al. Shiftaddnet: A hardware-inspired deep network
KR102318772B1 (en) Domain Separation Neural Networks
CN101253493B (en) Training convolutional neural networks on graphics processing units
CN113052109A (en) 3D target detection system and 3D target detection method thereof
CN110956655B (en) Dense depth estimation method based on monocular image
JP7169328B2 (en) Neural Task Planner for Autonomous Vehicles
Lofqvist et al. Accelerating deep learning applications in space
CN112652066B (en) Three-dimensional geological model-based geological characterization condition evaluation method and system
Ji et al. An encoder-decoder deep learning method for multi-class object segmentation from 3D tunnel point clouds
CN112052736A (en) Cloud computing platform-based field tea tender shoot detection method
CN113724388A (en) Method, device and equipment for generating high-precision map and storage medium
Xue et al. Rock segmentation visual system for assisting driving in TBM construction
Nubert et al. Learning-based localizability estimation for robust lidar localization
CN115583510B (en) Automatic soil discharging control method and system based on laser scanner
JP2022102966A (en) Information processing device and information processing method
CN116994093A (en) Hyperspectral image classification method based on dual-graph convolution neural network
Kumar et al. Efficient volume estimation for dynamic environments using deep learning on the edge
CN112949841B (en) Attention-based pedestrian re-identification method of CNN neural network
CN114821033A (en) Three-dimensional information enhanced detection and identification method and device based on laser point cloud
Bousias Alexakis et al. Evaluation of semi-supervised learning for CNN-based change detection
CN112766462A (en) Data processing method, device and computer readable storage medium
Iorga et al. An image processing vliw architecture for real-time depth detection
Abrão Zeni et al. Semiautomatic determination of the geological strength index using SfM and ANN techniques

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant