CN115115498A - Excavator operation material judgment device and method based on visual identification - Google Patents
Excavator operation material judgment device and method based on visual identification Download PDFInfo
- Publication number
- CN115115498A CN115115498A CN202210721214.2A CN202210721214A CN115115498A CN 115115498 A CN115115498 A CN 115115498A CN 202210721214 A CN202210721214 A CN 202210721214A CN 115115498 A CN115115498 A CN 115115498A
- Authority
- CN
- China
- Prior art keywords
- bucket
- information
- data
- visual identification
- excavator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0014—Image feed-back for automatic industrial control, e.g. robot with camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Abstract
The invention discloses an excavator operating material judgment device and method based on visual identification, which comprises the following steps: the visual identification information transmission unit is used for identifying the operation information in the bucket operation process and transmitting the operation information to the information receiving unit; the information receiving unit is used for receiving the job information transmitted by the visual identification information transmission unit and transmitting the job information to the data analysis and statistics platform; and the data analysis and statistics platform is used for carrying out analysis and statistics on the operation information transmitted by the information receiving unit and calculating quantitative judgment on the bucket workload. The device is simple, the method is simple and convenient, errors caused by a large number of process parameters are eliminated, the operation information data can be reserved, the operation process can be traced, and the quantitative measurement of the excavator materials in the actual operation process is realized.
Description
Technical Field
The invention relates to an excavator operating material judgment device and method based on visual identification, and belongs to the technical field of excavators.
Background
An excavator is an earth moving machine that excavates material above or below a bearing surface with a bucket and loads it into a transport vehicle or unloads it to a stockyard. The materials excavated by the excavator mainly comprise soil, coal, silt, soil subjected to pre-loosening and rocks. At present, most of projects adopt a method for measuring soil loading amount by a measuring tool after the soil loading of a bucket is leveled to measure the bucket capacity, for example, an online weighing method (CN104132721A) of materials in a bucket of a hydraulic excavator discloses that the method comprises the following steps: acquiring pressure value P of hydraulic cylinder of bucket rod in real time 1 And pressure value P of hydraulic cylinder of bucket 2 (ii) a By excavatorsThe center of a hinged point of the base and the movable arm is an original point, the horizontal advancing direction is an X axis, the direction perpendicular to the X axis is a y axis, the inclination angle theta of the bucket rod relative to the horizontal plane is obtained in real time, the angle theta is positive above the X axis, and the angle theta is negative below the X axis; real-time acquisition of length l of bucket rod hydraulic cylinder AB Length l of hydraulic bucket cylinder DE (ii) a And then the data processor of the material weighing module brings the acquired data into a formula, so that the weight of the material in the bucket can be obtained.
Therefore, through actual dress soil volume measurement scraper bowl fill volume, not only can't use in the actual operation, also consuming time and wasting power, a large amount of errors can be introduced in the calculation to material automatic weighing's mode in addition, and can't solve the centrobaric problem of material, can only revise the material barycenter according to calculated result and material density, pile up the state.
Disclosure of Invention
In view of the problems in the prior art, the invention provides an excavator working material determination device and method based on visual recognition, which can measure quantitative parameters of bucket workload in the excavator working process.
In order to achieve the above object, an excavator work material determination device according to the present invention includes:
the visual identification information transmission unit is used for identifying the operation information in the bucket operation process and transmitting the operation information to the information receiving unit;
the information receiving unit is used for receiving the job information transmitted by the visual identification information transmission unit and transmitting the job information to the data analysis and statistics platform;
and the data analysis and statistics platform is used for carrying out analysis and statistics on the operation information transmitted by the information receiving unit and calculating quantitative judgment on the bucket workload.
Preferably, the job information includes real-time image information and feature image information.
Preferably, the visual recognition information transmission unit is installed above the bucket.
Preferably, the visual identification information transmission unit comprises a visual identification camera, a signal processing board card and a signal transmitter; the vision recognition camera is connected with the signal processing board card, and the signal processing board card is connected with the signal transmitter.
Preferably, the information receiving unit adopts a signal receiver which is matched with a signal transmitter in the visual identification information transmission unit for signal transmission.
Preferably, the data analysis and statistics platform adopts a computer.
In addition, the invention also provides a method for adopting the excavator operating material judgment device based on the visual identification, which realizes point cloud mapping processing of materials by taking material image data in a bucket in the excavator operating process as input data and combining a target detection algorithm, and finally forms quantitative judgment of the workload relative to the bucket capacity in the actual operating process.
Preferably, the method specifically comprises the following steps:
1) an installation determination device for performing camera calibration: the method comprises the following steps of (1) installing and connecting a judgment device, calibrating camera reference distance and bucket size information after the installation is finished, and calculating the flat loading capacity of a bucket;
2) monitoring bucket operation information: after calibration is completed, monitoring the bucket by adopting a deep learning target detection algorithm, and recording real-time image information in the actual operation process of the bucket;
3) bucket work motion capture: according to the comparison of the bucket position characteristic information, performing motion capture of bucket operation and recording characteristic image information;
4) creating and excavating material point cloud data mapping: according to the image information captured by the action, the bucket range is taken as a boundary to carry out image information segmentation and point cloud data mapping;
5) performing algorithm processing based on bucket leveling capacity data, and outputting a workload quantitative coefficient: comparing the mapped data model with the bucket calibration information, and calculating quantitative judgment of bucket workload;
6) recording target captured image data and corresponding workload coefficients: the captured image data and corresponding workload coefficients are recorded and saved.
Preferably, in the step 1), the determination device is installed and connected, after the installation is completed, the camera reference distance and the bucket size information are calibrated, the distance between the bottom of the bucket and the plane is measured by taking the top plane of the bucket as a reference to form a point cloud model, and bucket leveling volume fitting calculation is performed according to the point cloud model to obtain the bucket leveling capacity.
Preferably, in the step 5), the mapped data model is compared with bucket calibration information, the established data set is used as a model input to respectively train a target detection network and a classification and rating network, then the trained network is deployed to an edge computing terminal, and camera image data is used as data to output a bucket workload quantitative coefficient.
Compared with the prior art, the method provided by the invention has the advantages that the point cloud mapping processing of the material is realized by taking the material image data in the bucket as input data in the excavator operation process and combining a target detection algorithm, and finally, the quantitative judgment of the operation amount relative to the bucket capacity in the actual operation process is formed. The device is simple, the method is simple and convenient, errors caused by a large number of process parameters are eliminated, the operation information data can be reserved, the operation process can be traced, and the quantitative measurement of the excavator materials in the actual operation process is realized.
Drawings
FIG. 1 is a schematic view of a measurement process according to the present invention;
FIG. 2 is a flow chart of data transmission according to the present invention;
FIG. 3 is a schematic structural diagram of the apparatus of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below. It should be understood, however, that the description herein of specific embodiments is only intended to illustrate the invention and not to limit the scope of the invention.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, and the terms used herein in the specification of the present invention are for the purpose of describing particular embodiments only and are not intended to limit the present invention.
As shown in fig. 1, 2 and 3, the excavator work material determination device based on visual recognition includes:
the visual identification information transmission unit 1 is arranged above the bucket, and the visual identification information transmission unit 1 is used for identifying real-time image information and characteristic image information in the bucket operation process and transmitting the image information to the information receiving unit 2;
an information receiving unit 2 for receiving the image information transmitted by the visual identification information transmitting unit 1 and transmitting the image information to the data analysis and statistics platform 3;
and a data analysis and statistics platform 3 for analyzing and counting the image information transmitted by the information receiving unit 2 and calculating quantitative judgment of the bucket workload.
As an improvement of the embodiment, as shown in fig. 2 and 3, the visual identification information transmission unit 1 includes a visual identification camera, a signal processing board, and a signal transmitter, the visual identification camera is connected to the signal processing board, and the signal processing board is connected to the signal transmitter.
As a modification of the embodiment, the information receiving unit 2 may adopt a conventional signal receiver, and the signal receiver performs signal transmission in cooperation with a signal transmitter in the visual identification information transmission unit 1; the data analysis and statistics platform 3 may employ a conventional computer.
In addition, as shown in fig. 1 to 3, the present invention also provides a method using the excavator work material determination device based on visual recognition, including the steps of:
1) an installation determination device that performs camera calibration: mounting and connecting a judgment device, as shown in FIG. 3, calibrating the reference distance of the camera and the size information of the bucket after the mounting is finished, and calculating the flat loading capacity of the bucket;
2) monitoring bucket operation information: after calibration is completed, monitoring the bucket by adopting a deep learning target detection algorithm, and recording real-time image information in the actual operation process of the bucket;
3) bucket work motion capture: according to the comparison of the bucket position characteristic information, performing motion capture of bucket operation and recording characteristic image information;
4) creating and excavating material point cloud data mapping: according to the image information captured by the action, the bucket range is taken as a boundary to carry out image information segmentation and point cloud data mapping;
5) performing algorithm processing based on bucket leveling capacity data, and outputting a workload quantitative coefficient: comparing the mapped data model with the bucket calibration information, and calculating quantitative judgment of bucket workload;
6) recording target captured image data and corresponding workload coefficients: the captured image data and corresponding workload coefficients are recorded and saved.
Example 1
A method for adopting an excavator operating material judgment device based on visual identification comprises the following steps:
1) an installation determination device for performing camera calibration: the method comprises the following steps of (1) installing and connecting a judgment device, calibrating camera reference distance and bucket size information after the installation is finished, measuring the distance between the bottom of a bucket and a plane by taking the top plane of the bucket as a reference, forming a point cloud model, and carrying out bucket flat volume fitting calculation according to the point cloud model to obtain the flat capacity of the bucket;
2) monitoring bucket operation information: after calibration is completed, monitoring the bucket by adopting a deep learning target detection algorithm, and recording real-time image information in the actual operation process of the bucket;
3) bucket work motion capture: according to the comparison of the bucket position characteristic information, performing motion capture of bucket operation and recording characteristic image information;
4) creating and excavating material point cloud data mapping: according to the image information captured by the action, the bucket range is taken as a boundary to carry out image information segmentation and point cloud data mapping;
5) performing algorithm processing based on bucket leveling capacity data, and outputting a workload quantitative coefficient: comparing the mapped data model with bucket calibration information, inputting the established data set as a model into a training target detection network and a classification and rating network respectively, deploying the trained network to an edge computing terminal, and outputting a bucket workload quantitative coefficient by taking camera image data as data;
6) recording target captured image data and corresponding workload coefficients: the captured image data and corresponding workload coefficients are recorded and saved.
According to the invention, the point cloud mapping processing of the material is realized by taking the material image data in the bucket as input data in the excavator operation process and combining a target detection algorithm, and finally, the quantitative judgment of the workload relative to the bucket capacity in the actual operation process is formed. The device is simple, the method is simple and convenient, errors caused by a large number of process parameters are eliminated, the operation information data can be reserved, the operation process can be traced, and the quantitative measurement of the excavator materials in the actual operation process is realized.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents or improvements made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. The utility model provides an excavator operation material decision maker based on visual identification which characterized in that includes:
the visual identification information transmission unit is used for identifying the operation information in the bucket operation process and transmitting the operation information to the information receiving unit;
the information receiving unit is used for receiving the job information transmitted by the visual identification information transmission unit and transmitting the job information to the data analysis and statistics platform;
and the data analysis and statistics platform is used for carrying out analysis and statistics on the operation information transmitted by the information receiving unit and calculating quantitative judgment on the bucket workload.
2. The excavator work material determination device based on visual recognition as claimed in claim 1, wherein the work information includes real-time image information and characteristic image information.
3. The visual recognition-based excavator working material determination device according to claim 1, wherein the visual recognition information transmission unit is installed above the bucket.
4. The excavator working material judgment device based on visual identification as claimed in claim 3, wherein the visual identification information transmission unit comprises a visual identification camera, a signal processing board card and a signal transmitter;
the vision recognition camera is connected with the signal processing board card, and the signal processing board card is connected with the signal transmitter.
5. The excavator working material judging device based on visual identification as claimed in claim 4, wherein the information receiving unit adopts a signal receiver, and the signal receiver is matched with a signal transmitter in the visual identification information transmission unit for signal transmission.
6. The excavator working material determination device based on visual recognition as claimed in claim 1, wherein the data analysis statistical platform is a computer.
7. The method for adopting the excavator working material judgment device based on the visual identification as claimed in any one of claims 1 to 6 is characterized in that point cloud mapping processing of materials is realized by taking material image data in a bucket in the excavator working process as input data and combining a target detection algorithm, and finally, quantitative judgment of the working amount relative to the bucket capacity in the actual working process is formed.
8. The method for determining the operating material of the excavator based on the visual recognition as claimed in claim 7, which comprises the following steps:
1) an installation determination device for performing camera calibration: the method comprises the following steps of (1) installing and connecting a judgment device, calibrating camera reference distance and bucket size information after the installation is finished, and calculating the flat loading capacity of a bucket;
2) bucket operation information monitoring: after calibration is completed, monitoring the bucket by adopting a deep learning target detection algorithm, and recording real-time image information in the actual operation process of the bucket;
3) bucket work motion capture: according to the comparison of the bucket position characteristic information, performing motion capture of bucket operation and recording characteristic image information;
4) creating and excavating material point cloud data mapping: according to the image information captured by the action, the bucket range is taken as a boundary to carry out image information segmentation and point cloud data mapping;
5) performing algorithm processing based on bucket leveling capacity data, and outputting a workload quantitative coefficient: comparing the mapped data model with the bucket calibration information, and calculating quantitative judgment of bucket workload;
6) recording target captured image data and corresponding workload coefficients: the captured image data and corresponding workload coefficients are recorded and saved.
9. The method for determining the excavator working material based on the visual recognition as claimed in claim 8, wherein in the step 1), the determination device is installed and connected, after the installation is completed, the camera reference distance and the bucket size information are calibrated, the distance between the bottom of the bucket and the plane is measured by taking the top plane of the bucket as a reference, a point cloud model is formed, and the bucket flat-loading volume is calculated according to the point cloud model by fitting to obtain the bucket flat-loading capacity.
10. The method for determining the excavator working material based on the visual recognition as recited in claim 8, wherein in the step 5), the mapped data model is compared with bucket calibration information, the established data set is used as the model input to respectively train the target detection network and the classification and rating network, then the trained network is deployed to the edge computing terminal, and the camera image data is used as the data to output the quantitative coefficient of the bucket workload.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210721214.2A CN115115498A (en) | 2022-06-24 | 2022-06-24 | Excavator operation material judgment device and method based on visual identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210721214.2A CN115115498A (en) | 2022-06-24 | 2022-06-24 | Excavator operation material judgment device and method based on visual identification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115115498A true CN115115498A (en) | 2022-09-27 |
Family
ID=83328980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210721214.2A Pending CN115115498A (en) | 2022-06-24 | 2022-06-24 | Excavator operation material judgment device and method based on visual identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115115498A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115307548A (en) * | 2022-10-12 | 2022-11-08 | 北京鸿游科技有限公司 | Dynamic monitoring device for excavating equipment and storage medium thereof |
-
2022
- 2022-06-24 CN CN202210721214.2A patent/CN115115498A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115307548A (en) * | 2022-10-12 | 2022-11-08 | 北京鸿游科技有限公司 | Dynamic monitoring device for excavating equipment and storage medium thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11417008B2 (en) | Estimating a volume of contents in a container of a work vehicle | |
| US11634987B2 (en) | Safety early warning method and device for full-section tunneling of tunnel featuring dynamic water and weak surrounding rock | |
| Kong et al. | Automatic identification and characterization of discontinuities in rock masses from 3D point clouds | |
| CN110805093B (en) | Container angle sensing with feedback loop control using vision sensors | |
| US8930091B2 (en) | Measurement of bulk density of the payload in a dragline bucket | |
| CN104502990A (en) | Geological survey method of tunnel face based on digital image | |
| CN111691888B (en) | Accurate shovel loading system and method for surface mine | |
| CN111062254B (en) | Method and device for evaluating bulk density of materials in loader bucket | |
| CN108921165B (en) | Water level identification method based on water gauge image | |
| CN111705856B (en) | Numerical analysis-based foundation pit settlement monitoring system and prediction method | |
| CN115115498A (en) | Excavator operation material judgment device and method based on visual identification | |
| WO2020093631A1 (en) | Antenna downtilt angle measurement method based on depth instance segmentation network | |
| CN112184707A (en) | Method and system for judging earth and stone load of muck vehicle based on point cloud data | |
| CN115588043A (en) | Excavator operation pose monitoring method based on vision | |
| CN110926426B (en) | Real-time detection device and detection method for residual muck volume of subway shield segment | |
| CN112800530A (en) | Digital data processing method for reinforcing stress strain of broken soft surrounding rock face | |
| CN113446932A (en) | Non-contact crack measuring method and system | |
| CN115791803A (en) | Deep-buried tunnel surrounding rock blasting damage test system and test method | |
| CN115821880A (en) | Tamping settlement measuring method based on TOF camera | |
| CN113484867A (en) | Imaging sonar-based fish school density detection method in closed space | |
| CN116309552B (en) | Method, device, equipment and medium for evaluating health state of existing line old retaining wall | |
| Hu et al. | Three-dimensional characterization method of pile–rock interface roughness based on fractal geometry | |
| CN116681778B (en) | Distance measurement method based on monocular camera | |
| CN114800873B (en) | 5G data transmission-based intelligent detection method and system for gravel material level AI of concrete mixing plant | |
| CN117516487B (en) | Medium-small river video flow test method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |