CN113469154A - Method and system for monitoring unloading progress of muck truck based on artificial intelligence - Google Patents
Method and system for monitoring unloading progress of muck truck based on artificial intelligence Download PDFInfo
- Publication number
- CN113469154A CN113469154A CN202111033902.1A CN202111033902A CN113469154A CN 113469154 A CN113469154 A CN 113469154A CN 202111033902 A CN202111033902 A CN 202111033902A CN 113469154 A CN113469154 A CN 113469154A
- Authority
- CN
- China
- Prior art keywords
- muck
- unloading
- truck
- muck truck
- rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Abstract
The invention relates to the field of artificial intelligence, in particular to a method and a system for monitoring the unloading progress of a muck truck based on artificial intelligence. The method comprises the following steps: the method comprises the steps of acquiring a carriage lifting speed, a muck flatness change speed and a muck accumulation speed by collecting unloading images of a muck truck, acquiring a reminding signal of forward movement of the muck truck by combining the carriage lifting speed, the initial movement speed of the muck truck and the muck accumulation speed, adjusting the movement speed of the muck truck through the current muck flatness and the current movement speed after the muck truck starts to move, and acquiring an adjusting speed and a reminding signal of unloading completion by combining the muck accumulation speed. The invention realizes the real-time monitoring of the unloading progress of the muck truck, outputs and adjusts the movement rate of the muck truck and greatly saves the space of an unloading site.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a method and a system for monitoring the unloading progress of a muck truck based on artificial intelligence.
Background
In cities, a lot of construction projects exist, a large amount of muck garbage is generated every day, and for the treatment of the garbage, muck trucks are mainly used for transporting the garbage to muck treatment places at present.
The existing muck truck unloading process is as follows: the muck truck stops after running to a target position, a driver operates the carriage, the carriage starts to ascend, and muck begins to fall from the carriage and is accumulated on the ground. However, the maximum rising height of the carriage behind the muck truck is fixed, so that when the muck pile is piled to a certain height, the muck in the truck can not continuously fall off by means of inertia, and the ground muck pile can not change even if a driver continuously controls the carriage of the muck truck to move up and down. At the moment, the muck truck needs to be driven forwards by a driver and moved out of the open ground, muck in the muck truck can continuously fall off, and the muck truck carriage can be controlled to shake up and down before the driver drives the muck truck forwards or in the driving process.
After the muck truck is fully loaded, the weight of the muck truck is at least more than 20 tons, and when the muck truck is unloaded, a driver cannot directly know the unloading progress due to the large size and the limitation of an unloading field. In the prior art, the unloading process is mainly visually observed manually, the progress is timely fed back to a driver, the driver is reminded of carrying out corresponding operation, however, the unloading state of the muck truck cannot be timely and accurately informed to the driver due to delayed judgment caused by careless observation in the process of monitoring the unloading of the muck truck by a monitoring person, the unloading efficiency is reduced, and the space of an unloading site cannot be saved.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide an artificial intelligence-based muck truck unloading progress monitoring method and system, and the adopted technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides an artificial intelligence-based muck truck unloading progress monitoring method, including:
collecting continuous multiframe muck truck unloading images;
acquiring the carriage lifting rate of the carriage of the muck truck according to the lifting degree of the carriage of the muck truck in each frame of unloading image of the muck truck;
acquiring the muck flatness of each frame of muck car unloading image for reflecting the muck flatness according to the difference of muck pixel points in each frame of muck car unloading image, so as to obtain the muck flatness change rate;
acquiring the area of a muck area in each frame of the muck car unloading image according to each frame of the muck car unloading image, so as to obtain the change rate of the muck area;
obtaining a muck accumulation rate according to the muck flatness change rate and the muck area change rate;
the method comprises the steps of obtaining initial movement rate of a muck truck, outputting a movement reminding signal for reminding the muck truck to move forward and starting to detect the current movement rate and the current flatness of muck if the initial movement rate of the muck truck is 0, the absolute value of the carriage lifting rate is smaller than the preset rate and the muck area change rate is 0, inputting the current movement rate and the current flatness of muck truck into a preset muck truck movement rate adjusting network if the current movement rate of muck truck is not 0, and outputting the adjusted movement rate of muck truck.
Further, the step of obtaining the carriage lifting rate of the slag car carriage according to the lifting degree of the slag car carriage in each frame of the slag car unloading image comprises:
acquiring an included angle between a muck truck carriage in the muck truck unloading image and a horizontal line;
acquiring the lifting degree of the carriage of the muck truck according to the included angle;
and acquiring the carriage lifting rate of the carriage of the muck truck according to the change of the lifting degree of the carriage of the muck truck in the unloading images of the muck trucks of the adjacent frames.
Further, the obtaining of the included angle between the carriage of the muck truck in the muck truck unloading image and the horizontal line includes:
acquiring a straight line at the bottom of the muck truck carriage as a first straight line through the muck truck unloading image;
and taking an acute angle in an included angle between the first straight line and the horizontal line as an included angle between the carriage of the muck truck and the horizontal line.
Further, the step of obtaining the muck flatness change rate by obtaining the muck flatness degree of each frame of the muck car unloading image according to the difference of muck pixel points in each frame of the muck car unloading image, wherein the muck flatness degree is used for reflecting the muck flatness degree, comprises the following steps of:
acquiring a pixel point set of a muck area through the muck truck unloading image;
constructing a pixel coordinate system in the unloading image of the slag car by taking the horizontal direction in the unloading image of the slag car as the horizontal coordinate direction of the coordinate system and taking the vertical direction in the unloading image of the slag car as the vertical coordinate direction of the coordinate system;
determining a dividing line through the pixel coordinate system; the dividing line is a horizontal straight line passing through the target pixel point; the target pixel points are the pixel points with the smallest horizontal coordinate and the pixel points with the largest horizontal coordinate in the pixel point set;
acquiring the number of pixel points of each coordinate point on the partition line in the pixel point set above the partition line in the longitudinal coordinate direction;
obtaining the difference value of the number of pixel points corresponding to any two adjacent coordinate points on the dividing line to obtain a pixel point difference value sequence;
acquiring the flatness of the residue soil according to the variance of the difference sequence;
and acquiring the change rate of the flatness of the slag soil according to the difference value of the flatness of the slag soil in the unloading images of the slag soil truck of the adjacent frames.
Further, the obtaining the flatness of the muck according to the variance of the difference sequence includes:
and calculating the variance of the difference sequence, wherein the residue soil flatness is the reciprocal of the variance.
Further, the step of obtaining the area of the muck area in each frame of the muck car unloading image according to each frame of the muck car unloading image to obtain the change rate of the muck area comprises:
acquiring the area of the muck area according to the pixel point set;
and acquiring the change rate of the area of the muck area according to the difference value of the area of the muck truck unloading images of adjacent frames.
Further, obtaining the muck accumulation rate according to the muck flatness change rate and the muck area change rate comprises:
and weighting and summing the change rate of the slag soil flatness and the change rate of the slag soil area to obtain the slag soil accumulation rate.
Further, the process for acquiring the initial movement rate of the muck truck comprises the following steps:
acquiring the coordinates of key points of the muck truck in the muck truck unloading image;
and acquiring the initial motion rate of the muck truck according to the key point coordinates of the muck truck in the muck truck unloading image of the adjacent frame.
Further, after the adjusted movement rate of the muck truck is output, the muck truck unloading progress monitoring method further comprises the following steps:
and detecting the adjusted movement rate of the muck truck, the lifting rate of the carriage and the muck accumulation rate, and if the adjusted movement rate of the muck truck is not 0, the absolute value of the lifting rate of the carriage is less than the preset rate and the muck accumulation rate is 0, outputting a finish reminding signal for reminding the completion of unloading of the muck truck.
In a second aspect, an embodiment of the present invention further provides an artificial intelligence-based muck truck unloading progress monitoring system, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements any one of the steps of the artificial intelligence-based muck truck unloading progress monitoring method when executing the computer program.
The embodiment of the invention has the following beneficial effects:
1. according to the embodiment of the invention, the unloading progress is monitored in real time and fed back to the driver in real time through the lifting speed and the muck accumulation speed of the carriage of the muck truck and the movement speed of the muck truck, so that the driver is reminded to carry out corresponding operation, the unloading speed is ensured, and the potential safety hazard of manual monitoring is reduced.
2. According to the embodiment of the invention, the movement speed of the slag car is adjusted by analyzing the flatness of the ground slag, so that the space of a discharge site is greatly saved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a method for monitoring the unloading progress of a slag car based on artificial intelligence according to an embodiment of the present invention;
fig. 2 is a diagram of a lifting angle of a muck truck carriage in a muck truck unloading progress monitoring method based on artificial intelligence according to an embodiment of the present invention;
fig. 3 is a diagram of a muck pixel point set in a muck truck unloading progress monitoring method based on artificial intelligence according to an embodiment of the present invention.
Detailed Description
In order to further illustrate the technical means and effects of the present invention adopted to achieve the predetermined invention purpose, the following detailed description will be given to the specific implementation, structure, features and effects of the method and system for monitoring the unloading progress of the muck truck based on artificial intelligence according to the present invention with reference to the accompanying drawings and the preferred embodiments. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The embodiment of the invention is suitable for a specific scene that the real-time monitoring of the unloading progress is carried out in a fixed unloading area of the muck truck by installing the camera at the front side of the unloading point. The method comprises the steps of acquiring images of the muck car and the ground accumulated muck in the unloading area, and obtaining segmentation images of the muck car and the muck area through a semantic segmentation network, wherein the segmentation images comprise the images of the muck car, a muck car carriage and the ground accumulated muck. And acquiring the lifting speed of the carriage and the muck accumulation speed through the segmentation image, and acquiring real-time unloading progress and unloading completion signals by combining the muck flatness change speed and the regulated muck truck movement speed.
When carrying out the dregs and unloading in dregs processing field, the driver control vaulting pole rises, and dregs drop in the carriage, when ground dropped dregs and piled up a take the altitude, the driver control dregs car moves forward, and dregs continue to drop in the carriage, and the completion is unloaded until finally, through camera real time monitoring dregs car discharge progress.
The concrete scheme of the method and the system for monitoring the unloading progress of the muck truck based on artificial intelligence is described in detail below with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of a method for monitoring unloading progress of a slag car based on artificial intelligence according to an embodiment of the present invention is shown, where the method includes:
step S1: collecting continuous multi-frame unloading images of the muck truck.
The camera is an RGB camera and can be a wide-angle camera, the camera is installed at an unloading point of the muck processing field, the camera is located on the right side of the unloading point, and an imaging surface of the camera is kept parallel to the unloading direction of the camera, namely the camera faces the side face of the muck truck. Because the unloading point position of the muck truck is fixed, the unloading direction is fixed, when the camera imaging plane is parallel to the unloading direction, the complete unloading process and the walking process of the muck truck can be acquired, and the camera position of the embodiment is positioned on the right side surface of the muck truck.
Shooting a plurality of frames of unloading images of the muck truck through a camera, wherein the specific number of the unloading images of the muck truck is determined according to actual needs. The unloading image of the slag car comprises the slag car, unloaded ground slag and other components. It should be understood that the capture by the camera, i.e. the acquisition of images, is not started until the muck truck has driven to the unloading point and stopped.
The images for unloading the muck truck can be segmented by a semantic segmentation network to obtain segmented images of the carriage of the muck truck and the ground muck, so that the analysis of different areas can be performed conveniently.
The method for acquiring the segmented image comprises the following steps: the method comprises the steps of sending muck truck unloading images collected by a camera into a trained semantic segmentation network to obtain pixel-level classification of different regions, wherein the semantic segmentation network is an end-to-end Encoder-Decoder structure, performing convolution operation through an Encoder to extract features, outputting results of the Encoder as a feature map, and operating the feature map through a Decoder to obtain a semantic segmentation image.
The training content of the semantic segmentation network is as follows:
1) and selecting the collected images for unloading the muck truck, which comprise the carriage of the muck truck, the ground muck and other types of images, as a training data set, wherein 80% of the data set is randomly selected as the training set, and the rest 20% of the data set is selected as a verification set.
2) And marking the data set, wherein the marked carriage is 1, the ground muck is 2, and the other classes are 0.
3) The loss function is trained using a cross entropy loss function.
4) Obtaining a semantically segmented image
。
Step S2: and acquiring the carriage lifting rate of the carriage of the muck truck according to the lifting degree of the carriage of the muck truck in the unloading image of each frame of muck truck.
Referring to fig. 2, a diagram of a lifting angle of a muck truck carriage in the method for monitoring a discharging progress of a muck truck based on artificial intelligence according to an embodiment of the present invention is shown.
Analyzing the connected domain of the divided muck truck carriage to obtain a carriage connected domain, establishing a coordinate system according to the muck truck unloading image, taking a certain point in the divided image, namely the muck truck unloading image, as a coordinate origin, for example, taking a point with the minimum horizontal coordinate and the minimum vertical coordinate, namely a point at the lower left corner of the image as the origin, and taking the horizontal direction as the horizontal direction
An axis in the vertical direction of
A shaft. As shown in fig. 2, in the cabin connectivity domain is obtainedThe point with the maximum abscissa and the point with the minimum ordinate are taken as a straight line, namely, the straight line at the bottom of the carriage of the muck truck is the first straight line of the carriage communication domain, and the acute angle in the included angle between the first straight line and the horizontal line, namely the rising angle of the carriage is obtained
And the degree of rise of the car
：
Wherein the content of the first and second substances,
the maximum lifting angle of the carriage.
Obtaining car lifting rate from car lifting degree change between continuous frames
：
Wherein the content of the first and second substances,
is composed of
The rate of lifting of the car at the moment,
is composed of
The degree of ascent of the vehicle compartment at that time,
is composed of
The degree of rise of the car at that time. Each image acquisition time corresponds to a camera frame one to one.
Step S3: and acquiring the muck flatness of each frame of muck car unloading image for reflecting the muck flatness according to the muck pixel point difference in each frame of muck car unloading image so as to obtain the muck flatness change rate.
Referring to fig. 3, a slag soil pixel point set diagram in the method for monitoring the unloading progress of the slag soil truck based on artificial intelligence according to an embodiment of the present invention is shown.
The method for acquiring the flatness of the residue soil comprises the following steps: the pixel point set of the muck area is obtained through the unloading image of the muck truck, namely the image of the muck area can be obtained through the image segmentation algorithm, and the pixel point set of the muck area is obtained. The horizontal direction in the unloading image of the muck truck is taken as the horizontal coordinate direction of the coordinate system, the vertical direction in the unloading image of the muck truck is taken as the vertical coordinate direction of the coordinate system, a set point in the unloading image of the muck truck is taken as an original point, a pixel coordinate system is constructed in the unloading image of the muck truck, and the pixel points can be collectively expressed in the pixel coordinate system by adopting the coordinate system established in the above.
Acquiring a pixel point with the minimum horizontal coordinate and a pixel point with the maximum vertical coordinate in the pixel point set as target pixel points, such as: if two pixel points with the minimum horizontal coordinates and three pixel points with the maximum horizontal coordinates exist in the pixel point set, the pixel point with the maximum vertical coordinate in the five pixel points is found out and used as a target pixel point, and if a plurality of pixel points with the maximum vertical coordinate exist, any one pixel point can be selected. Making a horizontal straight line passing through the target pixel as a partition line, and acquiring a pixel number sequence of each coordinate point on the partition line in the pixel set above the partition line in the vertical coordinate direction
：
Wherein the content of the first and second substances,
as a coordinate point
The quantity of the residue soil pixel points above the partition line in the vertical coordinate direction,
as a coordinate point
The quantity of the residue soil pixel points above the partition line in the vertical coordinate direction,
as a coordinate point
The quantity of the residue soil pixel points above the partition line in the vertical coordinate direction,
the coordinate point with the minimum abscissa in the pixel point set on the dividing line,
the coordinate point with the maximum abscissa in the pixel point set on the separation line is obtained. The more stable the values in the sequence, the flatter the shape of the ground muck. Calculating the difference between adjacent values in the sequence:
wherein the content of the first and second substances,
is a sequence of
Middle coordinate point
And
the difference value of the number of the corresponding pixel points,
is an absolute value in the mathematical sense.
Obtaining each difference value, constructing difference value sequence, and using variance of said difference value sequence
The reciprocal of (a) is taken as the flatness of the ground residue soil
：
The higher the flatness is, the more flat the ground dregs are.
Calculating the difference of the slag flatness between the continuous frames to obtain the change rate of the slag flatness
：
Wherein the content of the first and second substances,
is composed of
Speed of change of soil residue flatness at any momentThe ratio of the total weight of the particles,
is composed of
The flatness of the residue soil at any moment,
is composed of
And (4) the flatness of the residue soil at any moment. A smaller rate of change in the slag soil flatness indicates a smaller change in the ground slag soil of the adjacent frame.
Step S4: and acquiring the area of the muck area in the unloading image of each frame of muck truck according to the unloading image of each frame of muck truck so as to obtain the change rate of the muck area.
Obtaining the number of pixel points and coordinate points according to the pixel point set graph in the text, and calculating the area of the muck area
：
Wherein the content of the first and second substances,
as a coordinate point
The quantity of the muck pixel points in the longitudinal coordinate direction, namely the coordinate points in the pixel point set
The number of pixels above the dividing line and the number of pixels below the dividing line in the ordinate direction,
and
are adjacent coordinate points on the segmentation line.
As another implementation manner, the number of pixels in the pixel set may be obtained in another manner, and the number of pixels is the area of the muck area.
Obtaining a rate of change of a muck area based on the area of the muck area between successive frames
：
Wherein the content of the first and second substances,
is composed of
The change rate of the area of the residue soil at the moment,
is composed of
The area of the muck area at the moment,
is composed of
Area of the soil area at the moment.
Step S5: and obtaining the muck accumulation rate according to the muck flatness change rate and the muck area change rate.
The change rate of the flatness of the muck
Area change speed of mixed muckRate of change
Obtaining the muck accumulation rate by weighted summation
：
Wherein the content of the first and second substances,
is composed of
The ground residue soil accumulation rate at the moment,
is the weight coefficient of the change rate of the flatness of the dregs,
is the weight coefficient of the change rate of the area of the residue soil.
In the embodiment of the invention, the weight coefficient of the change rate of the flatness of the muck
And the weight coefficient of the change rate of the area of the residue soil
The values are all 0.5, and in other embodiments, the weight coefficient values can be adjusted according to requirements.
Step S6: the method comprises the steps of obtaining an initial movement speed of a muck truck, outputting a movement reminding signal for reminding the muck truck to move forward and starting to detect the current movement speed and the current flatness of muck if the initial movement speed of the muck truck is 0, the absolute value of the carriage lifting speed is less than a preset speed and the area change speed of muck is 0, and inputting the current movement speed and the current flatness of muck into a preset muck truck movement speed adjusting network and outputting the adjusted movement speed of the muck truck if the current movement speed of the muck truck is not 0.
The method for acquiring the initial movement rate of the muck truck comprises the following steps: key point coordinates are detected by carrying out key point detection on segmentation images of the muck truck
The key point coordinates can be coordinates of the center point of the head of the muck truck, and the initial movement rate of the muck truck is obtained according to the key point coordinates of adjacent frames
：
Wherein the content of the first and second substances,
is composed of
The initial movement rate of the muck car at the moment,
is composed of
The coordinates of the key points of the time of day,
is composed of
The key point coordinates of the moment.
It should be appreciated that the rate of movement of the muck vehicle may also be obtained through communicative interaction with the muck vehicle.
After the muck truck reaches the designated unloading area, the camera starts to acquire images, and the camera can be started and controlled by special personnel. Driver control vaulting pole rises this moment, dregs car carriage constantly rises, dregs begin to drop because of gravity and inertia, ground dregs area begins to change, along with constantly dropping of dregs, ground dregs increase gradually, ground dregs heap is bigger and bigger, nevertheless because dregs car carriage is highly certain, when dregs heap is piled up the take the altitude, dregs can't continue to rely on inertia to drop in the car, even the driver continues to control dregs car carriage up-and-down motion, ground dregs heap also can not change, dregs area change rate is 0 promptly. At the moment, the muck truck is reminded to move forwards, so that the residual muck in the carriage continuously falls.
The sending condition of the reminding signal is as follows: when the initial movement rate of the muck truck
And the absolute value of the carriage lifting speed is less than the preset speed and the change speed of the muck area is 0, namely when the muck truck does not move, indicating that the muck accumulated on the ground is not increased any more and the muck in the truck can not continuously drop by means of inertia, outputting a movement reminding signal for reminding the muck truck to move forwards, for example, outputting the movement reminding signal through a loudspeaker to ensure that a driver can hear the movement reminding signal. After hearing the movement reminding signal, the driver operates the muck truck to move forward. The absolute value of the lifting speed of the carriage is smaller than the preset speed so as to ensure that the muck all normally falls to the unloading area and prevent the muck from being lifted to other areas.
After the movement reminding signal is output, the current movement speed and the current flatness of the muck truck are detected, the current movement speed and the current flatness of the muck truck are input into a preset muck truck movement speed adjusting network, and the adjusted movement speed of the muck truck is output, so that the shape of the muck piled on the ground is smoother, and the space is saved. The movement rate can be adjusted only once, or a reminding signal can be sent every two seconds, the output movement rate signal is informed to a muck truck driver, for example, the adjusted muck truck movement rate is output through a loudspeaker, and the muck truck driver is reminded to manually adjust the muck truck movement rate.
The muck truck motion rate adjusting network comprises a plurality of groups of corresponding relations of current motion rates, the current muck flatness and the adjusted muck truck motion rates, the current motion rates and the current muck flatness of the muck trucks are input into the muck truck motion rate adjusting network, and the adjusted muck truck motion rates corresponding to the current motion rates and the current muck flatness of the muck trucks are obtained.
The training standard of the muck truck movement rate adjusting network is as follows: the smaller the flatness of the ground muck area is, the more uneven the muck area is, and the forward movement speed of the muck vehicle is reduced at the moment so that the ground muck tends to be flat.
Step S7: and outputting the adjusted movement rate of the muck truck and then further monitoring the unloading progress.
And after outputting the adjusted movement rate of the muck truck, controlling the muck truck to continue to move forwards according to the adjusted movement rate of the muck truck by a driver. In the forward movement process of the muck truck, detecting the adjusted movement rate of the muck truck, the lifting rate of the carriage and the muck accumulation rate, if the adjusted movement rate of the muck truck is not 0, the absolute value of the lifting rate of the carriage is less than the preset rate and the muck accumulation rate is 0, indicating that the muck truck finishes unloading, outputting a finish reminding signal for reminding the completion of unloading of the muck truck, and enabling a driver to perform other subsequent related operations after acquiring the finish reminding signal, such as reducing the carriage to be horizontal and leaving an unloading area.
In summary, in the embodiment of the invention, the unloading image of the muck truck is collected, the lifting speed of the carriage, the change speed of the muck flatness and the muck accumulation speed are obtained, the reminding signal of the forward movement of the muck truck is obtained by combining the lifting speed of the carriage, the initial movement speed of the muck truck and the muck accumulation speed, the movement speed of the muck truck is adjusted by the current muck flatness and the current movement speed when the muck truck starts to move, the flatness of the muck on the ground is ensured, the unloading space is saved, and the reminding signal of the adjustment speed and the unloading completion is obtained by combining the muck accumulation speed, so that the unloading efficiency is improved.
The embodiment of the invention also provides an artificial intelligence-based muck truck unloading progress monitoring system which comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the steps of the artificial intelligence-based muck truck unloading progress monitoring method are realized when the processor executes the computer program.
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. An artificial intelligence-based muck truck unloading progress monitoring method is characterized by comprising the following steps:
collecting continuous multiframe muck truck unloading images;
acquiring the carriage lifting rate of the carriage of the muck truck according to the lifting degree of the carriage of the muck truck in each frame of unloading image of the muck truck;
acquiring the muck flatness of each frame of muck car unloading image for reflecting the muck flatness according to the difference of muck pixel points in each frame of muck car unloading image, so as to obtain the muck flatness change rate;
acquiring the area of a muck area in each frame of the muck car unloading image according to each frame of the muck car unloading image, so as to obtain the change rate of the muck area;
obtaining a muck accumulation rate according to the muck flatness change rate and the muck area change rate;
the method comprises the steps of obtaining initial movement rate of a muck truck, outputting a movement reminding signal for reminding the muck truck to move forward and starting to detect the current movement rate and the current flatness of muck if the initial movement rate of the muck truck is 0, the absolute value of the carriage lifting rate is smaller than the preset rate and the muck area change rate is 0, inputting the current movement rate and the current flatness of muck truck into a preset muck truck movement rate adjusting network if the current movement rate of muck truck is not 0, and outputting the adjusted movement rate of muck truck.
2. The method for monitoring the unloading progress of the slag car based on artificial intelligence as claimed in claim 1, wherein the step of obtaining the car lifting rate of the slag car according to the lifting degree of the slag car in each frame of the unloading image of the slag car comprises:
acquiring an included angle between a muck truck carriage in the muck truck unloading image and a horizontal line;
acquiring the lifting degree of the carriage of the muck truck according to the included angle;
and acquiring the carriage lifting rate of the carriage of the muck truck according to the change of the lifting degree of the carriage of the muck truck in the unloading images of the muck trucks of the adjacent frames.
3. The method for monitoring the unloading progress of the muck truck based on artificial intelligence as claimed in claim 2, wherein the step of obtaining the included angle between the carriage of the muck truck in the unloading image of the muck truck and the horizontal line comprises the following steps:
acquiring a straight line at the bottom of the muck truck carriage as a first straight line through the muck truck unloading image;
and taking an acute angle in an included angle between the first straight line and the horizontal line as an included angle between the carriage of the muck truck and the horizontal line.
4. The method for monitoring the unloading progress of the muck truck based on artificial intelligence as claimed in claim 1, wherein the step of obtaining the muck flatness change rate by obtaining the muck flatness degree of each frame of the muck truck unloading image according to the difference of muck pixel points in each frame of the muck truck unloading image, the muck flatness degree being used for reflecting the muck flatness degree, comprises:
acquiring a pixel point set of a muck area through the muck truck unloading image;
constructing a pixel coordinate system in the unloading image of the slag car by taking the horizontal direction in the unloading image of the slag car as the horizontal coordinate direction of the coordinate system and taking the vertical direction in the unloading image of the slag car as the vertical coordinate direction of the coordinate system;
determining a dividing line through the pixel coordinate system; the dividing line is a horizontal straight line passing through the target pixel point; the target pixel points are the pixel points with the smallest horizontal coordinate and the pixel points with the largest horizontal coordinate in the pixel point set;
acquiring the number of pixel points of each coordinate point on the partition line in the pixel point set above the partition line in the longitudinal coordinate direction;
obtaining the difference value of the number of pixel points corresponding to any two adjacent coordinate points on the dividing line to obtain a pixel point difference value sequence;
acquiring the flatness of the residue soil according to the variance of the difference sequence;
and acquiring the change rate of the flatness of the slag soil according to the difference value of the flatness of the slag soil in the unloading images of the slag soil truck of the adjacent frames.
5. The method for monitoring the unloading progress of the muck truck based on artificial intelligence as claimed in claim 4, wherein the obtaining the flatness of the muck according to the variance of the difference sequence comprises:
and calculating the variance of the difference sequence, wherein the residue soil flatness is the reciprocal of the variance.
6. The method for monitoring the unloading progress of the muck truck based on artificial intelligence as claimed in claim 1, wherein the step of obtaining the area of the muck area in each frame of the unloading image of the muck truck according to each frame of the unloading image of the muck truck so as to obtain the change rate of the area of the muck includes:
acquiring the area of the muck area according to the pixel point set;
and acquiring the change rate of the area of the muck area according to the difference value of the area of the muck truck unloading images of adjacent frames.
7. The method for monitoring the unloading progress of the muck truck based on artificial intelligence as claimed in claim 1, wherein the obtaining of the muck accumulation rate according to the muck flatness change rate and the muck area change rate comprises:
and weighting and summing the change rate of the slag soil flatness and the change rate of the slag soil area to obtain the slag soil accumulation rate.
8. The method for monitoring the unloading progress of the slag car based on the artificial intelligence as claimed in claim 1, wherein the process of obtaining the initial movement rate of the slag car comprises:
acquiring the coordinates of key points of the muck truck in the muck truck unloading image;
and acquiring the initial motion rate of the muck truck according to the key point coordinates of the muck truck in the muck truck unloading image of the adjacent frame.
9. The method for monitoring the unloading progress of the muck truck based on artificial intelligence as claimed in claim 1, wherein after the adjusted movement rate of the muck truck is output, the method for monitoring the unloading progress of the muck truck further comprises:
and detecting the adjusted movement rate of the muck truck, the lifting rate of the carriage and the muck accumulation rate, and if the adjusted movement rate of the muck truck is not 0, the absolute value of the lifting rate of the carriage is less than the preset rate and the muck accumulation rate is 0, outputting a finish reminding signal for reminding the completion of unloading of the muck truck.
10. An artificial intelligence based muck truck discharge progress monitoring system comprising a memory, a processor and a computer program stored in said memory and executable on said processor, wherein said processor when executing said computer program performs the steps of the method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111033902.1A CN113469154B (en) | 2021-09-03 | 2021-09-03 | Method and system for monitoring unloading progress of muck truck based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111033902.1A CN113469154B (en) | 2021-09-03 | 2021-09-03 | Method and system for monitoring unloading progress of muck truck based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113469154A true CN113469154A (en) | 2021-10-01 |
| CN113469154B CN113469154B (en) | 2021-11-16 |
Family
ID=77868021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111033902.1A Active CN113469154B (en) | 2021-09-03 | 2021-09-03 | Method and system for monitoring unloading progress of muck truck based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113469154B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105487520A (en) * | 2016-01-18 | 2016-04-13 | 福建航天星联信息科技有限公司 | System and method for monitoring illegal discharging of muck vehicles |
| CN106952479A (en) * | 2017-05-18 | 2017-07-14 | 南京云趟信息技术有限公司 | A kind of video frequency identifying method for monitoring of being unloaded for slag-soil truck |
| CN107071368A (en) * | 2017-05-18 | 2017-08-18 | 南京云计趟信息技术有限公司 | A kind of video monitoring method that unloading is lifted for slag-soil truck |
| CN206741700U (en) * | 2017-05-18 | 2017-12-12 | 南京云计趟信息技术有限公司 | A kind of video recognition system for the monitoring of slag-soil truck container |
| CN207650394U (en) * | 2017-09-13 | 2018-07-24 | 南京云计趟信息技术有限公司 | A kind of dregs intelligent analysis system based on TOF technologies |
| CN109002421A (en) * | 2018-06-25 | 2018-12-14 | 深圳市四叶草物联网科技有限公司 | Slag-soil truck transports yardage statistical method, system and server, storage medium |
| CN110766039A (en) * | 2019-09-02 | 2020-02-07 | 厦门卫星定位应用股份有限公司 | Muck truck transportation state identification method, medium, equipment and muck truck |
| CN110929670A (en) * | 2019-12-02 | 2020-03-27 | 合肥城市云数据中心股份有限公司 | Muck truck cleanliness video identification and analysis method based on yolo3 technology |
| CN112073480A (en) * | 2020-08-26 | 2020-12-11 | 湖南金锐科技有限公司 | Method and device for monitoring consumption of muck vehicle through neural network algorithm self-organizing mapping |
-
2021
- 2021-09-03 CN CN202111033902.1A patent/CN113469154B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105487520A (en) * | 2016-01-18 | 2016-04-13 | 福建航天星联信息科技有限公司 | System and method for monitoring illegal discharging of muck vehicles |
| CN106952479A (en) * | 2017-05-18 | 2017-07-14 | 南京云趟信息技术有限公司 | A kind of video frequency identifying method for monitoring of being unloaded for slag-soil truck |
| CN107071368A (en) * | 2017-05-18 | 2017-08-18 | 南京云计趟信息技术有限公司 | A kind of video monitoring method that unloading is lifted for slag-soil truck |
| CN206741700U (en) * | 2017-05-18 | 2017-12-12 | 南京云计趟信息技术有限公司 | A kind of video recognition system for the monitoring of slag-soil truck container |
| CN207650394U (en) * | 2017-09-13 | 2018-07-24 | 南京云计趟信息技术有限公司 | A kind of dregs intelligent analysis system based on TOF technologies |
| CN109002421A (en) * | 2018-06-25 | 2018-12-14 | 深圳市四叶草物联网科技有限公司 | Slag-soil truck transports yardage statistical method, system and server, storage medium |
| CN110766039A (en) * | 2019-09-02 | 2020-02-07 | 厦门卫星定位应用股份有限公司 | Muck truck transportation state identification method, medium, equipment and muck truck |
| CN110929670A (en) * | 2019-12-02 | 2020-03-27 | 合肥城市云数据中心股份有限公司 | Muck truck cleanliness video identification and analysis method based on yolo3 technology |
| CN112073480A (en) * | 2020-08-26 | 2020-12-11 | 湖南金锐科技有限公司 | Method and device for monitoring consumption of muck vehicle through neural network algorithm self-organizing mapping |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113469154B (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4389999B2 (en) | Exposure control device and exposure control program | |
| WO2013135033A1 (en) | Tunnel deformation online monitoring system based on image analysis and application thereof | |
| CN112191355B (en) | Ore processing method, device, electronic equipment and computer readable storage medium | |
| US7957633B2 (en) | Focus adjusting apparatus and focus adjusting method | |
| JP2018071994A (en) | Attached matter detection apparatus and attached matter detection method | |
| CN108174170B (en) | Camera-based projection area size self-detection method, system and equipment | |
| CN102663696A (en) | Denoising method of enlarged image and system thereof | |
| CN102446352A (en) | Video image processing method and device | |
| CN103186887A (en) | Image demisting device and image demisting method | |
| CN103327219A (en) | Video image contrast adjustment method and device | |
| CN106829393A (en) | Ribbon conveyer automatic speed-regulating method | |
| CN106373139A (en) | Image processing method and device | |
| CN107767399A (en) | Fork truck monitoring method and monitoring system | |
| CN113469154B (en) | Method and system for monitoring unloading progress of muck truck based on artificial intelligence | |
| DE112016002564T5 (en) | IMAGE PROCESSING DEVICE, IMAGE PROCESSING AND PROGRAM | |
| JP6265602B2 (en) | Surveillance camera system, imaging apparatus, and imaging method | |
| CN116778348B (en) | Machine vision-based method, device, equipment and medium for monitoring falling off of outer wall | |
| CN107979729B (en) | Method and equipment for displaying preview image | |
| US20230005239A1 (en) | Image capturing method and device, apparatus, and storage medium | |
| CN110407045B (en) | Method for displaying personnel distribution information in elevator and intelligent elevator system | |
| CN108830834B (en) | Automatic extraction method for video defect information of cable climbing robot | |
| KR20240019361A (en) | Methods, measuring devices, welding systems, and programs for measuring the behavior of welding phenomena | |
| JP4680087B2 (en) | Image transmission device | |
| WO2020077936A1 (en) | Method for calculating exposure evaluation value and imaging device | |
| CN110796610A (en) | Image defogging method, device and equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220309 Address after: 226100 No. 38 Zhonghua East Road, three Factory Street, Haimen City, Nantong, Jiangsu Patentee after: Nantong an art design Co.,Ltd. Address before: 226100 No. 10, Changhong Road, Sanchang Town, Nantong City, Jiangsu Province Patentee before: Haimen heavy mining machinery factory |