CN114882400A - Aggregate detection and classification method based on AI intelligent machine vision technology - Google Patents

Aggregate detection and classification method based on AI intelligent machine vision technology Download PDF

Info

Publication number
CN114882400A
CN114882400A CN202210461094.7A CN202210461094A CN114882400A CN 114882400 A CN114882400 A CN 114882400A CN 202210461094 A CN202210461094 A CN 202210461094A CN 114882400 A CN114882400 A CN 114882400A
Authority
CN
China
Prior art keywords
image
aggregate
detection
machine vision
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210461094.7A
Other languages
Chinese (zh)
Inventor
袁胜
杜钰莹
上官林建
黄伟
郭灿波
马军旭
刘明堂
胡旭峰
汪璐
岳爽
李斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Henan Sanhe Hydraulic Machinery Group Co ltd
Henan Sanhe Hydraulic New Building Material Machinery Co ltd
Zhengzhou Sanhe Hydraulic Machinery Co ltd
North China University of Water Resources and Electric Power
Original Assignee
Henan Sanhe Hydraulic Machinery Group Co ltd
Henan Sanhe Hydraulic New Building Material Machinery Co ltd
Zhengzhou Sanhe Hydraulic Machinery Co ltd
North China University of Water Resources and Electric Power
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Henan Sanhe Hydraulic Machinery Group Co ltd, Henan Sanhe Hydraulic New Building Material Machinery Co ltd, Zhengzhou Sanhe Hydraulic Machinery Co ltd, North China University of Water Resources and Electric Power filed Critical Henan Sanhe Hydraulic Machinery Group Co ltd
Priority to CN202210461094.7A priority Critical patent/CN114882400A/en
Publication of CN114882400A publication Critical patent/CN114882400A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details

Abstract

The invention discloses an aggregate detection and classification method based on an AI intelligent machine vision technology, which comprises the following steps: s1, shooting an aggregate transport video by using a high-definition camera, and transmitting the aggregate transport video in real time through a 5G network to obtain an aggregate image; s2, processing the aggregate image by an image edge segmentation processing method, and extracting edge features; s3, manually labeling the aggregate type of the processed aggregate image; s4, inputting the labeled aggregate image into a machine vision algorithm, training, extracting features, and outputting as a weight file; and S5, replaying the trained weight in the detection system, and detecting and identifying the aggregate. Aggregate images are acquired in real time through a 5G network by using a camera, videos are processed by using a computer image processing technology, and state images of different types of aggregates are detected and identified by combining an AI intelligent algorithm, so that the problem that the existing aggregate monitoring method is complex in operation is solved, and meanwhile, the function of real-time processing is realized.

Description

Aggregate detection and classification method based on AI intelligent machine vision technology
Technical Field
The invention belongs to the technical field of aggregate detection, and particularly relates to an aggregate detection and classification method based on an AI intelligent machine vision technology.
Background
Aggregate is an important component of concrete, the classification problem of sand aggregate is an important factor for determining the performance and quality of the concrete, common concrete (concrete for short) is composed of cement, sand, stone and water, and the aggregate generally accounts for 70-80% of the concrete. The factors influencing the concrete strength are many, and mainly influence the cement strength and the water-cement ratio, influence of aggregate gradation and shape, influence of curing temperature, influence of curing period and the like. The aggregate processing system is one of main auxiliary production systems in large-scale hydraulic and hydroelectric engineering construction, and the quality improvement of the sandstone aggregate has important significance for promoting the benign development of the engineering construction industry and is also extremely important for improving the engineering quality and optimizing the engineering cost. Different sand aggregates have different effects on the performance of the concrete. For the particle size shape of the aggregate, the regulations of the coarse aggregate needle flake particles in the current specifications are wide, and the aggregate with good quality needs to have the specified particle size shape. Therefore, the quality requirement of the sandstone aggregate must be guaranteed, and the raw materials are reasonably selected, so that the quality of the concrete can be guaranteed. Therefore, finding a proper aggregate classification detection method is particularly important.
The sand aggregate classification problem is an important working premise for solving the concrete raw material proportioning, and a plurality of image processing related technical problems are involved in the working preparation and operation. Aggregate types in concrete raw materials are various, detection conditions are complex, detection distances are inconsistent, and weather reasons, light changes, humidity changes, distance changes and the like all influence the aggregate detection effect. The searching of the aggregate characteristics is an important problem, and a detection method suitable for identifying the aggregate category needs to be searched. The traditional aggregate detection methods generally comprise a screening method, particle size identification and the like, but the methods have limitations on image identification and real-time processing, are time-consuming and labor-consuming, are difficult to solve practical problems, do not have the requirements on rapidness, accuracy, real-time processing and the like, and cannot be popularized and applied comprehensively. In recent years, aggregate detection and identification technologies have developed a profile with modern technology, such as application of convolutional neural networks, imaging, use of back-light and high-resolution camera measurements, and laser-based detection and analysis. Since each method and technology has its own application conditions and limitations, any single method is not universal, and sometimes it may be difficult to obtain the expected effect if only one kind of parameters is taken as the basis for explanation in practice, so in practice, a comprehensive aggregate detection method is generally adopted to improve the accuracy of aggregate detection.
With the development of science and technology, the aggregate detection method and the detection equipment thereof make great progress and development, and the technical level of aggregate detection is continuously improved. However, various detection methods have application premises and limitations of the detection methods, and in practical application, most of the existing methods only modernize instruments, and the methods are not innovative, so that a single method is adopted in aggregate detection, and a good effect is sometimes difficult to obtain. Therefore, in the application of aggregate detection, a technician should select a reasonable aggregate detection combination method according to the combination of the aggregate image characteristics and the machine vision technology.
At present, no special real-time monitoring design scheme aiming at aggregate detection exists in concrete manufacturing engineering. In engineering construction, high-definition industrial monitoring equipment is generally installed, and the position and the unloading state of an aggregate transport vehicle can be monitored. However, the existing monitoring equipment is generally distributed at the upper end of a transport vehicle, can only monitor and cannot identify and process images; in addition, after the monitoring data is obtained, only the human eyes of workers are needed to judge the aggregate category, and the aggregate category is not processed and analyzed more effectively.
Based on the defects in the prior art, the invention provides an aggregate detection and classification method based on an AI intelligent machine vision technology, which is characterized in that an aggregate image is obtained in real time through a 5G network by using a camera, then a video is processed by applying a computer image processing technology, and different types of aggregate state images are detected and identified by combining an AI intelligent algorithm, so that the problem of complex operation of the existing aggregate monitoring method is solved, and the function of real-time processing is realized.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide an aggregate detection and classification method based on an AI intelligent machine vision technology.
The invention provides the following technical scheme:
an aggregate detection and classification method based on an AI intelligent machine vision technology comprises the following steps:
s1, shooting an aggregate transport video by using a high-definition camera, and transmitting the aggregate transport video in real time through a 5G network to obtain an aggregate image;
s2, processing the aggregate image by an image edge segmentation processing method, and extracting edge features;
s3, manually labeling the aggregate type of the processed aggregate image;
s4, inputting the labeled aggregate image into a machine vision algorithm, training, extracting features, and outputting as a weight file;
and S5, replaying the trained weight in the detection system, and detecting and identifying the aggregate.
Preferably, in step S2, the image edge segmentation processing method specifically includes the following steps:
a, introducing an aggregate image, carrying out gray processing on the aggregate image, and then solving a gradient map;
b, processing the image by using a watershed algorithm on the basis of the gradient image to obtain edge lines of the segmented image;
c, removing small objects in the picture and burr interference influencing the target by using image opening operation;
and D, performing adhesion segmentation to segment the objects adhered together.
Preferably, in step S4, the machine vision algorithm training process specifically includes the following steps:
a, selecting 5 detection areas for each marked aggregate image;
b, performing image convolution on each detection area respectively, and extracting image features;
c, performing up-sampling processing on the image, and restoring the feature map after convolution into the image;
d, integrating image features by using a tensor splicing algorithm;
e, outputting the classification result to a detection head, wherein the detection head is used for classifying the image and judging the classification result of the identified image through a probability function;
and f, identifying the result and outputting the detection result of the image.
Preferably, in step a, the 5 detection regions are a central point, an upper left region, an upper right region, a lower left region and a lower right region of the aggregate image.
Preferably, in step b, the image features include edge features, color features and texture features of the aggregate.
Preferably, in step c, the image is subjected to an upsampling process, so that the image conforms to the size of the display area, and an interpolation method is adopted, that is, a suitable interpolation algorithm is adopted to insert new elements between pixel points on the basis of the original image pixels.
Preferably, in step C, the gradient image is thresholded by a processing method that modifies the gradient function so that the water collection basin only responds to the target to be detected, so as to reduce excessive segmentation caused by the change of gray scale.
Preferably, the method further comprises the following steps: and S6, testing the new image, judging whether the sample size is sufficient or not according to the test condition, keeping the sample size as uniform as possible, if the accuracy of the weight to a certain class is lower than 90%, increasing the corresponding sample size, and re-extracting the features until the recognition rate of each class of aggregates reaches more than 90%, so that the requirements are met.
Preferably, in step B, the watershed computation process is divided into two steps, one is a ranking process and one is a flooding process. Firstly, the gray levels of each pixel are sequenced from low to high, and then a first-in first-out (FIFO) structure is adopted to judge and mark each local minimum value in an influence domain of h-order height in the process of realizing inundation from low to high. The watershed transform obtains a catchbasin image of the input image, and boundary points between catchbasins are watershed. Clearly, the watershed represents the input image maxima points. Therefore, to obtain edge information of an image, a gradient image is usually taken as an input image, i.e.
g(x,y)＝grad(f(x,y))＝{[f(x,y)-f(x-1,y)]×2f(x,y)-f(x,y-1)]×2}×0.5
In the formula, f (x, y) represents an original image, and grad represents gradient operation.
Preferably, in step C, to reduce the over-segmentation caused by the watershed algorithm, the gradient function is modified, and a simple method is to perform threshold processing on the gradient image to eliminate the over-segmentation caused by the slight change of the gray level. Namely, it is
g(x,y)＝max(grad(f(x,y)),gθ)
In the formula, g θ represents a threshold value.
The program adopts the method: and limiting the gradient image by using a threshold value to eliminate excessive segmentation caused by slight change of gray values, obtaining a proper amount of regions, sequencing the gray levels of edge points of the regions from low to high, and then calculating the gradient image by using a Sobel operator in the process of realizing inundation from low to high.
The operator comprises two sets of 3x3 matrices, horizontal and vertical, respectively, which are then subjected to planar convolution with the image to obtain horizontal and vertical luminance difference approximations. The formula is as follows:
Figure BDA0003620470160000061
Figure BDA0003620470160000062
the lateral and longitudinal gradient approximations for each pixel of the image may be combined using the following equations to calculate the magnitude of the gradient.
Figure BDA0003620470160000063
Wherein A represents an original image, G x And G y Representing the transverse and longitudinal edge detected images, respectively.
After preprocessing, edge information is obtained, and a minimum rectangular frame is selected.
Preferably, in step S4, using an algorithm will do the following:
classifying pictures: inputting the picture into a multilayer convolutional neural network, outputting a feature vector and feeding the feature vector back to a softmax unit to predict the type of the picture.
Positioning and classifying: judging whether a detection target exists in the graph by using an algorithm, marking the position of the target, marking by using a border (Bounding Box), and identifying and positioning the target.
Object detection: the picture can contain a plurality of objects, and a part of single picture also has a plurality of different classified objects to detect the plurality of objects.
Softmax function, normalized exponential function, is in fact a gradient log normalization of a finite discrete probability distribution. In multiple-term logistic regression and linear discriminant analysis, the input to the function is the result from K different linear functions, and the probability formula that the sample vector x belongs to the jth class is:
Figure BDA0003620470160000064
this can be viewed as a composite of the Softmax function of the K linear functions.
The positioning of the target is represented by an anchor Box, the input picture is divided into S-S grids, and each small grid can generate n anchor boxes; the real frame of the image and an anchor box generated by a small grid where the central point of the image is located are subjected to IOU calculation; the returned Box is a Bounding Box; when the anchors box is closer to the real width and height, the performance of the model is better; clustering the width, height and dimension cluster (dimension cluster) of a representative shape in the group route box of all samples in the training set by using a k-means algorithm; clustering a plurality of different anchor box groups, respectively applying the groups to the model, and finally finding out the optimal anchor box group which is a compromise between the complexity of the model and the high recall rate (high recall).
The Bounding Box formula is:
Figure BDA0003620470160000071
Figure BDA0003620470160000072
in the formula a w And a h The width and height of the anchor box,
t w and t h The width and height directly predicted for the bounding box,
b w and b h For the actual width and height of the prediction after conversion,
this is also the width and height of the output in the final prediction.
The loss function for object detection is as follows:
Figure BDA0003620470160000073
in the above formula, N is the total number of categories, yi is the probability of the current category obtained through the excitation function, yi is the probability of judging whether the prior frame is responsible for the target (0 or 1), if so, 0, otherwise, 1.
Compared with the prior art, the invention has the following beneficial effects:
(1) the invention relates to an aggregate detection and classification method based on an AI intelligent machine vision technology, which is characterized in that an aggregate image is obtained in real time through a 5G network by using a camera, then a computer image processing technology is applied to process a video, and detection and identification are carried out on different types of aggregate state images by combining an AI intelligent algorithm, so that the problem of complex operation of the existing aggregate monitoring method is solved, and meanwhile, the function of real-time processing is realized.
(2) The invention relates to an aggregate detection and classification method based on an AI intelligent machine vision technology, which extracts aggregate edge information, texture characteristics and aggregate particle size characteristics according to an acquired aggregate image, processes the image through an image processing technology, and preprocesses the image to make the characteristics of aggregates more obvious.
(3) According to the aggregate detection and classification method based on the AI intelligent machine vision technology, each acquired image is divided into 5 detection areas, so that one image can be judged for 5 times, and one-time identification and multiple judgments can be realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a data acquisition roadmap according to the present invention;
FIG. 2 is a flow chart of the intelligent aggregate detection and identification method of the present invention;
FIG. 3 is a schematic diagram of image processing and recognition according to the present invention;
FIG. 4 is a flow chart of the aggregate image edge segmentation of the present invention;
FIG. 5 is a schematic diagram of edge segmentation according to the present invention;
FIG. 6 is a schematic view of a machine vision model of the present invention;
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described in detail and completely with reference to the accompanying drawings. It is to be understood that the described embodiments are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
Example 1
Referring to fig. 1-6, the detection objects of the present invention are five types of aggregates, namely, pebbles, small pebbles, machine-made sand, small machine-made sand and surface sand, wherein the particle size of the pebbles is in the range of 3-4 cm; the particle size of the small stones is within the range of 2-3 cm; the grain size of the machine-made sand is within the range of 1-2 cm; the grain size of the facing sand is within the range of 0.1-0.5 cm; the collected aggregate images comprise different light states, aggregate dry and wet states and states with different shooting distances.
Fig. 1 is a schematic view of a scene in which a high definition camera is positioned obliquely above a vehicle, and when a transportation vehicle is in place, a video is shot and frame sequence pictures are extracted. The aggregate detection model system is formed by two parts, wherein one part is used for positioning the transport vehicle and acquiring a high-definition camera image; the other part is an image data processing and result displaying part.
An aggregate detection and classification method based on an AI intelligent machine vision technology comprises the following steps:
s1, shooting an aggregate transport video by using a high-definition camera, and transmitting the aggregate transport video in real time through a 5G network to obtain an aggregate image;
s2, processing the aggregate image by an image edge segmentation processing method, and extracting edge characteristics;
s3, manually labeling the aggregate type of the processed aggregate image;
s4, inputting the labeled aggregate image into a machine vision algorithm, training, extracting features, and outputting as a weight file;
and S5, replaying the trained weight in the detection system, and detecting and identifying the aggregate.
In step S2, the image edge segmentation processing method specifically includes the following steps:
a, introducing an aggregate image, carrying out gray processing on the aggregate image, and then solving a gradient map;
b, processing the image by using a watershed algorithm on the basis of the gradient image to obtain edge lines of the segmented image;
c, removing small objects in the picture and burr interference influencing the target by using image opening operation;
and D, performing adhesion segmentation to segment the objects adhered together.
FIG. 4 is a flowchart of an image edge segmentation technique according to the present invention. First, a captured image is imported. And carrying out graying processing on the image. Secondly, processing the image by using a watershed algorithm to make the target edge more obvious; and thirdly, removing small objects in the picture by using image opening operation to influence the burr interference of the target. And finally, performing adhesion segmentation to segment the objects adhered together.
FIG. 5 is a schematic view of the present invention showing the adhesion separation to separate the edges of the aggregates.
In practical application, aggregate accumulation is considered, edge information is mainly distinguished by a color light and shade boundary, and in the segmentation process, the similarity between adjacent pixels is used as an important reference, so that pixel points which are close in spatial position and have close gray values (gradient calculation) are connected with one another to form a closed outline.
The method comprises the following operation steps: graying the color image, then obtaining a gradient image, and finally performing a watershed algorithm on the basis of the gradient image to obtain the edge line of the segmented image.
In real images, due to the existence of noise points or other interference factors, the phenomenon of over-segmentation often exists by using the watershed algorithm, because many very small local extremum points exist. To solve the problem of over-segmentation, a mark (mark) image-based watershed algorithm can be used, i.e. guided by a priori knowledge, so as to obtain better image segmentation effect.
In step B, the watershed computation process is divided into two steps, one is a ranking process and one is a flooding process. Firstly, the gray levels of each pixel are sequenced from low to high, and then a first-in first-out (FIFO) structure is adopted to judge and mark each local minimum value in an influence domain of h-order height in the process of realizing inundation from low to high. The watershed transform obtains a catchbasin image of the input image, and boundary points between catchbasins are watershed. Clearly, the watershed represents the input image maxima points. Therefore, to obtain edge information of an image, a gradient image is usually taken as an input image, i.e.
g(x,y)＝grad(f(x,y))＝{[f(x,y)-f(x-1,y)]×2f(x,y)-f(x,y-1)]×2}×0.5
In the formula, f (x, y) represents an original image, and grad represents gradient operation.
The watershed algorithm has good response to weak edges, and noise in an image and slight gray level change of the surface of an object can generate an over-segmentation phenomenon. But it should be seen that the watershed algorithm has a good response to weak edges, and is guaranteed to close continuous edges. In addition, the closed water collecting basin obtained by the watershed algorithm provides possibility for analyzing the regional characteristics of the image.
In step C, to reduce the excessive segmentation caused by the watershed algorithm, the gradient function is modified, and a simple method is to perform threshold processing on the gradient image to eliminate the excessive segmentation caused by the small change of the gray scale. Namely, it is
g(x,y)＝max(grad(f(x,y)),gθ)
In the formula, g θ represents a threshold value.
The program adopts the method: and limiting the gradient image by using a threshold value to eliminate excessive segmentation caused by slight change of gray values, obtaining a proper amount of regions, sequencing the gray levels of edge points of the regions from low to high, and then calculating the gradient image by using a Sobel operator in the process of realizing inundation from low to high.
The operator comprises two sets of 3x3 matrices, horizontal and vertical, respectively, which are then subjected to planar convolution with the image to obtain horizontal and vertical luminance difference approximations. The formula is as follows:
Figure BDA0003620470160000121
Figure BDA0003620470160000122
the approximate values of the transverse and longitudinal gradients for each pixel of the image can be combined using the following formula to calculate the magnitude of the gradient.
Figure BDA0003620470160000123
Wherein A represents an original image, G x And G y Representing the images with their lateral and longitudinal edges detected, respectively.
After preprocessing, edge information is obtained, and a minimum rectangular frame is selected.
In step S4, the machine vision algorithm training process specifically includes the following steps:
a, selecting 5 detection areas for each marked aggregate image;
b, performing image convolution on each detection area respectively, and extracting image features;
c, performing up-sampling processing on the image, and restoring the feature map after convolution into the image;
d, integrating image features by using a tensor splicing algorithm;
e, outputting the classification result to a detection head, wherein the detection head is used for classifying the image and judging the classification result of the identified image through a probability function;
and f, identifying the result and outputting the detection result of the image.
Because the final purpose that this patent expects to reach is to carry out real-time detection discernment to the aggregate, and detects the background complicacy, needs discernment under the condition of different illumination, different distance and the different water content of aggregate, and the characteristic has the variety, and simple image processing probably can not satisfy the judgement to the sample characteristic, consequently uses machine vision algorithm to carry out the input and the training of sample under the different states.
After the collected images are processed in the steps, manually labeling, labeling aggregate types, inputting into a machine vision algorithm, and extracting features.
The machine vision algorithm used in this patent is mainly 6 working steps. Firstly, the method comprises the following steps: selecting 5 detection areas, dividing an image into 5 working areas, judging a final result by using odd areas, and selecting an identification category with 5 identification probabilities higher than 50% as the final result. Secondly, the method comprises the following steps: and (3) image convolution, namely performing image convolution on each area to achieve the purpose of extracting image features, mainly extracting edge features, color features and texture features of aggregates. Thirdly, the method comprises the following steps: and (3) sampling the image to enable the image to accord with the size of a display area, and inserting new elements by adopting an interpolation method, namely, on the basis of the original image pixels, adopting a proper interpolation algorithm between pixel points. Fourthly: tensor splicing, in which three channels (RGB) of an image need to be processed respectively before, and then recombined to generate a new image, is required, and tensor splicing is used to integrate image features. Fifth, the method comprises the following steps: and outputting the classification result to a detection head, wherein the detection head is used for classifying the image, and the classification result of the image is judged and identified through a probability function. Sixth: and identifying the result, and outputting the detection result of the image, namely the maximum probability result that the image is considered to be of a certain class by the algorithm.
Fig. 2 is a main flow chart of the aggregate detection method and system based on the AI intelligent visual processing technology, and the flow includes four parts of aggregate region image acquisition, image preprocessing, image feature extraction, and aggregate detection and judgment. The system firstly judges whether the vehicle is in place, if so, real-time image information is obtained, and area selection and image preprocessing are carried out; then, detecting the image by using a machine vision algorithm for identification; recording the recognition result, and outputting a final result if the result probability of the selected 5 images is greater than 50%; and judging whether the vehicle starts to pour, if so, stopping judging.
FIG. 3 is a technical route of the present patent, which includes preprocessing a training sample image, segmenting aggregate edges, performing feature extraction and training learning, and establishing an intelligent processing model for efficient detection of aggregates; and then processing and comparing the tested sample images to obtain an actual aggregate identification processing result.
FIG. 6 is a flow chart of a machine vision algorithm process used by the present invention. Firstly, extracting 5 regions from each image, namely an image central point, an upper left region, an upper right region, a lower left region and a lower right region, so as to realize one-time identification and multiple judgment and process each region; extracting image features such as light, angle, texture and the like through image convolution; then, carrying out up-sampling processing on the picture, and restoring the feature map after convolution into the picture; then, expanding the image dimensionality by using a tensor splicing algorithm; and finally, predicting the result by using the detection head. The final recognition result of 5 regions is the most probable consensus result, with more than 50% of the 5 regions having the same prediction result. And judging that all 5 areas are identified to finish the detection.
In the step a, the 5 detection areas are respectively a central point, an upper left area, an upper right area, a lower left area and a lower right area of the aggregate image.
In step b, the image features include edge features, color features and texture features of the aggregate.
In step c, the image is sampled to make the image conform to the size of the display area, and an interpolation method is adopted, i.e. a proper interpolation algorithm is adopted to insert new elements between pixel points on the basis of the original image pixels.
In step C, a processing method of modifying the gradient function so that the water collection basin only responds to the target to be detected is adopted to perform threshold processing on the gradient image, so as to reduce excessive segmentation caused by the change of gray scale.
Example 2
On the basis of the embodiment 1, the method further comprises the following steps: and S6, testing the new image, judging whether the sample size is sufficient or not according to the test condition, keeping the sample size as uniform as possible, if the accuracy of the weight to a certain class is lower than 90%, increasing the corresponding sample size, and re-extracting the features until the recognition rate of each class of aggregates reaches more than 90%, so that the requirements are met.
In step S4, using the algorithm will do the following:
classifying pictures: inputting the picture into a multilayer convolutional neural network, outputting a feature vector and feeding the feature vector back to a softmax unit to predict the type of the picture.
Positioning and classifying: judging whether a detection target exists in the graph by using an algorithm, marking the position of the target, marking by using a border (Bounding Box), and identifying and positioning the target.
Object detection: the picture can contain a plurality of objects, and a part of single picture also has a plurality of different classified objects to detect the plurality of objects.
Softmax function, normalized exponential function, is in fact a gradient log normalization of a finite discrete probability distribution. In multiple logistic regression and linear discriminant analysis, the input to the function is the result from K different linear functions, and the probability formula that the sample vector x belongs to the jth class is:
Figure BDA0003620470160000161
this can be viewed as a composite of the Softmax function of the K linear functions.
The positioning of the target is represented by an anchor Box, the input picture is divided into S-S grids, and each small grid can generate n anchor boxes; the real frame of the image and an anchor box generated by a small grid where the central point of the image is located are subjected to IOU calculation; the returned Box is a Bounding Box; when the anchors box is closer to the real width and height, the performance of the model is better; clustering the width, height and dimension cluster (dimension cluster) of a representative shape in the group route box of all samples in the training set by using a k-means algorithm; clustering a plurality of different anchor box groups, respectively applying the groups to the model, and finally finding out the optimal anchor box group which is a compromise between the complexity of the model and the high recall rate (high recall).
The Bounding Box formula is:
Figure BDA0003620470160000162
Figure BDA0003620470160000163
in the formula a w And a h The width and height of the anchor box,
t w and t h The width and height directly predicted for the bounding box,
b w and b h For the actual width and height of the prediction after conversion,
this is also the width and height of the output in the final prediction.
The loss function for object detection is as follows:
Figure BDA0003620470160000171
in the above formula, N is the total number of categories, yi is the probability of the current category obtained through the excitation function, yi is the probability of judging whether the prior frame is responsible for the target (0 or 1), if so, 0, otherwise, 1.
According to the invention, after the aggregate image is obtained, the edge characteristics are extracted by an image processing method, and the aggregate characteristics are enhanced. Firstly, graying the image, and extracting the image edge by a watershed algorithm. After all collected images are preprocessed, manually marking the images, inputting the images into a machine vision algorithm, extracting features, and outputting the weight files. The working process comprises the steps of shooting an aggregate transport video by using a high-definition camera, carrying out real-time transmission through a 5G network, extracting a key area in an image frame sequence of a monitoring area, starting to carry out primary processing on an image, carrying out preprocessing by using an image processing method, carrying out edge detection, carrying out manual marking after processing a collected image, inputting the marked image into a machine vision algorithm, carrying out training, and finally replaying the trained right in a detection system to detect and identify the aggregate. Aggregate images are obtained in real time through a 5G network by using a camera, then a computer image processing technology is applied to process videos, and AI intelligent algorithms are combined to detect and identify different types of aggregate state images, so that the problem that the existing aggregate monitoring method is complex in operation is solved, and meanwhile, the function of real-time processing is realized.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention; any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (7)

1. An aggregate detection and classification method based on an AI intelligent machine vision technology is characterized by comprising the following steps:
s1, shooting an aggregate transport video by using a high-definition camera, and transmitting the aggregate transport video in real time through a 5G network to obtain an aggregate image;
s2, processing the aggregate image by an image edge segmentation processing method, and extracting edge features;
s3, manually labeling the aggregate type of the processed aggregate image;
s4, inputting the labeled aggregate image into a machine vision algorithm, training, extracting features, and outputting as a weight file;
and S5, replaying the trained weight in the detection system, and detecting and identifying the aggregate.
2. The AI intelligent machine vision technology-based aggregate detection and classification method according to claim 1, wherein in step S2, the image edge segmentation processing method specifically includes the following steps:
a, introducing an aggregate image, carrying out gray processing on the aggregate image, and then solving a gradient map;
b, processing the image by using a watershed algorithm on the basis of the gradient image to obtain edge lines of the segmented image;
c, removing small objects in the picture and burr interference influencing the target by using image opening operation;
and D, performing adhesion segmentation to segment the objects adhered together.
3. The aggregate detection and classification method based on the AI intelligent machine vision technology as claimed in claim 1, wherein in step S4, the machine vision algorithm training process specifically includes the following steps:
a, selecting 5 detection areas for each marked aggregate image;
b, performing image convolution on each detection area respectively, and extracting image features;
c, performing up-sampling processing on the image, and restoring the feature map after convolution into the image;
d, integrating image features by using a tensor splicing algorithm;
e, outputting the classification result to a detection head, wherein the detection head is used for classifying the image and judging the classification result of the identified image through a probability function;
and f, identifying a result and outputting a detection result of the image.
4. The AI-intelligent machine vision-based aggregate detection and classification method according to claim 3, wherein in step a, the 5 detection areas are the center point, the upper left, the upper right, the lower left and the lower right of the aggregate image.
5. The AI-intelligent machine vision-based aggregate detection and classification method according to claim 3, wherein in step b, the image features comprise edge, color and texture features of the aggregate.
6. The AI intelligent machine vision technology-based aggregate detection and classification method according to claim 3, wherein in step c, the image is sampled to fit the size of the display area, and an interpolation method is used, in which new elements are inserted between pixel points by using a suitable interpolation algorithm based on the original image pixels.
7. The AI intelligent machine vision technology-based aggregate detection and classification method according to claim 2, characterized in that in step C, a gradient image is thresholded using a processing method that modifies a gradient function so that the water collection basin only responds to a target to be detected, so as to reduce excessive segmentation due to gray scale changes.
CN202210461094.7A 2022-04-28 2022-04-28 Aggregate detection and classification method based on AI intelligent machine vision technology Pending CN114882400A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210461094.7A CN114882400A (en) 2022-04-28 2022-04-28 Aggregate detection and classification method based on AI intelligent machine vision technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210461094.7A CN114882400A (en) 2022-04-28 2022-04-28 Aggregate detection and classification method based on AI intelligent machine vision technology

Publications (1)

Publication Number Publication Date
CN114882400A true CN114882400A (en) 2022-08-09

Family

ID=82672709

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210461094.7A Pending CN114882400A (en) 2022-04-28 2022-04-28 Aggregate detection and classification method based on AI intelligent machine vision technology

Country Status (1)

Country Link
CN (1) CN114882400A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116260990A (en) * 2023-05-16 2023-06-13 合肥高斯智能科技有限公司 AI asynchronous detection and real-time rendering method and system for multipath video streams
CN116571330A (en) * 2023-05-12 2023-08-11 江苏乐通彩业新型建材有限公司 Intelligent dry-mixed mortar production device and manufacturing method thereof
CN116664539A (en) * 2023-06-14 2023-08-29 中南大学 Method and device for processing water-permeable water-stable recycled aggregate mixture digital image

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116571330A (en) * 2023-05-12 2023-08-11 江苏乐通彩业新型建材有限公司 Intelligent dry-mixed mortar production device and manufacturing method thereof
CN116571330B (en) * 2023-05-12 2023-11-17 江苏乐通彩业新型建材有限公司 Intelligent dry-mixed mortar production device and manufacturing method
CN116260990A (en) * 2023-05-16 2023-06-13 合肥高斯智能科技有限公司 AI asynchronous detection and real-time rendering method and system for multipath video streams
CN116260990B (en) * 2023-05-16 2023-07-28 合肥高斯智能科技有限公司 AI asynchronous detection and real-time rendering method and system for multipath video streams
CN116664539A (en) * 2023-06-14 2023-08-29 中南大学 Method and device for processing water-permeable water-stable recycled aggregate mixture digital image
CN116664539B (en) * 2023-06-14 2024-03-12 中南大学 Digital image processing method and device for water-permeable water-stable recycled aggregate mixture

Similar Documents

Publication Publication Date Title
Akagic et al. Pavement crack detection using Otsu thresholding for image segmentation
CN106651872B (en) Pavement crack identification method and system based on Prewitt operator
CN109743879B (en) Underground pipe gallery leakage detection method based on dynamic infrared thermography processing
CN114882400A (en) Aggregate detection and classification method based on AI intelligent machine vision technology
CN114419025A (en) Fiberboard quality evaluation method based on image processing
Liu et al. Novel approach to pavement cracking automatic detection based on segment extending
Dorafshan et al. Automatic surface crack detection in concrete structures using OTSU thresholding and morphological operations
WO2016172827A1 (en) Stepwise-refinement pavement crack detection method
Bai et al. Image segmentation method for coal particle size distribution analysis
CN111814686A (en) Vision-based power transmission line identification and foreign matter invasion online detection method
Quan et al. The method of the road surface crack detection by the improved Otsu threshold
CN109816645B (en) Automatic detection method for steel coil loosening
CN113935666B (en) Building decoration wall tile abnormity evaluation method based on image processing
CN110503637B (en) Road crack automatic detection method based on convolutional neural network
Daniel et al. Automatic road distress detection and analysis
CN112232138A (en) Channel slope damage intelligent identification method based on superpixel characteristics
CN109598681B (en) No-reference quality evaluation method for image after repairing of symmetrical Thangka
CN113436157A (en) Vehicle-mounted image identification method for pantograph fault
Sorncharean et al. Crack detection on asphalt surface image using enhanced grid cell analysis
Abbas et al. Automated pavement distress detection using image processing techniques
CN107516315B (en) Tunneling machine slag tapping monitoring method based on machine vision
Cheng et al. Image segmentation technology and its application in digital image processing
CN106530292B (en) A kind of steel strip surface defect image Fast Identification Method based on line scan camera
CN111860500A (en) Shoe print wear area detection and edge tracing method
CN108229562B (en) Method for obtaining classification condition of concrete pavement damage

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination