Method and system for intelligently detecting cleanliness of aerial work platform equipment
Technical Field
The invention belongs to the technical field of image recognition, and relates to a method and a system for intelligently detecting cleanliness of aerial work platform equipment.
Background
Aerial work platform equipment can play a great role in modern industrial industries. In daily equipment maintenance work, cleaning is an essential item, and is directly related to whether the equipment can continuously and normally operate. Because the high-altitude operation platform equipment is higher in position and time-consuming and labor-consuming to clean, a regular cleaning mode or a mode of cleaning after manually collecting and reporting the equipment condition is generally adopted. The former may waste resources when cleaning is not required, while the latter is not free of human error. If the automatic judgment and identification can be carried out on the aerial work platform equipment by applying artificial intelligence, the problems can be effectively solved.
With the increasing maturity of artificial intelligence theory and technology, the research and application fields are continuously expanded, including robots, language recognition, image recognition, natural language processing, expert systems, and the like. Through analysis, the image recognition mode is considered to be possibly applied to cleanliness recognition of aerial work platform equipment. Currently, no relevant method is available in the industry.
Disclosure of Invention
In order to solve the problems, the invention discloses a method and a system for intelligently detecting the cleanliness of aerial work platform equipment, which can intelligently identify pollutants of an aerial work platform and calculate the cleaning duration according to the pollutants.
In order to achieve the purpose, the invention provides the following technical scheme:
a method for intelligently detecting cleanliness of aerial work platform equipment comprises the following steps:
step 1, data labeling and machine learning
Marking picture characteristics, marking a main body, a pollutant outline and a pollutant type based on the existing aerial work equipment picture, meshing the pollutant outline picture, calculating the area of the pollutant in each grid and judging, when the area of the pollutant in one grid is smaller than a first threshold value, marking the pollutant type as clean and the grid as the main body, when the area of the pollutant in one grid is larger than a second threshold value, marking the specific pollutant type and the grid as the main body, and if the area is between the first threshold value and the second threshold value, marking the pollutant type as 0 and the grid as the main body; marking the mesh of the non-body part as a non-body;
step 2, model design
A full convolution network model is used, a main body identification module and a pollutant classification module are added to a characteristic layer at the bottom of the model, and the pollutant classification module adopts a 1X1 convolution layer; the main body identification module comprises a down-sampling unit and an up-sampling unit connected with the down-sampling unit;
step 3, training the model
Inputting the processed label image into a model for training; the main body identification module is realized by adopting a facial _ loss with mask; the pollutant classification module is realized by using a cross entropy loss function with mask;
step 4, collecting photos of each position of the aerial work platform equipment, identifying the equipment picture compliance in real time, and performing corresponding error prompt when the equipment picture is not compliant;
step 5, inputting data to the model to obtain corresponding output
Processing the image acquired in the step 4 into an rgb three-channel image with a proper size, and inputting the image into a model; the obtained model is output as a matrix, each pixel point corresponds to an area in the original image, and the identification result of the area is predicted through multiple channels; and finally, identifying and distinguishing the main body and the background, and obtaining the type of the polluted medium of each pixel.
Further, the method also comprises a step 6 of calculating the cleanliness and the cleaning time of the current vehicle body, wherein the formula is as follows:
cleaning duration of single-side car body is equal to cleaning coefficient of pollutant medium, cleaning area of pollutant medium and rusting coefficient
And calculating the total cleaning time length as the sum of the cleaning time lengths of all the side vehicle bodies.
Further, in step 1, the first threshold is 0.3, and the second threshold is 0.7.
Further, the training strategy in the step 4 comprises data enhancement and learning rate attenuation.
Further, in the step 5, rgb three-channel images with the size of 320X320 are input into the model; the obtained model output is a matrix of 10X10X6, each pixel point corresponds to a 32X32 area in the original image, and the identification result of the area is predicted by adopting six channels; wherein, the 1 st channel is the confidence of the subject identification; the 2 nd, 3 rd, 4 th, 5 th and 6 th channels respectively correspond to the following contamination situations: cleaning, removing dust, cement mortar, paint and paint, removing paint and rusting, and selecting the pollutant type corresponding to the maximum value.
Further, in step 6, the contaminated area is the total number of grids corresponding to the contaminant/the total number of grids corresponding to the main body.
Further, in step 6, a specific calculation formula of the total cleaning time is as follows:
when total clean for long ═ when the cat ladder side automobile body is clean long + when the cat ladder left side automobile body is clean long + when the front side automobile body is clean long + when the cat ladder right side is clean long + when the top platform is clean long.
The invention also provides a system for intelligently detecting the cleanliness of the aerial work platform equipment, which comprises the following components: the system comprises a data labeling and machine learning module, a picture acquisition module, a model design module, a model training module, a model input and output module and a cleanliness calculation module; the data labeling and machine learning module is used for labeling the pollutant outline according to the existing picture and generating a label file; the picture acquisition module is used for acquiring pictures of all positions of the aerial work platform equipment and checking the picture compliance of the equipment; the model design module is used for establishing a full convolution network comprising a main body identification module and a pollutant classification module; the model training module is used for training the model obtained by the model design module; the model input and output module is used for inputting data to the trained model and acquiring an output result; and the cleanliness calculation module is used for calculating cleanliness and cleaning time according to the results obtained by the model input and output module.
Compared with the prior art, the invention has the following advantages and beneficial effects:
according to the invention, the idea of gridding marking is adopted, the local singleness of the pollutant types in the actual scene is utilized, and meanwhile, the fuzzy processing method is adopted for the outline edges of the pollutants, so that the pollutant identification precision is improved, different pollutants and areas of the vehicle body can be effectively identified, and the rules of cleanliness and cleaning duration are output based on the method; thereby realize aerial working platform equipment outward appearance pollution degree and clean long intelligent recognition, accomplish equipment cleaning effect check automatically, reduce artifical work load of checking, promote work efficiency.
Drawings
FIG. 1 is a flow chart of the present invention.
Fig. 2 is a contaminant profile plot.
Fig. 3 is a schematic diagram of a pollutant outline grid, wherein a is an outline label graph after gridding, and b is a result graph after calculation by using double thresholds according to the occupied area of the pollutant in each grid after gridding.
Fig. 4 is a schematic diagram of a full convolution network model adopted by the present invention.
FIG. 5 is a diagram illustrating an updated learning rate curve during model training, with the abscissa being the training step size. The ordinate is the learning rate used for each step in the training model.
Detailed Description
The technical solutions provided by the present invention will be described in detail below with reference to specific examples, and it should be understood that the following specific embodiments are only illustrative of the present invention and are not intended to limit the scope of the present invention. Additionally, the steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions and, although a logical order is illustrated in the flow charts, in some cases, the steps illustrated or described may be performed in an order different than here.
A method for intelligently detecting cleanliness of aerial work platform equipment comprises the following steps
Step 1, data labeling and machine learning
Firstly, a machine learning model is designed, a large number of existing aerial work equipment pictures in the system are input, and computer learning and identification are assisted through marking (manual marking) of appearance attributes of the equipment. On the basis of obtaining the auxiliary marking data, the model is further trained, and the AI is assisted to identify the pollutants and the pollution area through marking the pollutant medium and the pollution area in the picture.
Specifically, the labeling process is divided into main body labeling and pollutant labeling:
labeling the outline of the pollutant: manually marking a pollutant outline through software labelme as shown in fig. 2, marking the type of the pollutant, wherein the position with the pollutant (namely the pollutant outline and the part in the outline) also belongs to a main body part, namely the method is provided with a main body mark;
b, generating a label file: the original is gridded by 32X32 (pixels) as shown in fig. 3. Calculating the area of the pollutant in each grid, if the area of the pollutant in one grid is less than 0.3 (namely the area ratio of the pollutant to the area of the grid is less than 30%), marking the grid as clean, and the grid is provided with a main body and is corresponding to a Mask 2Mask mark as 1 (the Mask 2Mask marks are of two types, 1 is provided with the main body and 0 is provided with no main body); greater than 0.7 is marked as the contaminant (i.e., the type of contaminant identified in step A), and the grid is subject, corresponding to Mask 2Mask marked as 1; if the area is between 0.3 and 0.7, the Mask 1Mask corresponding to the pollutant type is marked as 0 (the Mask 1Mask corresponding to the pollutant type is marked as six types: clean, dust, cement mortar, paint and rust, and other (0)), which means that whether the network has the artificially marked pollutant or not cannot be judged accurately, but the grid is provided with a main body and the Mask 2 corresponding to the grid is marked as 1. While the non-body part (i.e., the part with no body label in step a) is labeled 0 corresponding to Mask 2 Mask.
Step 2, model design (network structure)
A full convolution network model (FCN) was used, but two more functional structures (a subject identification module and a contaminant classification module) were added to the bottom feature layer, as shown in fig. 4. The pollutant identification focuses more on local features, and the pollutants are directly classified by a convolution layer of 1X 1. The main body identification focuses on global features, so that the main body identification module comprises a down-sampling unit and an up-sampling unit, and then up-sampling is performed after down-sampling is performed once (similar to a Unet structure), and the applicability of the model to any scene is effectively improved.
Step 3, training the model
Inputting the processed label image into a model for training; the main body identification module adopts the facial _ loss with mask; the contaminant classification module uses a cross-entropy loss function with mask.
Training a strategy:
A. data enhancement (rotation, mirroring, cropping, noise addition) before model training
B. Adopting a learning rate attenuation strategy in the model training process
And 4, shooting pictures of all positions of the aerial work platform equipment by using an intelligent AI camera, identifying the equipment picture compliance in real time, including whether the pictures meet shooting standards, whether the pictures are inclined, complete and clear, whether an interface detects a target or not, prompting a corresponding error in real time, assisting in manual shooting, and making a basis for further identification and application of the images.
Step 5, inputting data to the model to obtain corresponding output
Inputting the image obtained in step 4 (rgb three-channel image to be processed into size 320X320 (pixels)) into the model; the obtained model output is a matrix of 10X10X6, each pixel point corresponds to a 32X32 area in the original image, and the identification result of the area is predicted by six channels. Wherein, the 1 st channel is the confidence coefficient of the subject identification, the subject is judged if the value of the confidence coefficient is more than 0.5, and the background is judged if the value of the confidence coefficient is less than 0.5; the 2 nd, 3 rd, 4 th, 5 th and 6 th channels respectively correspond to the following contamination situations: cleaning, dust, cement mortar, paint and paint removal and corrosion, and selecting the pollutant type corresponding to the maximum output value of each channel as the pollutant output type of the channel.
And finally obtaining the pollutant type of each pixel point through model output.
In the existing pollutant identification algorithm, a better identification result can be obtained only for a single pollutant image, but in practical application, the situation that various pollutants are mixed is more common, and if a detection algorithm is used, the problem that labeling is difficult or even impossible occurs. The method adopts the idea of gridding marking, utilizes the local singleness of pollutant types in the actual scene, and simultaneously adopts a fuzzy processing method for the outline edge of the pollutant, thereby solving the problem of difficult marking. And the pollutant identification precision is improved.
Totally 700 pieces of labeled data are adopted, a training set, a verification set and a test set are divided according to 0.6, 0.2 and 0.2, and an optimal model in the verification set is selected for testing through 300epoch iteration, wherein the main body identification precision reaches 96.7% and the pollutant identification precision reaches 76.2% in the test. In the prior pollutant identification model, the pollutant identification precision is only 68.1%.
And 6, calculating the cleanliness and the cleaning time of the current vehicle body by using the pollutant medium and the pollution area and integrating other dimensional factors such as corrosion degree, purposes and the like. The formula is as follows:
area of pollutant is the total number of grids corresponding to pollutant/total number of grids corresponding to main body
Cleaning duration of single-side car body is equal to cleaning coefficient of pollutant medium, cleaning area of pollutant medium and rusting coefficient
Wherein, the cleaning coefficient and the corrosion coefficient of the pollutant medium are preset. When the unilateral vehicle body has multiple pollutants, aiming at each pollutant, multiplying the area of the pollutant by the corrosion coefficient and the pollutant cleaning coefficient to obtain the pollution cleaning time, and adding the cleaning time of the pollutants on the unilateral vehicle body.
When total clean for long ═ when the cat ladder side automobile body is clean long + when the cat ladder left side automobile body is clean long + when the front side automobile body is clean long + when the cat ladder right side is clean long + when the top platform is clean long.
The invention also provides a system for intelligently detecting the cleanliness of the aerial working platform equipment, which can realize the method and comprises a data labeling and machine learning module, a picture acquisition module, a model design module, a model training module, a model input and output module and a cleanliness calculation module. The data labeling and machine learning module is used for labeling the pollutant outline according to the pictures in the database and generating a label file, namely realizing the content of the step 1; the picture acquisition module is used for acquiring pictures of all positions of the aerial work platform equipment and checking the equipment picture compliance, namely realizing the content of the step 2; the model design module is used for establishing a full convolution network comprising a main body identification module and a pollutant classification module, namely realizing the content of the step 3; the model training module is used for training a model, namely realizing the content of the step 4; the model input and output module is used for inputting data to the trained model and acquiring an output result, namely the content of the step 5 is realized; and the cleanliness calculating module is used for calculating cleanliness and cleaning time, namely realizing the content of the step 6. The system is realized by adopting a computer software technology.
The technical means disclosed in the invention scheme are not limited to the technical means disclosed in the above embodiments, but also include the technical scheme formed by any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and such improvements and modifications are also considered to be within the scope of the present invention.