CN116258912A - Workpiece identification method based on convolutional neural network - Google Patents

Workpiece identification method based on convolutional neural network Download PDF

Info

Publication number
CN116258912A
CN116258912A CN202310296461.7A CN202310296461A CN116258912A CN 116258912 A CN116258912 A CN 116258912A CN 202310296461 A CN202310296461 A CN 202310296461A CN 116258912 A CN116258912 A CN 116258912A
Authority
CN
China
Prior art keywords
convolutional neural
neural network
workpieces
improved
data set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310296461.7A
Other languages
Chinese (zh)
Inventor
侯勇
陈妍
毛润华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University of Science and Technology
Original Assignee
Tianjin University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University of Science and Technology filed Critical Tianjin University of Science and Technology
Priority to CN202310296461.7A priority Critical patent/CN116258912A/en
Publication of CN116258912A publication Critical patent/CN116258912A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention designs a workpiece identification method based on a convolutional neural network. The core of the accurate recognition of the workpieces is to accurately detect the types and positions of different target workpieces, and an improved convolutional neural network is designed for the purpose. The method comprises the steps that a feature decomposition calculation module is added in an original YOLOv7 convolutional neural network, an Elan module in the convolutional neural network is removed, and the Elan module is replaced by a Conv convolutional layer, so that the calculation burden is reduced while the recognition accuracy is not reduced; performing recognition experiments on workpieces which are singly placed and stacked and placed in four different types through the improved convolutional neural network; the invention can realize accurate identification of workpieces placed in different types and different positions, and the weight, parameters and calculated amount of the improved network model are obviously reduced, the mPA is increased by 1.8%, the calculation processing load is reduced, the detection speed is increased, and the detection capability of the model on small targets and dense targets is improved.

Description

Workpiece identification method based on convolutional neural network
Technical Field
The invention relates to the field of deep learning, in particular to a workpiece identification method based on an improved convolutional neural network.
Background
Target detection is a fundamental task in the field of computer vision, and has been a research hotspot for students. Along with the rapid development of the science and technology of China, the manufacturing industry of China starts to introduce various intelligent technologies such as computers and robots, the China continuously promotes corresponding intelligent measures, and China gradually becomes an intelligent manufacturing strong country. Industry automation demands for the application of image vision technology are significantly raised by various industries, and industrial-level vision technology application is currently spread over large factories. Since the deep learning algorithm is applied to the target detection task, the target detection algorithm based on the deep learning is rapidly developed, and new target detection algorithms are continuously updated to record the detection precision and speed.
In various industrial processes, workpieces are the basic component parts in industrial processes. The quality of the workpiece directly affects the quality of products, especially manufacturing enterprises with large production scale, various products and large quantity, and the detection, identification and classification statistics of various products are very difficult by adopting a manual mode. Compared with the traditional target detection method, the target detection method based on the deep learning is better, the detection accuracy is higher, the speed is higher, and the development of the target detection model by using the deep learning method is becoming a trend.
Disclosure of Invention
The invention aims to solve the problem that different workpiece types can be accurately identified in a complex workpiece identification environment, and provides a workpiece identification method based on a convolutional neural network. The method mainly comprises the steps of optimizing a convolutional neural network, adding a feature decomposition calculation module in the YOLOv7 convolutional neural network, and decomposing and calculating input features along the vertical and horizontal directions by the feature decomposition calculation module to form two independent direction feature patterns; and an Elan module in the YOLOv7 convolutional neural network is replaced by a Conv convolutional layer, so that the burden of calculation processing is effectively reduced, and the precision and speed of workpiece identification are improved.
In order to solve the technical problems, the invention is realized by the following technical scheme:
the workpiece identification method based on the convolutional neural network is characterized by comprising the following execution sequences:
step 1: a dataset is created. Acquiring workpiece images under complex conditions through a camera and a lens, wherein the workpiece images comprise multiple workpiece images which are densely distributed and overlapped, and carrying out data marking on four different types of workpieces, wherein marking results are respectively 0, 1, 2 and 3; and performing Mosaic image enhancement on the images, randomly selecting the images from the photographed data sets, performing random cutting, splicing and random distribution operation, and expanding the data sets to 7500, wherein the ratio of the training set to the testing set to the verification set is 8:1:1.
Step 2: convolutional neural networks are improved. The convolutional neural network model is improved, a feature decomposition calculation module is mainly added in the network, feature information of small objects and dense objects in a workpiece is effectively extracted, and detection accuracy is further improved; and the Elan module in the original network is removed and changed into a Conv convolution layer, so that the load of calculation processing is increased by the Elan module in the original network, and the calculation speed can be increased while the calculation accuracy is not reduced by replacing the Conv convolution layer for some application scenes in which GPU acceleration cannot be provided by industry.
Step 3: the improved convolutional neural network is deployed and trained. Training the improved convolutional neural network by setting a relevant training strategy; and performing experimental comparison on the original convolutional neural network and the improved convolutional neural network to obtain improved data.
Step 4: and carrying out recognition experiments on four different types of randomly placed workpieces through the verification data set to obtain a recognition effect graph.
Compared with the existing workpiece identification technology, the invention has the beneficial effects that:
firstly, the convolutional neural network is improved, a feature decomposition calculation module is added on the basis of the original convolutional neural network, the sensitivity of the module to information such as direction and position in a network model is further enhanced, and the accuracy of workpiece detection is improved; meanwhile, the original Elan module is removed, and a Conv convolution layer is used for replacement, so that the calculation load is reduced, and the detection speed is increased.
Secondly, the convolutional neural network after improvement reduces the model difficulty, can provide convenience for some industrial scenes which cannot be accelerated by using the GPU, reduces the cost while increasing the detection precision, and can be widely applied to industrial detection scenes.
Drawings
FIG. 1 is a schematic diagram of the overall process of the invention.
Fig. 2 is a schematic diagram of workpiece image data enhancement contrast.
FIG. 3 is a schematic diagram of a feature decomposition calculation module.
Fig. 4 is a schematic diagram of a workpiece recognition experiment.
Detailed Description
For the purpose of enabling those of ordinary skill in the art to understand and practice the invention, the invention is explained in detail below with reference to the drawings and examples, in which:
the invention relates to a workpiece identification method based on a convolutional neural network, which is mainly used for improving the convolutional neural network and detecting and identifying workpiece images. The method specifically comprises the following steps:
step 1: a dataset is created. By selecting four different kinds of workpieces, randomly placing the positions of the workpieces for photographing, obtaining 800 data sets, carrying out data enhancement and data set marking on the data sets, and expanding the data sets, the specific steps are as follows:
step 1.1: in the shot 800 data sets, performing a Mosaic enhancement mode on the image, wherein the mode comprises affine transformation, gaussian filtering, horizontal overturning and other operations;
step 1.2: marking all workpiece types by a data set, wherein the workpiece types are 0, 1, 2 and 3;
step 1.3: and randomly scaling and cutting the data set, and then randomly distributing and splicing the data set, so that the data set is fully expanded to 7500 pictures, wherein the ratio of the training set to the testing set to the verification set is 8:1:1.
Step 1.4: and carrying out self-adaptive anchor frame calculation on the data set, wherein in each calculation, the network is automatically adapted to calculate the optimal anchor frame value in different training sets.
Step 1.5: the image sizes are unified through a gray filling mode, the length-width equal ratio scaling of the original image is corresponding to the unified size, the remained blank part is filled with gray, and the unified image size is 640 x 640 and is taken as the input image size. Fig. 2 is a schematic diagram of workpiece image data enhancement contrast.
As shown in fig. 2, the image information of the workpiece subjected to data enhancement is clearer, and the image characteristics can be further enhanced under the condition of weak light.
Step 2: convolutional neural networks are improved. Because a plurality of irregularly placed and densely placed workpieces exist on an automatic production line, a feature decomposition calculation module is added on the basis of a YOLOv7 convolutional neural network model, and a Conv convolutional layer is used for replacing an original Elan module, so that the calculation burden is reduced while the recognition accuracy is not reduced. The method comprises the following specific steps:
step 2.1: and adding a characteristic decomposition calculation module in the original convolutional neural network model. The module can take any intermediate tensor x= [ X ] 1 ，x 2 ，...，x A ]∈R A×B×C As input, and outputs an enhanced output y= [ Y ] of the same size 1 ，y 2 ，...，y A ]. The schematic diagram of the feature decomposition calculation module is shown in fig. 3, and the specific steps are as follows:
step 2.1.1: the feature decomposition computation module decomposes the input features into two one-dimensional feature encodings, and aggregates the input features into two separate directions along the vertical and horizontal directions, respectively, to perceive a feature map. In order to ensure that the feature decomposition calculation module can capture remote space interaction with accurate position information, the global pooling is decomposed according to the following formula, and is converted into one-to-one dimensional feature coding operation, wherein the decomposition formula is as follows:
Figure BSA0000296759020000031
step 2.1.2: for input X, each channel is first encoded along the horizontal and vertical coordinates using pooled kernels of sizes (B, 1) and (1, c), respectively, resulting in an a-th channel output of height B and an a-th channel output of width c, the output formula being as follows:
Figure BSA0000296759020000032
Figure BSA0000296759020000041
step 2.1.3: in order to utilize the global receptive field and the accurate position information generated by the characteristic decomposition calculation module added in the convolutional neural network, a characteristic information generation module is designed. The module complies with three criteria: firstly, in the application of a mobile environment, conversion needs to be as simple and efficient as possible; secondly, the captured position information can be fully applied to accurately position the region of interest; finally, the relationship between channels can be captured effectively. After transformation in the feature decomposition calculation module, carrying out series operation on the transformed part, wherein the series formula is as follows:
f＝δ(F 1 ([Z b ，Z c ]))
step 2.1.4: then transforming f by using convolution transformation function, dividing f into f b And f c And performing dimension lifting operation by using 1*1 convolution kernels respectively, and combining a sigmoid activation function to obtain a convolution transformation vector, wherein the convolution transformation vector has the following formula:
g b ＝σ(F b (f b ))
g c ＝σ(F c (f c ))
step 2.1.5: after a series of transformation operations, the output Y formula of the feature information generating module is obtained as follows:
Figure BSA0000296759020000042
step 2.2: the Elan module is replaced with a Conv convolutional layer. To speed up the computation, the module is changed to Conv convolutional layer. The Conv convolution layer includes three parts of "convolution", "Batch Normalization" and "activation function" on the input feature map, where the convolution kernel size is 3*3 and the step size is 1. The activation function is a silu function, wherein a convolution layer is selected to perform local perception through a convolution kernel, local features in an image are extracted, calculation parameters of a model are greatly reduced, and a silu activation function formula is as follows:
silu(x)＝x*σ(x)
step 3: the improved convolutional neural network is deployed and trained. The method comprises the following specific steps:
step 3.1: a computer is deployed for training and testing. Wherein the CPU model is i5-12500H, the GPU is NVIDIA RTX3080, and the memory size is 10GB; the operating system is windows11; the deep learning framework is Pytorch;
step 3.2: setting a relevant training strategy. Wherein the learning rate is 0.05, batch_size is 32 in each iteration, the weight attenuation coefficient is set to 0.0005, and the momentum parameter is set to 0.935;
step 3.3: and training the image with the size of 640 x 640 after image processing by using the created 6000 training data sets as input of the convolutional neural network, and finally obtaining a detection result of an external rectangular frame in the training set image, wherein the detection result comprises type identification and position information of the workpiece.
Step 3.4: the original convolutional neural network and the improved convolutional neural network model are subjected to experimental comparison, and the following table shows comparison results of the two models in the aspects of Weight, mPA, parameter Params and the like:
Figure BSA0000296759020000051
through the improved convolution network model, the mPA of the improved convolution network model is improved by 1.8%, the Weight, the parameter parameters and the calculated amount are obviously reduced, and the burden of a computer is reduced.
Step 4: four different kinds of workpieces were subjected to identification experiments. The method comprises the following specific steps:
step 4.1: the verification data set is taken as an input image. The device comprises four different kinds of workpieces, wherein the four different kinds of workpieces are respectively marked as 0, 1, 2 and 3, the four different kinds of workpieces are singly placed and the different kinds of workpieces are stacked, and the verification data set is used as an experimental input image;
step 4.2: four workpieces are identified through the improved convolutional neural network with the deployed network, and detection results of the external rectangular frames where the different types of workpieces are located and the types of workpieces are located in the images are output. Fig. 4 is a schematic diagram of a workpiece recognition experiment.
As shown in fig. 4, four different kinds of workpieces of 0, 1, 2 and 3 are detected in total; for different positions of four different types of single workpieces, the improved convolutional neural network can accurately identify the types and positions of the workpieces; for the workpieces stacked and placed in four different types, the improved convolutional neural network can accurately identify the types and positions of the workpieces.

Claims (5)

1. A workpiece identification method based on a convolutional neural network, which is characterized by comprising the following steps:
step 1: an image data set is created, image enhancement is carried out on four different kinds of workpieces, and the data set is marked on the four workpieces, so that the data set is effectively enlarged.
Step 2: convolutional neural networks are improved. Adding a feature decomposition calculation module in the YOLOv7 convolutional neural network model, and decomposing and re-polymerizing input features along two directions; and an Elan module in the network model is replaced by a Conv convolution layer, so that the calculation load is reduced, and the detection speed is increased.
Step 3: and deploying and training the improved convolutional neural network, and performing experimental comparison on the original convolutional neural network and the improved convolutional neural network to obtain improved data.
Step 4: four different types of workpieces placed at different positions are subjected to identification experiments through the verification data set.
2. The method for identifying workpieces according to claim 1, wherein the creating of the data set in step 1 is specifically,
step 1.1: selecting four different kinds of workpiece samples, randomly placing the workpiece positions for photographing, and obtaining 800 data sets; performing Mosaic image enhancement on the acquired data set, wherein the Mosaic image enhancement comprises Gaussian filtering, affine transformation, horizontal overturning and other operations;
step 1.2: carrying out data set marking on the acquired data sets, wherein four different kinds of workpieces are respectively marked as 0, 1, 2 and 3;
step 1.3: the data are randomly cut, scaled and randomly distributed, the number of data set samples is fully enlarged, and the number of data sets is increased to 7500 data sets, wherein the ratio of a training set to a testing set to a verifying set is 8:1:1;
step 1.4: performing self-adaptive anchor frame calculation on the enhanced data set, wherein in each calculation, the network automatically adapts to calculate the optimal anchor frame values in different training sets;
step 1.5: the enhanced image size is unified by using a gray filling mode, and the unified input image size is 640 x 640.
3. The method for recognizing workpieces according to claim 1, wherein the step 2 is to improve a convolutional neural network, specifically,
step 2.1: in a convolutional neural network model, input features are decomposed and globally pooled, and are converted into one-to-one dimensional feature coding operation, wherein the global pooling formula is as follows:
Figure QLYQS_1
step 2.2: for a given input x= [ X ] 1 ，x 2 ，...，x A ]∈R A×B×C Firstly, coding each channel along the horizontal coordinate and the vertical coordinate by using a pooling kernel with the dimensions of (B, l) and (l, C), and obtaining an a-th channel output with the height of B and an a-th channel output with the width of C as follows:
Figure QLYQS_2
Figure QLYQS_3
step 2.3: and carrying out series operation transformation on the generated characteristic diagram of A x l x C, wherein the formula is as follows:
f＝δ(F 1 ([Z b ，Z c ]))
step 2.4: dividing f into f by split operation along space dimension b And f c And performing dimension lifting operation by using 1*1 convolution kernels respectively, and combining a sigmoid activation function to obtain a convolution transformation vector, wherein the convolution transformation vector has the following formula:
g b ＝σ(F b (f b ))
g c ＝σ(F c (f c ))
step 2.5: according to the algorithm described above, the output Y is obtained for a given input X, the output formula being:
Figure QLYQS_4
step 2.6: replacing an Elan module in the original convolutional neural network with a Conv convolutional layer; the modified Conv convolution layer performs "convolution", "Batch Normalization", "Silu activation function" on the input feature map, where the convolution kernel size is 3*3 and the step size is 1.
4. The method of claim 1, wherein the step 3 of deploying and training the improved convolutional neural network operation is specifically,
step 3.1: deploying network setting, wherein an operating system is windows11, a CPU model is i5-12500H, a GPU is NVIDIA RTX3080, the memory size is 10GB, and a used deep learning frame is Pytorch;
step 3.2: setting a related training strategy, wherein the learning rate is 0.05, batch_size is 32 in each iteration, the weight attenuation coefficient is 0.0005, and the momentum parameter is 0.935;
step 3.3: and training the improved convolutional neural network by taking the training data set as the input of the convolutional neural network to obtain the type information and the position information of each workpiece in the training set image.
Step 3.4: and under the same training strategy, experimental comparison is carried out on the convolutional neural network before and after improvement, and the comparison result of the data in all aspects is obtained.
5. The method for identifying workpieces according to claim 1, wherein the step 4 is to perform an identification experiment on four different types of workpieces placed at different positions,
step 4.1: taking the verification set workpiece images as input, wherein the verification set workpiece images comprise four different types of workpieces which are randomly and independently placed and stacked, and the types of the workpieces are 0, 1, 2 and 3;
step 4.2: and carrying out an identification experiment on the verification data set through the deployed improved convolutional neural network to obtain an identification result graph.
CN202310296461.7A 2023-03-24 2023-03-24 Workpiece identification method based on convolutional neural network Pending CN116258912A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310296461.7A CN116258912A (en) 2023-03-24 2023-03-24 Workpiece identification method based on convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310296461.7A CN116258912A (en) 2023-03-24 2023-03-24 Workpiece identification method based on convolutional neural network

Publications (1)

Publication Number Publication Date
CN116258912A true CN116258912A (en) 2023-06-13

Family

ID=86686241

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310296461.7A Pending CN116258912A (en) 2023-03-24 2023-03-24 Workpiece identification method based on convolutional neural network

Country Status (1)

Country Link
CN (1) CN116258912A (en)

Similar Documents

Publication Publication Date Title
CN110322510B (en) 6D pose estimation method using contour information
CN109583483B (en) Target detection method and system based on convolutional neural network
CN110135503B (en) Deep learning identification method for parts of assembly robot
CN111080693A (en) Robot autonomous classification grabbing method based on YOLOv3
CN111462120B (en) Defect detection method, device, medium and equipment based on semantic segmentation model
CN113065546B (en) Target pose estimation method and system based on attention mechanism and Hough voting
CN111553949B (en) Positioning and grabbing method for irregular workpiece based on single-frame RGB-D image deep learning
CN112070727B (en) Metal surface defect detection method based on machine learning
CN111507357B (en) Defect detection semantic segmentation model modeling method, device, medium and equipment
CN112295933B (en) Method for robot to rapidly sort goods
CN113222982A (en) Wafer surface defect detection method and system based on improved YOLO network
CN110633738B (en) Rapid classification method for industrial part images
CN114743259A (en) Pose estimation method, pose estimation system, terminal, storage medium and application
CN113034593A (en) 6D pose marking method and system and storage medium
CN112164115A (en) Object pose identification method and device and computer storage medium
CN113269723A (en) Unordered grasping system for three-dimensional visual positioning and mechanical arm cooperative work parts
CN115272204A (en) Bearing surface scratch detection method based on machine vision
CN113762159B (en) Target grabbing detection method and system based on directional arrow model
CN113012228B (en) Workpiece positioning system and workpiece positioning method based on deep learning
CN111368637B (en) Transfer robot target identification method based on multi-mask convolutional neural network
CN113313694A (en) Surface defect rapid detection method based on light-weight convolutional neural network
CN116258912A (en) Workpiece identification method based on convolutional neural network
CN115601345A (en) Machine vision-based large-size punching plate hole group leakage-proof detection method and device
CN114611667A (en) Reconstruction method for calculating characteristic diagram boundary based on small-scale parameter matrix
CN113569737A (en) Notebook screen defect detection method and medium based on autonomous learning network model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication