CN117058241B - Electronic element positioning method and system based on artificial intelligence - Google Patents
Electronic element positioning method and system based on artificial intelligence Download PDFInfo
- Publication number
- CN117058241B CN117058241B CN202311305193.7A CN202311305193A CN117058241B CN 117058241 B CN117058241 B CN 117058241B CN 202311305193 A CN202311305193 A CN 202311305193A CN 117058241 B CN117058241 B CN 117058241B
- Authority
- CN
- China
- Prior art keywords
- circuit board
- image
- board image
- electronic element
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 19
- 238000001514 detection method Methods 0.000 claims abstract description 58
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000007781 pre-processing Methods 0.000 claims abstract description 18
- 238000012937 correction Methods 0.000 claims abstract description 10
- 238000001931 thermography Methods 0.000 claims abstract description 8
- 239000011159 matrix material Substances 0.000 claims description 48
- 239000013598 vector Substances 0.000 claims description 47
- 230000007246 mechanism Effects 0.000 claims description 24
- 238000006243 chemical reaction Methods 0.000 claims description 17
- 238000000354 decomposition reaction Methods 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 7
- 238000003708 edge detection Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 230000003213 activating effect Effects 0.000 claims description 3
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 239000012141 concentrate Substances 0.000 claims description 3
- 230000007797 corrosion Effects 0.000 claims description 3
- 238000005260 corrosion Methods 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 5
- 238000012512 characterization method Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 239000003990 capacitor Substances 0.000 description 2
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000005476 soldering Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/146—Aligning or centring of the image pick-up or image-field
- G06V30/1463—Orientation detection or correction, e.g. rotation of multiples of 90 degrees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/18—Extraction of features or characteristics of the image
- G06V30/1801—Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes or intersections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
- G06V30/19173—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of computer vision and discloses an electronic element positioning method and system based on artificial intelligence, wherein a circuit board image to be positioned is acquired through an infrared thermal imaging technology, and a LabelImg marking tool is adopted to mark the circuit board image to be positioned, so that a circuit board image data set is obtained; preprocessing an image in the circuit board image data set to obtain a preprocessed circuit board image data set; inputting the preprocessed circuit board image data set into a detection positioning model which is trained in advance, detecting and positioning the electronic element on the circuit board image through the detection positioning model, and outputting a detection positioning result; sequentially performing character recognition processing and correction processing on the electronic element characters on the circuit board image to obtain character recognition results of the electronic elements; obtaining position information of the electronic element on the circuit board image based on the detection and positioning result and the character recognition result of the electronic element; the invention improves the detection efficiency and accuracy of the circuit board.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to an electronic element positioning method and system based on artificial intelligence.
Background
An electronic component (electronic component) is a fundamental element in an electronic circuit, typically a separate package, and has two or more leads or metal contacts. The electronic components must be interconnected to form an electronic circuit with specific functions, such as: one of the common ways to connect electronic components is by soldering to a printed circuit board, amplifiers, radio transceivers, oscillators, etc. The electronic components may be individual packages (resistors, capacitors, inductors, transistors, diodes, etc.), or groups of various complexity, such as: integrated circuits (operational amplifiers, resistors, logic gates, etc.).
Along with the high-speed development of the electronic information industry, electronic products are mostly controlled by using a circuit board, and the circuit board is not separated from electronic elements such as various capacitors, resistors and inductors, and the electronic elements are marked by manpower in the traditional quality detection of the circuit board, so that the electronic information processing method is time-consuming, is easy to error, cannot meet the use requirement, and has high labor cost.
Disclosure of Invention
The invention aims to solve the problems and designs an electronic element positioning method and system based on artificial intelligence.
The first aspect of the invention provides an electronic component positioning method based on artificial intelligence, which comprises the following steps:
acquiring circuit board images to be positioned through an infrared thermal imaging technology, and marking the circuit board images to be positioned by using a LabelImg marking tool to obtain a circuit board image data set, wherein each circuit board image to be positioned contains a plurality of electronic elements;
preprocessing the images in the circuit board image data set to obtain a preprocessed circuit board image data set;
inputting the preprocessed circuit board image dataset into a detection positioning model which is obtained by training in advance, detecting and positioning the electronic element on the circuit board image through the detection positioning model, and outputting a detection positioning result;
sequentially performing character recognition processing and correction processing on the electronic element characters on the circuit board image to obtain character recognition results of the electronic elements;
and obtaining the position information of the electronic element on the circuit board image based on the detection and positioning result and the character recognition result of the electronic element.
Optionally, in a first implementation manner of the first aspect of the present invention, the preprocessing the image in the circuit board image dataset to obtain a preprocessed circuit board image dataset includes:
carrying out graying treatment on the images in the circuit board image data set to obtain a graying image data set;
converting an image sequence in the gray-scale image dataset from a three-dimensional matrix to a two-dimensional coordinate system matrix by adopting a multi-coordinate conversion method;
sequentially arranging the data in the converted image pixels, and performing matrix conversion treatment on the obtained column vectors to obtain a one-dimensional matrix;
sequentially ordering the one-dimensional sequences according to the column direction to form a new two-dimensional matrix, and performing standardization processing on the new two-dimensional matrix;
decomposing the standardized two-dimensional matrix through a singular value decomposition theorem, and converting the first column vector of the decomposed matrix into a frame of still image to obtain the circuit board image data set after image enhancement.
Optionally, in a second implementation manner of the first aspect of the present invention, preprocessing an image in the circuit board image data set to obtain a preprocessed circuit board image data set, further includes:
reading the image in the image data set of the circuit board after image enhancement, and carrying out edge detection by a Canny edge detection algorithm to obtain an edge image;
enhancing edge characteristics in the edge image by applying a Laplace filter to obtain an enhanced edge image;
traversing the gradient direction and the gradient size of each pixel point in the enhanced edge image, and carrying out edge refinement on the enhanced edge image by adopting an image refinement algorithm to obtain a refined edge image;
and processing the edge image after the thinning treatment by adopting a morphological expansion and corrosion method so as to obtain a preprocessed circuit board image data set.
Optionally, in a third implementation manner of the first aspect of the present invention, inputting the preprocessed circuit board image dataset into a detection positioning model obtained by training in advance, detecting and positioning an electronic component on a circuit board image through the detection positioning model, and outputting a detection positioning result, where the method includes:
the method comprises the steps of using a YOLOv5 model as a basic model of a detection positioning model, modifying a YOLOv5 model head network, adding a group of anchor points, and upsampling an image in a preprocessed circuit board image dataset to obtain an enlarged feature map;
performing feature recognition through an SE attention mechanism, an SA attention mechanism and an ECA attention mechanism in the YOLOv5 model to obtain a new feature map with attention weights;
the method comprises the steps of adopting a feature pyramid network in a YOLOv5 model to convey reinforced semantic features from top to bottom, adopting a path aggregation network in the YOLOv5 model to convey reinforced position features from bottom to top, and carrying out feature fusion on a new feature map with attention weights through the semantic features and the position features;
and performing convolution operation on a Head output layer in the YOLOv5 model, and generating a corresponding prediction frame to obtain a detection positioning result of the electronic element.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the performing feature recognition through a SE attention mechanism, a SA attention mechanism and an ECA attention mechanism in a YOLOv5 model to obtain a new feature map with attention weights includes:
carrying out global average pooling on the images in the preprocessed circuit board image dataset by detecting an SE attention mechanism in the positioning model to generate a first feature vector;
weighting the image from different positions of the channel dimension by using a weighting matrix through a full connection layer based on the first feature vector to obtain different first weight information, and carrying out weight assignment on the image through the first weight information to obtain a first feature map;
grouping the channel features in the first feature map to obtain a plurality of sub-features, adopting an SA attention mechanism in a detection positioning model to concentrate space and channel attention on each sub-feature, collecting the sub-features, and fusing the features of different groups through channel shuffling operation to obtain a second feature map;
calculating the average value of all pixels of the second feature map in each channel by detecting an ECA attention mechanism in a positioning model, and reducing the dimension of the second feature map by global average pooling operation to obtain a second feature vector;
and carrying out one-dimensional convolution operation on the obtained second feature vector, activating by a Sigmoid function to obtain the weight of each channel, and carrying out product operation between the image in the preprocessed circuit board image dataset and the obtained weight of each channel corresponding to the channels to obtain a new feature map with attention weight.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the performing, in order, a character recognition process and a correction process on characters of the electronic component on the circuit board image to obtain a character recognition result of the electronic component includes:
acquiring electronic element characters on a circuit board image, and representing the electronic element characters and converting the electronic element characters into character vectors;
converting the character vector into a vector matrix by adopting a one-dimensional convolutional neural network, extracting the characteristics of the character, sliding the convolutional kernel up and down, and performing dot product operation to obtain characteristic values at different positions to obtain a characteristic vector sequence;
and inputting the feature vector sequence into an LSTM (least squares) for extracting the context features so as to obtain a character primary recognition result.
Optionally, in a sixth implementation manner of the first aspect of the present invention, before the acquiring the electronic component character on the circuit board image and characterizing the electronic component character, the method further includes:
corresponding background points of the circuit board image to polar coordinates, and calculating the inclination angle of the shadow circuit board image by using the accumulated value on the polar coordinates; the nearest neighbor clustering method is based on extracting the center point of the connection domain of the circuit board image in a sub-domain of the circuit board image, and obtaining the inclination angle of the circuit board image through the continuous relation between the points on the base line.
The invention provides an electronic element positioning system based on artificial intelligence, which comprises an image marking module, a preprocessing module, a detection positioning module, a character recognition module and a result output module, wherein the image marking module is used for acquiring a circuit board image to be positioned through an infrared thermal imaging technology and marking the circuit board image to be positioned by using a LabelImg marking tool to obtain a circuit board image data set;
the preprocessing module is used for preprocessing the images in the circuit board image data set to obtain a preprocessed circuit board image data set;
the detection positioning module is used for inputting the preprocessed circuit board image data set into a detection positioning model which is obtained by training in advance, detecting and positioning the electronic element on the circuit board image through the detection positioning model, and outputting a detection positioning result;
the character recognition module is used for sequentially carrying out character recognition processing and correction processing on the electronic element characters on the circuit board image to obtain character recognition results of the electronic elements;
and the result output module is used for obtaining the position information of the electronic element on the circuit board image based on the detection and positioning result and the character recognition result of the electronic element.
Optionally, in a first implementation manner of the second aspect of the present invention, the preprocessing module includes a graying processing sub-module, a first converting sub-module, a second converting sub-module, a normalizing sub-module and a third converting sub-module, where the graying processing sub-module is configured to perform graying processing on an image in the image dataset of the circuit board to obtain a graying image dataset;
the first converting sub-module is used for converting the image sequence in the grayscale image dataset from a three-dimensional matrix to a two-dimensional coordinate system matrix by adopting a multi-coordinate conversion method;
the second conversion sub-module is used for sequentially arranging the data in the converted image pixels and performing matrix conversion treatment on the obtained column vectors to obtain a one-dimensional matrix;
the standardized sub-module is used for sequentially sequencing the one-dimensional sequences according to the column direction to form a new two-dimensional matrix, and then carrying out standardized processing on the new two-dimensional matrix;
and the third conversion sub-module is used for decomposing the standardized two-dimensional matrix through a singular value decomposition theorem, converting the first column vector of the decomposed matrix into a frame of still image, and obtaining the circuit board image data set after image enhancement.
Optionally, in a second implementation manner of the second aspect of the present invention, the character recognition module includes a characterization sub-module, a dot product operation sub-module, and a feature extraction sub-module, wherein,
the characterization submodule is used for acquiring electronic element characters on the circuit board image, characterizing the electronic element characters and converting the electronic element characters into character vectors;
the dot product operation sub-module is used for converting the character vector into a vector matrix by adopting a one-dimensional convolutional neural network, extracting the characteristics of the character, sliding the convolutional kernel up and down, and performing dot product operation to obtain characteristic values at different positions to obtain a characteristic vector sequence;
and the feature extraction sub-module is used for inputting the feature vector sequence into the LSTM to perform contextual feature extraction so as to obtain a character primary recognition result.
In the technical scheme provided by the invention, a circuit board image to be positioned is acquired through an infrared thermal imaging technology, and a LabelImg marking tool is adopted to mark the circuit board image to be positioned, so that a circuit board image data set is obtained; preprocessing the images in the circuit board image data set to obtain a preprocessed circuit board image data set; inputting the preprocessed circuit board image dataset into a detection positioning model which is obtained by training in advance, detecting and positioning the electronic element on the circuit board image through the detection positioning model, and outputting a detection positioning result; sequentially performing character recognition processing and correction processing on the electronic element characters on the circuit board image to obtain character recognition results of the electronic elements; obtaining position information of the electronic element on the circuit board image based on the detection and positioning result and the character recognition result of the electronic element; according to the invention, the electronic element position information on the circuit board is obtained through the artificial intelligence technology, so that the detection efficiency and accuracy of the circuit board are improved, and the labor cost is reduced.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention.
FIG. 1 is a schematic diagram of a first embodiment of an electronic component positioning method based on artificial intelligence according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a second embodiment of an electronic component positioning method based on artificial intelligence according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a third embodiment of an electronic component positioning method based on artificial intelligence according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic component positioning system based on artificial intelligence according to an embodiment of the present invention.
Detailed Description
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
For ease of understanding, the following describes a specific flow of an embodiment of the present invention, and please refer to fig. 1 for a schematic diagram of a first embodiment of an electronic component positioning method based on artificial intelligence, which specifically includes the following steps:
step 101, acquiring a circuit board image to be positioned by an infrared thermal imaging technology, and marking the circuit board image to be positioned by using a LabelImg marking tool to obtain a circuit board image data set;
in this embodiment, each circuit board image to be positioned includes a plurality of electronic components;
102, preprocessing an image in a circuit board image data set to obtain a preprocessed circuit board image data set;
step 103, inputting the preprocessed circuit board image dataset into a detection positioning model obtained by training in advance, detecting and positioning the electronic element on the circuit board image through the detection positioning model, and outputting a detection positioning result;
104, sequentially performing character recognition processing and correction processing on the electronic element characters on the circuit board image to obtain character recognition results of the electronic elements;
in the embodiment, electronic element characters on a circuit board image are acquired, and the electronic element characters are characterized and converted into character vectors; converting a character vector into a vector matrix by adopting a one-dimensional convolutional neural network, extracting the characteristics of the character, sliding the convolutional kernel up and down, and performing dot product operation to obtain characteristic values at different positions to obtain a characteristic vector sequence; and inputting the feature vector sequence into the LSTM for extracting the context features so as to obtain a character primary recognition result.
In the embodiment, the background point of the circuit board image is corresponding to the polar coordinate, and the cumulative value on the polar coordinate is utilized to calculate the inclination angle of the shadow circuit board image; the nearest neighbor clustering method is based on extracting the center point of the connection domain of the circuit board image in a sub-domain of the circuit board image, and obtaining the inclination angle of the circuit board image through the continuous relation between the points on the base line.
And 105, obtaining the position information of the electronic element on the circuit board image based on the detection and positioning result and the character recognition result of the electronic element.
In the embodiment of the invention, a circuit board image to be positioned is acquired through an infrared thermal imaging technology, and a LabelImg marking tool is adopted to mark the circuit board image to be positioned, so that a circuit board image data set is obtained; preprocessing an image in the circuit board image data set to obtain a preprocessed circuit board image data set; inputting the preprocessed circuit board image data set into a detection positioning model which is trained in advance, detecting and positioning the electronic element on the circuit board image through the detection positioning model, and outputting a detection positioning result; sequentially performing character recognition processing and correction processing on the electronic element characters on the circuit board image to obtain character recognition results of the electronic elements; obtaining position information of the electronic element on the circuit board image based on the detection and positioning result and the character recognition result of the electronic element; according to the invention, the electronic element position information on the circuit board is obtained through the artificial intelligence technology, so that the detection efficiency and accuracy of the circuit board are improved, and the labor cost is reduced.
Referring to fig. 2, a second embodiment of an electronic component positioning method based on artificial intelligence according to an embodiment of the present invention is shown, and the method includes:
step 201, carrying out graying treatment on images in the image dataset of the circuit board to obtain a graying image dataset;
step 202, converting an image sequence in a gray-scale image dataset from a three-dimensional matrix to a two-dimensional coordinate system matrix by adopting a multi-coordinate conversion method;
step 203, sequentially arranging the data in the converted image pixels, and performing matrix conversion processing on the obtained column vectors to obtain a one-dimensional matrix;
204, sequentially ordering the one-dimensional sequences according to the column direction to form a new two-dimensional matrix, and then carrying out standardization processing on the new two-dimensional matrix;
step 205, decomposing the standardized two-dimensional matrix through a singular value decomposition theorem, and converting a first column vector of the decomposed matrix into a frame of still image to obtain a circuit board image data set after image enhancement;
step 206, reading the image in the image data set of the circuit board after image enhancement, and performing edge detection by a Canny edge detection algorithm to obtain an edge image;
step 207, enhancing edge characteristics in the edge image by applying a laplace filter to obtain an enhanced edge image;
step 208, traversing the gradient direction and the gradient size of each pixel point in the enhanced edge image, and carrying out edge refinement on the enhanced edge image by adopting an image refinement algorithm to obtain a refined edge image;
and 209, processing the edge image after the thinning treatment by adopting a morphological expansion and corrosion method to obtain a preprocessed circuit board image data set.
Referring to fig. 3, a third embodiment of an electronic component positioning method based on artificial intelligence according to an embodiment of the present invention is shown, and the method includes:
step 301, using a YOLOv5 model as a basic model of a detection positioning model, modifying a YOLOv5 model head network, adding a group of anchor points, and upsampling an image in a preprocessed circuit board image data set to obtain an enlarged feature map;
step 302, performing feature recognition through an SE attention mechanism, an SA attention mechanism and an ECA attention mechanism in a YOLOv5 model to obtain a new feature map with attention weights;
in the embodiment, global average pooling is carried out on the images in the preprocessed circuit board image dataset by detecting an SE attention mechanism in the positioning model, so as to generate a first feature vector; weighting the image from different positions of the channel dimension by using a weighting matrix through a full connection layer based on the first feature vector to obtain different first weight information, and carrying out weight assignment on the image through the first weight information to obtain a first feature map; grouping channel features in the first feature map to obtain a plurality of sub-features, adopting an SA attention mechanism in a detection positioning model to concentrate space and channel attention on each sub-feature, collecting the sub-features, and fusing different groups of features through channel shuffling operation to obtain a second feature map; calculating the average value of all pixels of the second feature map in each channel by detecting an ECA attention mechanism in the positioning model, and reducing the dimension of the second feature map by global average pooling operation to obtain a second feature vector; and carrying out one-dimensional convolution operation on the obtained second feature vector, activating by a Sigmoid function to obtain the weight of each channel, and carrying out product operation between the image in the preprocessed circuit board image dataset and the obtained weight of each channel corresponding to the channels to obtain a new feature map with attention weight.
Step 303, adopting a feature pyramid network in the YOLOv5 model to convey the reinforced semantic features from top to bottom, adopting a path aggregation network in the YOLOv5 model to convey the reinforced position features from bottom to top, and carrying out feature fusion on the new feature map with the attention weight through the semantic features and the position features;
and 304, performing convolution operation on a Head output layer in the YOLOv5 model, and generating a corresponding prediction frame to obtain a detection positioning result of the electronic element.
Referring to fig. 4, a schematic structural diagram of an electronic component positioning system based on artificial intelligence according to an embodiment of the present invention includes an image marking module, a preprocessing module, a detection positioning module, a character recognition module, and a result output module, where:
the image marking module 401 is used for acquiring the image of the circuit board to be positioned through an infrared thermal imaging technology, and marking the image of the circuit board to be positioned by adopting a LabelImg marking tool to obtain a circuit board image data set;
a preprocessing module 402, configured to preprocess an image in the circuit board image dataset to obtain a preprocessed circuit board image dataset;
the detection positioning module 403 is configured to input the preprocessed circuit board image dataset into a detection positioning model that is obtained by training in advance, detect and position an electronic component on a circuit board image through the detection positioning model, and output a detection positioning result;
the character recognition module 404 is configured to sequentially perform character recognition processing and correction processing on the electronic component characters on the circuit board image, so as to obtain a character recognition result of the electronic component;
and the result output module 405 is configured to obtain location information of the electronic component on the circuit board image based on the detection and positioning result and the character recognition result of the electronic component.
In this embodiment, the preprocessing module includes a grayscale processing sub-module, a first conversion sub-module, a second conversion sub-module, a normalization sub-module, and a third conversion sub-module, where:
the grey processing submodule is used for carrying out grey processing on the images in the image dataset of the circuit board to obtain a grey image dataset;
the first converting sub-module is used for converting the image sequence in the grayscale image dataset from a three-dimensional matrix to a two-dimensional coordinate system matrix by adopting a multi-coordinate conversion method;
the second conversion sub-module is used for sequentially arranging the data in the converted image pixels and performing matrix conversion treatment on the obtained column vectors to obtain a one-dimensional matrix;
the standardized sub-module is used for sequentially sequencing the one-dimensional sequences according to the column direction to form a new two-dimensional matrix, and then carrying out standardized processing on the new two-dimensional matrix;
and the third conversion sub-module is used for decomposing the standardized two-dimensional matrix through a singular value decomposition theorem, converting the first column vector of the decomposed matrix into a frame of still image, and obtaining the circuit board image data set after image enhancement.
In this embodiment, the character recognition module includes a characterization sub-module, a dot product operation sub-module, and a feature extraction sub-module, where:
the characterization submodule is used for acquiring electronic element characters on the circuit board image, characterizing the electronic element characters and converting the electronic element characters into character vectors;
the dot product operation sub-module is used for converting the character vector into a vector matrix by adopting a one-dimensional convolutional neural network, extracting the characteristics of the character, sliding the convolutional kernel up and down, and performing dot product operation to obtain characteristic values of different positions to obtain a characteristic vector sequence;
and the feature extraction sub-module is used for inputting the feature vector sequence into the LSTM to extract the context features so as to obtain a character primary recognition result.
Through implementation of the scheme, the system comprises the image marking module, the preprocessing module, the detection positioning module, the character recognition module and the result output module, and the electronic element position information on the circuit board is obtained through the artificial intelligence technology, so that the detection efficiency and accuracy of the circuit board are improved, and the labor cost is reduced.
The foregoing has shown and described the basic principles, principal features and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the above-described embodiments, and that the above-described embodiments and descriptions are only preferred embodiments of the present invention, and are not intended to limit the invention, and that various changes and modifications may be made therein without departing from the spirit and scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (1)
1. An electronic component positioning method based on artificial intelligence is characterized by comprising the following steps:
acquiring circuit board images to be positioned through an infrared thermal imaging technology, and marking the circuit board images to be positioned by using a LabelImg marking tool to obtain a circuit board image data set, wherein each circuit board image to be positioned contains a plurality of electronic elements;
preprocessing the images in the circuit board image data set to obtain a preprocessed circuit board image data set;
inputting the preprocessed circuit board image dataset into a detection positioning model which is obtained by training in advance, detecting and positioning the electronic element on the circuit board image through the detection positioning model, and outputting a detection positioning result;
sequentially performing character recognition processing and correction processing on the electronic element characters on the circuit board image to obtain character recognition results of the electronic elements;
obtaining position information of the electronic element on the circuit board image based on the detection and positioning result and the character recognition result of the electronic element;
the preprocessing the image in the circuit board image data set to obtain a preprocessed circuit board image data set, comprising:
carrying out graying treatment on the images in the circuit board image data set to obtain a graying image data set;
converting an image sequence in the gray-scale image dataset from a three-dimensional matrix to a two-dimensional coordinate system matrix by adopting a multi-coordinate conversion method;
sequentially arranging the data in the converted image pixels, and performing matrix conversion treatment on the obtained column vectors to obtain a one-dimensional matrix;
sequentially sequencing the one-dimensional sequences to form a new two-dimensional matrix, and then carrying out singular value decomposition theorem decomposition standardization treatment on the new two-dimensional matrix;
decomposing the standardized two-dimensional matrix according to the singular value decomposition theorem to convert the standardized two-dimensional matrix to obtain a circuit board image dataset after image enhancement;
preprocessing the image in the circuit board image data set to obtain a preprocessed circuit board image data set, and further comprising:
reading the image in the image data set of the circuit board after image enhancement, and carrying out edge detection by a Canny edge detection algorithm to obtain an edge image;
enhancing edge characteristics in the edge image by applying a Laplace filter to obtain an enhanced edge image;
traversing the gradient direction and the gradient size of each pixel point in the enhanced edge image, and carrying out edge refinement on the enhanced edge image by adopting an image refinement algorithm to obtain a refined edge image;
processing the edge image after refinement treatment by adopting a morphological expansion and corrosion method to obtain a preprocessed circuit board image data set;
inputting the preprocessed circuit board image data set into a detection positioning model which is trained in advance, detecting and positioning the electronic element on the circuit board image through the detection positioning model, and outputting a detection positioning result, wherein the detection positioning method comprises the following steps of:
the method comprises the steps of using a YOLOv5 model as a basic model of a detection positioning model, modifying a YOLOv5 model head network, adding a group of anchor points, and upsampling an image in a preprocessed circuit board image dataset to obtain an enlarged feature map;
performing feature recognition through an SE attention mechanism, an SA attention mechanism and an ECA attention mechanism in the YOLOv5 model to obtain a new feature map with attention weights;
the method comprises the steps of adopting a feature pyramid network in a YOLOv5 model to convey reinforced semantic features from top to bottom, adopting a path aggregation network in the YOLOv5 model to convey reinforced position features from bottom to top, and carrying out feature fusion on a new feature map with attention weights through the semantic features and the position features;
performing convolution operation on a Head output layer in the YOLOv5 model, and generating a corresponding prediction frame to obtain a detection positioning result of the electronic element;
the feature recognition is performed through an SE attention mechanism, an SA attention mechanism and an ECA attention mechanism in a YOLOv5 model to obtain a new feature map with attention weight, which comprises the following steps:
carrying out global average pooling on the images in the preprocessed circuit board image dataset by detecting an SE attention mechanism in the positioning model to generate a first feature vector;
weighting the image from different positions of the channel dimension by using a weighting matrix through a full connection layer based on the first feature vector to obtain different first weight information, and carrying out weight assignment on the image through the first weight information to obtain a first feature map;
grouping the channel features in the first feature map to obtain a plurality of sub-features, adopting an SA attention mechanism in a detection positioning model to concentrate space and channel attention on each sub-feature, collecting the sub-features, and fusing the features of different groups through channel shuffling operation to obtain a second feature map;
calculating the average value of all pixels of the second feature map in each channel by detecting an ECA attention mechanism in a positioning model, and reducing the dimension of the second feature map by global average pooling operation to obtain a second feature vector;
performing one-dimensional convolution operation on the obtained second feature vector, activating through a Sigmoid function to obtain the weight of each channel, and performing product operation between the image in the preprocessed circuit board image dataset and the obtained weight of each channel corresponding to the channels to obtain a new feature map with attention weight;
the character recognition processing and correction processing are sequentially carried out on the electronic element characters on the circuit board image to obtain the character recognition result of the electronic element, and the character recognition method comprises the following steps:
acquiring electronic element characters on a circuit board image, and representing the electronic element characters and converting the electronic element characters into character vectors;
converting the character vector into a vector matrix by adopting a one-dimensional convolutional neural network, extracting the characteristics of the character, sliding the convolutional kernel up and down, and performing dot product operation to obtain characteristic values at different positions to obtain a characteristic vector sequence;
inputting the feature vector sequence into an LSTM (least squares) for extracting context features so as to obtain a character primary recognition result;
the method for obtaining the electronic element characters on the circuit board image, characterizing the electronic element characters, and before converting the electronic element characters into character vectors, further comprises the following steps:
corresponding background points of the circuit board image to polar coordinates, and calculating the inclination angle of the shadow circuit board image by using the accumulated value on the polar coordinates;
the nearest neighbor clustering method is based on extracting the center point of the connection domain of the circuit board image in a sub-domain of the circuit board image, and obtaining the inclination angle of the circuit board image through the continuous relation between the points on the base line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311305193.7A CN117058241B (en) | 2023-10-10 | 2023-10-10 | Electronic element positioning method and system based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311305193.7A CN117058241B (en) | 2023-10-10 | 2023-10-10 | Electronic element positioning method and system based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117058241A CN117058241A (en) | 2023-11-14 |
| CN117058241B true CN117058241B (en) | 2024-03-29 |
Family
ID=88666674
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311305193.7A Active CN117058241B (en) | 2023-10-10 | 2023-10-10 | Electronic element positioning method and system based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117058241B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017092427A1 (en) * | 2015-12-04 | 2017-06-08 | 广州视源电子科技股份有限公司 | Electronic element positioning method and apparatus |
| EP3218076A1 (en) * | 2014-11-10 | 2017-09-20 | Lego A/S | System and method for toy recognition |
| CN110909735A (en) * | 2019-11-07 | 2020-03-24 | 国网辽宁省电力有限公司电力科学研究院 | Intelligent electric energy meter chip visual identification comparison system and method based on convolutional neural network |
| CN113205511A (en) * | 2021-05-25 | 2021-08-03 | 中科芯集成电路有限公司 | Electronic component batch information detection method and system based on deep neural network |
| CN114092440A (en) * | 2021-11-22 | 2022-02-25 | 北京柏瑞安电子技术有限公司 | Printed circuit board non-standard element identification and positioning method and system based on vision |
| CN114769072A (en) * | 2022-06-16 | 2022-07-22 | 深圳徕科技术有限公司 | High-speed injection valve control method and device, electronic equipment and storage medium |
| CN116168017A (en) * | 2023-04-18 | 2023-05-26 | 南京信息工程大学 | Deep learning-based PCB element detection method, system and storage medium |
| CN116309427A (en) * | 2023-03-14 | 2023-06-23 | 盐城工学院 | PCB surface defect detection method based on improved YOLOv5 algorithm |
-
2023
- 2023-10-10 CN CN202311305193.7A patent/CN117058241B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3218076A1 (en) * | 2014-11-10 | 2017-09-20 | Lego A/S | System and method for toy recognition |
| WO2017092427A1 (en) * | 2015-12-04 | 2017-06-08 | 广州视源电子科技股份有限公司 | Electronic element positioning method and apparatus |
| CN110909735A (en) * | 2019-11-07 | 2020-03-24 | 国网辽宁省电力有限公司电力科学研究院 | Intelligent electric energy meter chip visual identification comparison system and method based on convolutional neural network |
| CN113205511A (en) * | 2021-05-25 | 2021-08-03 | 中科芯集成电路有限公司 | Electronic component batch information detection method and system based on deep neural network |
| CN114092440A (en) * | 2021-11-22 | 2022-02-25 | 北京柏瑞安电子技术有限公司 | Printed circuit board non-standard element identification and positioning method and system based on vision |
| CN114769072A (en) * | 2022-06-16 | 2022-07-22 | 深圳徕科技术有限公司 | High-speed injection valve control method and device, electronic equipment and storage medium |
| CN116309427A (en) * | 2023-03-14 | 2023-06-23 | 盐城工学院 | PCB surface defect detection method based on improved YOLOv5 algorithm |
| CN116168017A (en) * | 2023-04-18 | 2023-05-26 | 南京信息工程大学 | Deep learning-based PCB element detection method, system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117058241A (en) | 2023-11-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109993160B (en) | Image correction and text and position identification method and system | |
| CN111160352B (en) | Workpiece metal surface character recognition method and system based on image segmentation | |
| CN107133622B (en) | Word segmentation method and device | |
| CN110738207A (en) | character detection method for fusing character area edge information in character image | |
| CN103154974B (en) | Character recognition device, character recognition method and character recognition system | |
| CN111401372A (en) | Method for extracting and identifying image-text information of scanned document | |
| CN112085024A (en) | Tank surface character recognition method | |
| CN111639704A (en) | Target identification method, device and computer readable storage medium | |
| CN112381175A (en) | Circuit board identification and analysis method based on image processing | |
| CN106874913A (en) | A kind of vegetable detection method | |
| CN115810197A (en) | Multi-mode electric power form recognition method and device | |
| CN116758545A (en) | Paper medicine packaging steel seal character recognition method based on deep learning | |
| CN109800659B (en) | Action recognition method and device | |
| CN117058241B (en) | Electronic element positioning method and system based on artificial intelligence | |
| CN112200182A (en) | Deep learning-based wafer ID identification method and device | |
| US20230110558A1 (en) | Systems and methods for detecting objects | |
| Hwang et al. | Enabling product recognition and tracking based on text detection for mobile augmented reality | |
| CN111222355A (en) | Method and system for positioning bar code on PCB | |
| CN114494678A (en) | Character recognition method and electronic equipment | |
| CN114155540A (en) | Character recognition method, device and equipment based on deep learning and storage medium | |
| CN106557772A (en) | For extracting method, device and the image processing method of local feature | |
| Mohan et al. | Generation of Netlist from a Hand drawn Circuit through Image Processing and Machine Learning | |
| Zhu et al. | Chip surface character recognition based on improved LeNet-5 convolutional neural network | |
| Soora et al. | Object Recognition using Novel Geometrical Feature Extraction Techniques | |
| CN111553336B (en) | Print Uyghur document image recognition system and method based on link segment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |