CN112200164B - Integrated circuit metal round hole automatic identification method and system based on electron microscope image - Google Patents

Integrated circuit metal round hole automatic identification method and system based on electron microscope image Download PDF

Info

Publication number
CN112200164B
CN112200164B CN202011396899.5A CN202011396899A CN112200164B CN 112200164 B CN112200164 B CN 112200164B CN 202011396899 A CN202011396899 A CN 202011396899A CN 112200164 B CN112200164 B CN 112200164B
Authority
CN
China
Prior art keywords
image
round hole
integrated circuit
metal round
electron microscope
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011396899.5A
Other languages
Chinese (zh)
Other versions
CN112200164A (en
Inventor
沈丽君
洪贝
韩华
陈曦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Automation of Chinese Academy of Science
Original Assignee
Institute of Automation of Chinese Academy of Science
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Automation of Chinese Academy of Science filed Critical Institute of Automation of Chinese Academy of Science
Priority to CN202011396899.5A priority Critical patent/CN112200164B/en
Publication of CN112200164A publication Critical patent/CN112200164A/en
Application granted granted Critical
Publication of CN112200164B publication Critical patent/CN112200164B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/40—Document-oriented image-based pattern recognition
    • G06V30/42—Document-oriented image-based pattern recognition based on the type of document
    • G06V30/422—Technical drawings; Geographical maps
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/24323—Tree-organised classifiers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/30—Circuit design
    • G06F30/39—Circuit design at the physical level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Hardware Design (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Geometry (AREA)
  • Analysing Materials By The Use Of Radiation (AREA)
  • Image Processing (AREA)

Abstract

The invention belongs to the technical field of image recognition, and particularly relates to an integrated circuit metal round hole automatic recognition method and system based on an electron microscope image, aiming at solving the problems that the traditional image processing method has higher requirement on image quality when the integrated circuit metal round hole is recognized, the universality and high accuracy cannot be ensured, and an indirect logic judgment method needs to artificially define a judgment rule, and the accuracy and efficiency are lower. The invention comprises the following steps: performing characteristic enhancement on an electron microscope image of an integrated circuit to be analyzed; carrying out binarization on the enhanced image and carrying out morphological processing; converting the obtained candidate points of the metal round hole into candidate areas, and extracting gradient features; and carrying out characteristic classification prediction through the classification model to obtain round hole points of the electron microscope image of the integrated circuit to be analyzed. The invention can be used for automatically identifying and positioning the metal round hole structure of the integrated circuit in the electron microscope image, has high efficiency, good accuracy, high real-time performance and wide application range.

Description

Integrated circuit metal round hole automatic identification method and system based on electron microscope image
Technical Field
The invention belongs to the technical field of image recognition, and particularly relates to an integrated circuit metal round hole automatic recognition method and system based on an electron microscope image.
Background
Physical design analysis of integrated circuits is a process of reverse dissection, analysis and understanding of integrated circuits, and is of great significance to absorb existing advanced integrated circuit design experience and the like. Generally, the engineering can be divided into four steps of image acquisition and processing, circuit schematic diagram identification and extraction, layout identification and extraction, circuit comprehensive understanding and the like. The circuit schematic diagram identification and extraction aims to identify integrated circuit elements and connection relations among the elements presented on an integrated circuit microscopic image and restore the integrated circuit elements and the connection relations into a bottom layer circuit schematic diagram during design.
The premise of identifying and extracting the circuit schematic diagram is to accurately acquire the metal round holes on the integrated circuit, and by means of the high-speed development of an electron microscope, the microstructure on the integrated circuit can be observed more clearly. Therefore, how to automatically identify the metal circular holes of the integrated circuit efficiently and accurately is the key.
The existing method for identifying the metal round hole by the integrated circuit mainly comprises two methods: the first method of applying traditional image processing is to enhance the round hole points on the electron microscope image by an image enhancement technology, and then obtain the round hole points by threshold segmentation or directly performing threshold segmentation on the electron microscope image, but the method is established under the condition of better image quality, and aiming at large-scale data, because pollution caused by factors such as dust and the like introduced in the shooting of the electron microscope can reduce the identification accuracy, the effect of universality and high accuracy cannot be ensured; the second method is an indirect method, which has low efficiency and low accuracy, and adopts a logic judgment mode, which infers the position of the round holes of the integrated circuit according to the circuit board construction principle, because most of the round holes are at the end points of the linear structure and then are further screened out according to the hierarchical structure of the integrated circuit. Therefore, there is still a need in the art for an automatic high-precision identification method for identifying metal circular holes of an integrated circuit, so as to realize accurate and efficient structural analysis of the integrated circuit.
Disclosure of Invention
In order to solve the problems in the prior art that the traditional image processing method has higher requirements on image quality when identifying the metal round holes of the integrated circuit, the universality and the high accuracy cannot be ensured, and the indirect logic judgment method needs to artificially define judgment rules, so that the accuracy and the efficiency are lower, the invention provides an automatic identification method of the metal round holes of the integrated circuit based on the electron microscope image, which comprises the following steps:
step S10, acquiring an electron microscope image of the integrated circuit to be analyzed as an image to be processed;
step S20, performing enhancement processing on the image to be processed, enhancing the point structure in the image, and obtaining an enhanced image to be processed;
step S30, performing threshold processing for enhancing the image to be processed according to a preset first threshold, and performing morphological processing on a binary image obtained by the threshold processing to obtain a candidate point set of the metal round hole;
step S40, converting each metal round hole candidate point in the metal round hole candidate point set into a candidate region, and respectively calculating the gradient feature of each candidate region
Step S50, based on the gradient feature of the candidate region, performing feature classification prediction of the candidate region through a classification model, and taking the candidate region with the prediction result larger than a preset second threshold value as a round hole point of the image to be processed;
the classification model is a decision tree model which is constructed based on a classification decision tree and used for carrying out feature classification prediction by carrying out feature classification learning of a random forest with a supervised learning algorithm through a training sample with an artificial mark.
In some preferred embodiments, step S20 includes:
step S21, performing convolution operation of the image to be processed by adopting Gaussian function to obtain a preprocessing convolution matrix
Figure 581767DEST_PATH_IMAGE001
；
Step S22, for the preprocessing convolution matrix
Figure 348735DEST_PATH_IMAGE001
Performing second order derivative gradient image calculation, and acquiring enhanced image to be processed based on Hessian matrix eigenvalue
Figure 93837DEST_PATH_IMAGE002
。
In some preferred embodiments, the pre-processing convolution matrix
Figure 406001DEST_PATH_IMAGE001
The formula is expressed as:
Figure 970974DEST_PATH_IMAGE003
wherein,
Figure 276054DEST_PATH_IMAGE004
the parameters of the gaussian kernel are used as the parameters,
Figure 610083DEST_PATH_IMAGE005
the radius of the metal round hole in the image to be processed is shown;
Figure 582894DEST_PATH_IMAGE006
is the highest point of the Gaussian kernel function;
Figure 635164DEST_PATH_IMAGE007
representing in progress images
Figure 478355DEST_PATH_IMAGE007
Calculating points;
Figure 932470DEST_PATH_IMAGE008
is an electron microscope image of the integrated circuit to be analyzed.
In some preferred embodiments, step S22 includes:
step S221, based on the preprocessing convolution matrix
Figure 586436DEST_PATH_IMAGE001
Obtaining a Hessian matrix
Figure 126002DEST_PATH_IMAGE009
：
Figure 772884DEST_PATH_IMAGE010
Step S222, extracting the Hessian matrix
Figure 81506DEST_PATH_IMAGE009
Characteristic value of
Figure 404909DEST_PATH_IMAGE011
：
Figure 900612DEST_PATH_IMAGE012
Step S223 of calculating the Hessian matrix
Figure 351185DEST_PATH_IMAGE009
Characteristic value of
Figure 514313DEST_PATH_IMAGE011
To obtain an enhanced image to be processed
Figure 510082DEST_PATH_IMAGE002
：
Figure 24240DEST_PATH_IMAGE013
Wherein,
Figure 12924DEST_PATH_IMAGE004
the parameters of the gaussian kernel are used as the parameters,
Figure 30559DEST_PATH_IMAGE005
the radius of the metal round hole in the image to be processed is shown;
Figure 964273DEST_PATH_IMAGE006
is the highest point of the Gaussian kernel function;
Figure 700148DEST_PATH_IMAGE007
representing in progress images
Figure 164628DEST_PATH_IMAGE007
And (4) calculating points.
In some preferred embodiments, step S30 includes:
step S31, assigning the pixel which is larger than a preset third threshold value in the enhanced image to be processed as 1, and assigning the rest pixels as 0 to obtain an initial binary image;
step S32, assigning all pixels of the area smaller than the pixel size of the target area in the initial binary image to be 0 to obtain a final binary image;
and step S33, performing opening operation processing on the final binary image through a disc structure to obtain a metal round hole candidate point set.
In some preferred embodiments, in step S40, each metal round hole candidate point in the set of metal round hole candidate points is converted into a candidate region by:
step S411, expanding the candidate point coordinates into a rectangular frame through a preset fourth threshold for each metal round hole candidate point in the metal round hole candidate point set;
step S412, intercepting the image to be processed based on the rectangular frame corresponding to each metal round hole candidate point, and obtaining a candidate area corresponding to each metal round hole candidate point.
In some preferred embodiments, the gradient feature of each candidate region is calculated in step S40, respectively, by:
step S421, standardizing the candidate area by a Gamma correction method to obtain a standardized candidate area;
step S422, defining a horizontal gradient template
Figure 161402DEST_PATH_IMAGE014
And vertical gradient formwork
Figure 889187DEST_PATH_IMAGE015
And calculating each point in the standardized candidate region through the imfilter operator function
Figure 987724DEST_PATH_IMAGE016
Are respectively at
Figure 193578DEST_PATH_IMAGE017
Amount of change in direction
Figure 310438DEST_PATH_IMAGE018
And
Figure 209124DEST_PATH_IMAGE019
obtaining a point
Figure 27913DEST_PATH_IMAGE016
Gradient change of (2)
Figure 834195DEST_PATH_IMAGE020
And gradient angle
Figure 821874DEST_PATH_IMAGE021
；
Step S423, based on the point
Figure 625882DEST_PATH_IMAGE016
Gradient change of (2)
Figure 948279DEST_PATH_IMAGE020
And gradient angle
Figure 292672DEST_PATH_IMAGE021
Obtaining a gradient matrix
Figure 259491DEST_PATH_IMAGE022
And direction matrix
Figure 611232DEST_PATH_IMAGE023
；
Step S424, dividing the candidate area into cells with set pixel size according to the block parameters, and combining the gradient matrix
Figure 296291DEST_PATH_IMAGE022
And direction matrix
Figure 772272DEST_PATH_IMAGE023
In the direction gradient of each cellThe graph is taken as the characteristic of the corresponding cell;
step S425 is to use the set number of cells as one block, superimpose the features of all the cells in each block as the features of the corresponding block, and superimpose the features of all the blocks as the gradient features of the candidate region.
In some preferred embodiments, the gradient change amount
Figure 593597DEST_PATH_IMAGE020
The formula is expressed as:
Figure 880353DEST_PATH_IMAGE024
wherein,
Figure 787129DEST_PATH_IMAGE025
and
Figure 801222DEST_PATH_IMAGE026
representative point
Figure 742633DEST_PATH_IMAGE016
Are respectively at
Figure 167667DEST_PATH_IMAGE017
The amount of change in direction.
In some preferred embodiments, the gradient angle
Figure 561739DEST_PATH_IMAGE021
The formula is expressed as:
Figure 379522DEST_PATH_IMAGE027
wherein,
Figure 175440DEST_PATH_IMAGE025
and
Figure 803999DEST_PATH_IMAGE026
representative point
Figure 685367DEST_PATH_IMAGE016
Are respectively at
Figure 41262DEST_PATH_IMAGE017
The amount of change in direction.
In another aspect of the present invention, an integrated circuit metal round hole automatic identification system based on an electron microscope image is provided, which includes the following modules:
the image acquisition module is configured to acquire an electron microscope image of the integrated circuit to be analyzed as an image to be processed;
the image enhancement module is configured to enhance the image to be processed, strengthen the point structure in the image and obtain an enhanced image to be processed;
the threshold processing module is configured to perform threshold processing for enhancing the image to be processed according to a preset first threshold, and perform morphological processing on a binary image obtained by the threshold processing to obtain a candidate point set of the metal round hole;
the characteristic extraction module is configured to convert each metal round hole candidate point in the metal round hole candidate point set into a candidate region and respectively calculate the gradient characteristic of each candidate region;
the classification prediction module is configured to perform feature classification prediction on the candidate region through a classification model based on the gradient feature of the candidate region, and take the candidate region with the prediction result larger than a preset second threshold value as a round hole point of the image to be processed;
the classification model is a decision tree model which is constructed based on a classification decision tree and used for carrying out feature classification prediction by carrying out feature classification learning of a random forest with a supervised learning algorithm through a training sample with an artificial mark.
The invention has the beneficial effects that:
(1) the method for automatically identifying the metal round holes of the integrated circuit based on the electron microscope image sets Gaussian kernel parameters of a Gaussian function by combining the radius of the metal round holes in the electron microscope image of the integrated circuit to be analyzed, adopts the Gaussian function to simulate the section strength form of the round hole image, infers the corresponding relation, selects proper Gaussian distribution as a basis to enhance the hole-shaped structure, can clearly present the hole structure, selects proper number of round hole candidate points, and further improves the precision and accuracy of subsequent round hole identification.
(2) According to the integrated circuit metal round hole automatic identification method based on the electron microscope image, the standardization of the image color space is carried out through a Gamma correction method, the interference of inconsistent contrast in electron microscope shooting is reduced, the accuracy and precision of image feature extraction are improved, and the accuracy and precision of subsequent round hole identification are further improved.
(3) According to the integrated circuit metal round hole automatic identification method based on the electron microscope image, the end point of the line structure in the circuit board is locally similar to the round hole structure, and the identification precision cannot be guaranteed only according to the image enhancement method.
(4) According to the method for automatically identifying the metal circular holes of the integrated circuit based on the electron microscope image, the experimental results of circuit boards (m 1, m2 and m 3) of different models show that the comprehensive precision can reach more than 97%, the artificial correction time is greatly shortened, and the time for analyzing the bottom layer circuit schematic diagram of the integrated circuit is accelerated.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is a schematic flow chart of the method for automatically identifying a metal circular hole of an integrated circuit based on an electron microscope image according to the present invention;
FIG. 2 is an exemplary diagram of an integrated circuit under an electron microscope according to an embodiment of the method for automatically identifying a metal circular hole of an integrated circuit based on an electron microscope image;
FIG. 3 is an enhanced diagram of an integrated circuit according to an embodiment of the method for automatically identifying a metal circular hole of an integrated circuit based on an electron microscope image;
fig. 4 is a diagram illustrating the detection effect of the metal round hole of the integrated circuit according to an embodiment of the method for automatically identifying the metal round hole of the integrated circuit based on the electron microscope image.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
The invention discloses an integrated circuit metal round hole automatic identification method based on an electron microscope image, which comprises the following steps:
step S10, acquiring an electron microscope image of the integrated circuit to be analyzed as an image to be processed;
step S20, performing enhancement processing on the image to be processed, enhancing the point structure in the image, and obtaining an enhanced image to be processed;
step S30, performing threshold processing for enhancing the image to be processed according to a preset first threshold, and performing morphological processing on a binary image obtained by the threshold processing to obtain a candidate point set of the metal round hole;
step S40, converting each metal round hole candidate point in the metal round hole candidate point set into a candidate region, and respectively calculating the gradient feature of each candidate region
Step S50, based on the gradient feature of the candidate region, performing feature classification prediction of the candidate region through a classification model, and taking the candidate region with the prediction result larger than a preset second threshold value as a round hole point of the image to be processed;
the classification model is a decision tree model which is constructed based on a classification decision tree and used for carrying out feature classification prediction by carrying out feature classification learning of a random forest with a supervised learning algorithm through a training sample with an artificial mark.
In order to more clearly describe the method for automatically identifying a metal circular hole of an integrated circuit based on an electron microscope image, the following describes in detail the steps in the embodiment of the present invention with reference to fig. 1.
The method for automatically identifying the metal round hole of the integrated circuit based on the electron microscope image comprises the following steps of S10-S50, wherein the steps are described in detail as follows:
and step S10, acquiring the electron microscope image of the integrated circuit to be analyzed as the image to be processed.
Referring to fig. 2, a schematic diagram of an embodiment of an electronic microscope for automatically identifying metal circular holes of an integrated circuit according to the present invention is shown, wherein the white hole-shaped structure is a circular hole in the integrated circuit, the gray portion is a predetermined connection line in the integrated circuit, and the black portion is a substrate of the integrated circuit.
And step S20, performing enhancement processing on the image to be processed, and enhancing the point structure in the image to obtain an enhanced image to be processed.
Step S21, performing convolution operation of the image to be processed by adopting Gaussian function to obtain a preprocessing convolution matrix
Figure 691686DEST_PATH_IMAGE001
As shown in formula (1):
Figure 412517DEST_PATH_IMAGE028
wherein,
Figure 158013DEST_PATH_IMAGE004
the parameters of the gaussian kernel are used as the parameters,
Figure 192965DEST_PATH_IMAGE005
the radius of the metal round hole in the image to be processed is shown;
Figure 822529DEST_PATH_IMAGE006
is the highest point of the Gaussian kernel function;
Figure 917524DEST_PATH_IMAGE007
representing in progress images
Figure 648851DEST_PATH_IMAGE007
Calculating points;
Figure 221915DEST_PATH_IMAGE008
is an electron microscope image of the integrated circuit to be analyzed.
Step S22, for the preprocessing convolution matrix
Figure 971565DEST_PATH_IMAGE001
Performing second order derivative gradient image calculation, and acquiring enhanced image to be processed based on Hessian matrix eigenvalue
Figure 237461DEST_PATH_IMAGE002
。
Step S221, based on the preprocessing convolution matrix
Figure 689040DEST_PATH_IMAGE001
Obtaining a Hessian matrix
Figure 65795DEST_PATH_IMAGE009
As shown in formula (2):
Figure 669952DEST_PATH_IMAGE029
wherein,
Figure 106749DEST_PATH_IMAGE030
、
Figure 812668DEST_PATH_IMAGE031
、
Figure 461955DEST_PATH_IMAGE032
and
Figure 186198DEST_PATH_IMAGE033
the calculation methods of (a) are respectively shown in the formulas (3) to (5):
Figure 793897DEST_PATH_IMAGE034
Figure 234716DEST_PATH_IMAGE035
Figure 953273DEST_PATH_IMAGE036
step S222, extracting the Hessian matrix
Figure 532022DEST_PATH_IMAGE009
Characteristic value of
Figure 310622DEST_PATH_IMAGE011
As shown in formula (6):
Figure 725554DEST_PATH_IMAGE012
Figure 44540DEST_PATH_IMAGE037
step S223 of calculating the Hessian matrix
Figure 681058DEST_PATH_IMAGE009
Characteristic value of
Figure 630559DEST_PATH_IMAGE011
To obtain an enhanced image to be processed
Figure 188580DEST_PATH_IMAGE002
As shown in formula (7):
Figure 888420DEST_PATH_IMAGE038
as shown in fig. 3, which is an enhanced diagram of an integrated circuit according to an embodiment of the method for automatically identifying a metal circular hole of an integrated circuit based on an electron microscope image, it can be seen that the method of the present invention can enhance the metal circular hole portion of the integrated circuit well, and other possible disturbing circular points are further weakened.
Step S30, performing threshold processing for enhancing the image to be processed according to a preset first threshold, and performing morphological processing on the binary image obtained by the threshold processing to obtain a candidate point set of the metal round hole.
Step S31, assigning the pixel which is larger than a preset third threshold value in the enhanced image to be processed as 1, and assigning the rest pixels as 0 to obtain an initial binary image;
step S32, assigning all pixels of the area smaller than the pixel size of the target area in the initial binary image to be 0 to obtain a final binary image;
and step S33, performing opening operation processing on the final binary image through a disc structure to obtain a metal round hole candidate point set.
The disc radius of the selected disc structure is selected according to requirements, and a parameter between 3 and 6 pixels is generally selected. And then recording coordinate information of each connected domain, namely the central coordinates of the target connected domain, by adopting a regionprops function in Matlab, wherein the central coordinates are the coordinates of the round hole candidate points.
Step S40, converting each metal round hole candidate point in the set of metal round hole candidate points into a candidate region, and calculating a gradient feature of each candidate region respectively.
Step S411, for each metal round hole candidate point in the metal round hole candidate point set, expanding the candidate point coordinates into a rectangular frame through a preset fourth threshold, where the preset fourth threshold is
Figure 51548DEST_PATH_IMAGE039
Then, the rectangular frame area is represented by formula (8):
Figure 296585DEST_PATH_IMAGE040
step S412, intercepting the image to be processed based on the rectangular frame corresponding to each metal round hole candidate point, and obtaining a candidate area corresponding to each metal round hole candidate point.
Electron microscopy images on an integrated circuit to be analyzedIAnd (4) taking the image covered by the rectangular frame area as a candidate area of the round hole candidate point, and taking the candidate area as the basis of the next accurate classification. Specifically, for a truncated candidate region, we extract the histogram of oriented gradients (hog) feature of the region based on the following steps, which is used to calculate the directional density distribution of the gradients or edges as the appearance and shape of the characterizing local object.
Step S421, standardizing the candidate area by a Gamma correction method to obtain a standardized candidate area;
step S422, defining a horizontal gradient template
Figure 810743DEST_PATH_IMAGE014
And vertical gradient formwork
Figure 550160DEST_PATH_IMAGE015
And calculating each point in the standardized candidate region through the imfilter operator function
Figure 302215DEST_PATH_IMAGE016
Are respectively at
Figure 983732DEST_PATH_IMAGE017
Amount of change in direction
Figure 719607DEST_PATH_IMAGE018
And
Figure 498600DEST_PATH_IMAGE019
obtaining a point
Figure 636321DEST_PATH_IMAGE016
Gradient change of (2)
Figure 488739DEST_PATH_IMAGE020
And gradient angle
Figure 446331DEST_PATH_IMAGE021
；
Amount of gradient change
Figure 793130DEST_PATH_IMAGE020
As shown in formula (9):
Figure 785356DEST_PATH_IMAGE041
wherein,
Figure 543097DEST_PATH_IMAGE025
and
Figure 253564DEST_PATH_IMAGE026
representative point
Figure 637010DEST_PATH_IMAGE016
Are respectively at
Figure 483743DEST_PATH_IMAGE017
The amount of change in direction.
Gradient angle
Figure 350068DEST_PATH_IMAGE021
As shown in equation (10):
Figure 406885DEST_PATH_IMAGE042
step S423, based on the point
Figure 220121DEST_PATH_IMAGE016
Gradient change of (2)
Figure 62306DEST_PATH_IMAGE020
And gradient angle
Figure 37215DEST_PATH_IMAGE021
Obtaining a gradient matrix
Figure 846908DEST_PATH_IMAGE022
And direction matrix
Figure 198255DEST_PATH_IMAGE023
；
Step S424, dividing the candidate area into cells with set pixel size according to the block parameters, and combining the gradient matrix
Figure 396412DEST_PATH_IMAGE022
And direction matrix
Figure 542222DEST_PATH_IMAGE023
Taking the directional gradient histogram of each cell as the feature of the corresponding cell;
step S425 is to use the set number of cells as one block, superimpose the features of all the cells in each block as the features of the corresponding block, and superimpose the features of all the blocks as the gradient features of the candidate region.
And step S50, based on the gradient features of the candidate region, performing feature classification prediction of the candidate region through a classification model, and taking the candidate region of which the prediction result is greater than a preset second threshold value as a round hole point of the image to be processed.
And performing feature classification prediction on all the extracted candidate regions by using a trained classification model, and taking the candidate region of which the final prediction result is greater than a preset second threshold value as a finally accurately reserved round hole point. In a preferred embodiment of the present invention, the preset second threshold is 0.5.
The classification model is a decision tree model which is constructed based on a classification decision tree and used for carrying out feature classification prediction by carrying out feature classification learning of a random forest with a supervised learning algorithm through a training sample with an artificial mark.
In the classification model training process, aiming at a batch of electron microscope data, a plurality of training samples are marked manually, wherein the training samples are a series of point sets, and the point sets correspond to two categories which are respectively represented as a round hole area point 1 and a non-round hole area point 0. And calculating regional gradient characteristics of the artificially marked point set through the process, and performing characteristic classification learning on the random forest by using a supervised learning algorithm to obtain a classification decision tree (TreeBagger function of Matlab) serving as a classification model for automatically identifying the metal round holes of the integrated circuit.
As shown in fig. 4, which is a diagram illustrating the detection effect of the metal round hole of the integrated circuit according to an embodiment of the method for automatically identifying the metal round hole of the integrated circuit based on the electron microscope image, the white hole-shaped structure in the diagram is the round hole in the integrated circuit, and the black dots represent the identification result of the method of the present invention.
The integrated circuit metal round hole automatic identification system based on the electron microscope image comprises the following modules:
the image acquisition module is configured to acquire an electron microscope image of the integrated circuit to be analyzed as an image to be processed;
the image enhancement module is configured to enhance the image to be processed, strengthen the point structure in the image and obtain an enhanced image to be processed;
the threshold processing module is configured to perform threshold processing for enhancing the image to be processed according to a preset first threshold, and perform morphological processing on a binary image obtained by the threshold processing to obtain a candidate point set of the metal round hole;
the characteristic extraction module is configured to convert each metal round hole candidate point in the metal round hole candidate point set into a candidate region and respectively calculate the gradient characteristic of each candidate region;
the classification prediction module is configured to perform feature classification prediction on the candidate region through a classification model based on the gradient feature of the candidate region, and take the candidate region with the prediction result larger than a preset second threshold value as a round hole point of the image to be processed;
the classification model is a decision tree model which is constructed based on a classification decision tree and used for carrying out feature classification prediction by carrying out feature classification learning of a random forest with a supervised learning algorithm through a training sample with an artificial mark.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiments, and will not be described herein again.
It should be noted that, the integrated circuit metal round hole automatic identification system based on an electron microscope image provided in the foregoing embodiment is only illustrated by dividing the functional modules, and in practical applications, the functions may be allocated to different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
A storage device according to a third embodiment of the present invention stores a plurality of programs, and the programs are suitable for being loaded and executed by a processor to implement the above-mentioned method for automatically identifying a metal circular hole of an integrated circuit based on an electron microscope image.
A processing apparatus according to a fourth embodiment of the present invention includes a processor, a storage device; a processor adapted to execute various programs; a storage device adapted to store a plurality of programs; the program is suitable for being loaded and executed by a processor to realize the automatic identification method of the integrated circuit metal round hole based on the electron microscope image.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Those of skill in the art would appreciate that the various illustrative modules, method steps, and modules described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the software modules, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.

Claims (7)

1. An integrated circuit metal round hole automatic identification method based on electron microscope images is characterized by comprising the following steps:
step S10, acquiring an electron microscope image of the integrated circuit to be analyzed as an image to be processed;
step S20, performing convolution operation of the image to be processed by adopting Gaussian function to obtain a preprocessing convolution matrix
Figure DEST_PATH_IMAGE001
：
Figure DEST_PATH_IMAGE003
Based on the pre-processing convolution matrix
Figure 231287DEST_PATH_IMAGE001
Obtaining a Hessian matrix
Figure 438277DEST_PATH_IMAGE004
：
Figure 20568DEST_PATH_IMAGE006
Extracting the Hessian matrix
Figure 414640DEST_PATH_IMAGE004
Characteristic value of
Figure DEST_PATH_IMAGE007
：
Figure DEST_PATH_IMAGE009
Computing the Hessian matrix
Figure 574968DEST_PATH_IMAGE004
Characteristic value of
Figure 370886DEST_PATH_IMAGE007
To obtain an enhanced image to be processed
Figure 124078DEST_PATH_IMAGE010
：
Figure 271026DEST_PATH_IMAGE012
Wherein,
Figure DEST_PATH_IMAGE013
the parameters of the gaussian kernel are used as the parameters,
Figure 751555DEST_PATH_IMAGE014
the radius of the metal round hole in the image to be processed is shown;
Figure DEST_PATH_IMAGE015
is the highest point of the Gaussian kernel function;
Figure 870820DEST_PATH_IMAGE016
representing in progress images
Figure 794914DEST_PATH_IMAGE016
Calculating points;
Figure DEST_PATH_IMAGE017
an electron microscope image of the integrated circuit to be analyzed;
step S30, performing threshold processing for enhancing the image to be processed according to a preset first threshold, and performing morphological processing on a binary image obtained by the threshold processing to obtain a candidate point set of the metal round hole;
step S40, converting each metal round hole candidate point in the metal round hole candidate point set into a candidate region, and respectively calculating the gradient feature of each candidate region;
step S50, based on the gradient feature of the candidate region, performing feature classification prediction of the candidate region through a classification model, and taking the candidate region with the prediction result larger than a preset second threshold value as a round hole point of the image to be processed;
the classification model is a decision tree model which is constructed based on a classification decision tree and used for carrying out feature classification prediction, and the classification model is based on training samples with artificial marks and adopts a random forest algorithm to carry out supervised feature classification learning.
2. The method for automatically identifying the metal circular hole of the integrated circuit based on the electron microscope image as claimed in claim 1, wherein the step S30 includes:
step S31, assigning the pixel which is larger than a preset third threshold value in the enhanced image to be processed as 1, and assigning the rest pixels as 0 to obtain an initial binary image;
step S32, assigning all pixels of the area smaller than the pixel size of the target area in the initial binary image to be 0 to obtain a final binary image;
and step S33, performing opening operation processing on the final binary image through a disc structure to obtain a metal round hole candidate point set.
3. The method for automatically identifying the metal round holes of the integrated circuit based on the electron microscope image as claimed in claim 1, wherein in step S40, each metal round hole candidate point in the set of metal round hole candidate points is converted into a candidate region by:
step S411, expanding the candidate point coordinates into a rectangular frame through a preset fourth threshold for each metal round hole candidate point in the metal round hole candidate point set;
step S412, intercepting the image to be processed based on the rectangular frame corresponding to each metal round hole candidate point, and obtaining a candidate area corresponding to each metal round hole candidate point.
4. The method for automatically identifying the metal circular holes of the integrated circuit based on the electron microscope image as claimed in claim 1, wherein the step S40 is to calculate the gradient feature of each candidate region respectively, and the method comprises:
step S421, standardizing the candidate area by a Gamma correction method to obtain a standardized candidate area;
step S422, defining a horizontal gradient template
Figure 881688DEST_PATH_IMAGE018
And vertical gradient formwork
Figure DEST_PATH_IMAGE019
And calculating each point in the standardized candidate region through the imfilter operator function
Figure 651061DEST_PATH_IMAGE020
Are respectively at
Figure DEST_PATH_IMAGE021
Amount of change in direction
Figure 405259DEST_PATH_IMAGE022
And
Figure DEST_PATH_IMAGE023
obtaining a point
Figure 969095DEST_PATH_IMAGE020
Gradient change of (2)
Figure 825056DEST_PATH_IMAGE024
And gradient angle
Figure DEST_PATH_IMAGE025
；
Step S423, based on the point
Figure 650317DEST_PATH_IMAGE020
Gradient change of (2)
Figure 9754DEST_PATH_IMAGE024
And gradient angle
Figure 541230DEST_PATH_IMAGE025
Obtaining a gradient matrix
Figure 618907DEST_PATH_IMAGE026
And direction matrix
Figure DEST_PATH_IMAGE027
；
Step S424, dividing the candidate area into cells with set pixel size according to the block parameters, and combining the gradient matrix
Figure 244929DEST_PATH_IMAGE026
And direction matrix
Figure 990031DEST_PATH_IMAGE027
Taking the directional gradient histogram of each cell as the feature of the corresponding cell;
step S425 is to use the set number of cells as one block, superimpose the features of all the cells in each block as the features of the corresponding block, and superimpose the features of all the blocks as the gradient features of the candidate region.
5. The method for automatically identifying the metal circular hole of the integrated circuit based on the electron microscope image as claimed in claim 4, wherein the gradient variation is
Figure 426829DEST_PATH_IMAGE024
The formula is expressed as:
Figure 991803DEST_PATH_IMAGE028
。
6. the base of claim 4The method for automatically identifying the metal round hole of the integrated circuit on the electron microscope image is characterized in that the gradient angle
Figure 437827DEST_PATH_IMAGE025
The formula is expressed as:
Figure DEST_PATH_IMAGE029
。
7. the utility model provides an integrated circuit metal round hole automatic identification system based on electron microscope image which characterized in that, this system includes following module:
the image acquisition module is configured to acquire an electron microscope image of the integrated circuit to be analyzed as an image to be processed;
an image enhancement module configured to perform a convolution operation of the image to be processed by using a Gaussian function to obtain a pre-processing convolution matrix
Figure 755545DEST_PATH_IMAGE001
：
Figure 363244DEST_PATH_IMAGE030
Based on the pre-processing convolution matrix
Figure 415514DEST_PATH_IMAGE001
Obtaining a Hessian matrix
Figure 134071DEST_PATH_IMAGE004
：
Figure DEST_PATH_IMAGE031
Extracting the Hessian matrix
Figure 571874DEST_PATH_IMAGE004
Characteristic value of
Figure 616054DEST_PATH_IMAGE007
：
Figure 155619DEST_PATH_IMAGE032
Computing the Hessian matrix
Figure 943447DEST_PATH_IMAGE004
Characteristic value of
Figure 504266DEST_PATH_IMAGE007
To obtain an enhanced image to be processed
Figure 453767DEST_PATH_IMAGE010
：
Figure DEST_PATH_IMAGE033
Wherein,
Figure 949471DEST_PATH_IMAGE013
the parameters of the gaussian kernel are used as the parameters,
Figure 540989DEST_PATH_IMAGE014
the radius of the metal round hole in the image to be processed is shown;
Figure 953385DEST_PATH_IMAGE015
is the highest point of the Gaussian kernel function;
Figure 339367DEST_PATH_IMAGE016
representing in progress images
Figure 853525DEST_PATH_IMAGE016
Calculating points;
Figure 717575DEST_PATH_IMAGE017
an electron microscope image of the integrated circuit to be analyzed;
the threshold processing module is configured to perform threshold processing for enhancing the image to be processed according to a preset first threshold, and perform morphological processing on a binary image obtained by the threshold processing to obtain a candidate point set of the metal round hole;
the characteristic extraction module is configured to convert each metal round hole candidate point in the metal round hole candidate point set into a candidate region and respectively calculate the gradient characteristic of each candidate region;
the classification prediction module is configured to perform feature classification prediction on the candidate region through a classification model based on the gradient feature of the candidate region, and take the candidate region with the prediction result larger than a preset second threshold value as a round hole point of the image to be processed;
the classification model is a decision tree model which is constructed based on a classification decision tree and used for carrying out feature classification prediction, and the classification model is based on training samples with artificial marks and adopts a random forest algorithm to carry out supervised feature classification learning.
CN202011396899.5A 2020-12-03 2020-12-03 Integrated circuit metal round hole automatic identification method and system based on electron microscope image Active CN112200164B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011396899.5A CN112200164B (en) 2020-12-03 2020-12-03 Integrated circuit metal round hole automatic identification method and system based on electron microscope image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011396899.5A CN112200164B (en) 2020-12-03 2020-12-03 Integrated circuit metal round hole automatic identification method and system based on electron microscope image

Publications (2)

Publication Number Publication Date
CN112200164A CN112200164A (en) 2021-01-08
CN112200164B true CN112200164B (en) 2021-02-23

Family

ID=74033744

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011396899.5A Active CN112200164B (en) 2020-12-03 2020-12-03 Integrated circuit metal round hole automatic identification method and system based on electron microscope image

Country Status (1)

Country Link
CN (1) CN112200164B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115223034A (en) * 2021-04-16 2022-10-21 中国科学院上海药物研究所 Automatic hole selection method and device for cryoelectron microscope

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109615609A (en) * 2018-11-15 2019-04-12 北京航天自动控制研究所 A kind of solder joint flaw detection method based on deep learning
CN111006608A (en) * 2019-12-17 2020-04-14 湖南科创信息技术股份有限公司 Transparent plate through hole visual detection system and method

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9659359B1 (en) * 2015-05-05 2017-05-23 Flextronics Ap, Llc Method of and device for quality control process optimization
CN106327496B (en) * 2016-08-26 2019-04-23 西安电子科技大学 The detection system and method for PCB bare board blind hole defect based on AOI
CN107169528B (en) * 2017-06-06 2020-01-07 成都芯云微电子有限公司 Through hole recognition device of integrated circuit image based on machine learning
CN107392900B (en) * 2017-07-24 2020-07-28 太原理工大学 Multi-scale enhancement method for lung nodule image
CN108154502B (en) * 2017-12-22 2022-01-11 王华锋 Through hole welding spot identification method based on convolutional neural network
CN109859181A (en) * 2019-01-29 2019-06-07 桂林电子科技大学 A kind of PCB welding point defect detection method
CN110610199B (en) * 2019-08-22 2022-09-13 南京理工大学 Automatic optical detection method for printed circuit board resistance element welding spot based on svm and xgboost

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109615609A (en) * 2018-11-15 2019-04-12 北京航天自动控制研究所 A kind of solder joint flaw detection method based on deep learning
CN111006608A (en) * 2019-12-17 2020-04-14 湖南科创信息技术股份有限公司 Transparent plate through hole visual detection system and method

Also Published As

Publication number Publication date
CN112200164A (en) 2021-01-08

Similar Documents

Publication Publication Date Title
CN110349126B (en) Convolutional neural network-based marked steel plate surface defect detection method
CN113780259B (en) Road surface defect detection method and device, electronic equipment and readable storage medium
CN109658381B (en) Method for detecting copper surface defects of flexible IC packaging substrate based on super-pixels
CN109509187B (en) Efficient inspection algorithm for small defects in large-resolution cloth images
CN106909941A (en) Multilist character recognition system and method based on machine vision
CN108335311B (en) Method for extracting morphological structure characteristics of tobacco leaves, electronic equipment and storage medium
CN110210387B (en) Method, system and device for detecting insulator target based on knowledge graph
CN111540006B (en) Plant stomata intelligent detection and identification method and system based on deep migration learning
CN110942013A (en) Satellite image feature extraction method and system based on deep neural network
CN103940708B (en) A kind of Quick Measurement, sophisticated category method of steel holotype state crystal grain
CN108009567B (en) Automatic excrement character distinguishing method combining image color and HOG and SVM
CN112200164B (en) Integrated circuit metal round hole automatic identification method and system based on electron microscope image
CN110889374A (en) Seal image processing method and device, computer and storage medium
CN113609984A (en) Pointer instrument reading identification method and device and electronic equipment
CN104881668B (en) A kind of image fingerprint extracting method and system based on representative local mode
CN107133964B (en) Image matting method based on Kinect
CN109507193A (en) A kind of fabric defects detection method based on local contrast enhancing and binary pattern
CN114429577A (en) Flag detection method, system and equipment based on high beacon strategy
CN107145888A (en) Video caption real time translating method
CN114627113B (en) Method, system, device and medium for detecting defects of printed circuit board
CN111707672A (en) Method for detecting surface defects of wind power rotary supporting piece
CN105046264B (en) A kind of sparse terrain classification and marker method based on visible ray and lidar image
CN109615630A (en) Semi-continuous casting alusil alloy Analysis on Microstructure method based on image processing techniques
CN115619799A (en) Grain image segmentation method and system based on transfer learning
CN116434054A (en) Intensive remote sensing ground object extraction method based on line-plane combination

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant