TW202323807A - Computer architecture for through glass via defect inspection - Google Patents

Abstract

A computing device controls an illumination source and an imaging device to generate images of through glass vias (TGVs). The computing device generates images of the TGVs based on the scattering image signals detected by the imaging device. The computing device computes coordinates of the TGVs within the images based on the scattering image signals detected by the imaging device. The computing device generates, for one or more TGVs in the images and based on the computed coordinates, a TGV type classification, wherein the TGV type classification indicated whether the TGV has or lacks a defect and, if the TGV has a defect, a defect type. The computing device determines, based on the generated TGV type classifications, that multiple TGVs have a defect associated with a given defect type. The computing device identifies a manufacturing machine setting associated with the given defect type.

Description

Computer Architecture for Glass Via Defect Inspection

This application claims the benefit of priority under the Patents Act to U.S. Provisional Patent Application No. 63/211,717, filed June 17, 2021, which is hereby attached and incorporated by reference in its entirety.

Embodiments relate to computer architecture. Some embodiments relate to artificial intelligence. Some embodiments relate to computer architecture for glass via defect inspection.

Manufactured through glass vias (TGVs) are often defective. Techniques for identifying defective TGVs may be desirable.

none

The following description and drawings sufficiently illustrate specific embodiments to enable those skilled in the art to practice them. Other embodiments may incorporate structural, logical, electrical, process, and other changes. Portions and features of some embodiments may be included in, or substituted for, those of other embodiments. The embodiments set forth in the claims encompass all available equivalents of those claims.

Aspects of the technology can be implemented as part of a computer system. The computer system may be a physical machine, or may be distributed among multiple physical machines, for example by role or function, or by process threads in the case of a cloud computing distributed model. In various embodiments, aspects of the technology may be configured to run on virtual machines that in turn execute on one or more physical machines. Those skilled in the art will appreciate that the features of the present technology may be implemented in various suitable machine implementations.

The system includes various engines, each of which is structured, programmed, configured, or otherwise adapted to implement a function or set of functions. As used herein, the term "engine" means implemented using hardware (such as implemented by an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA)) or implemented as a combination of hardware and software A tangible device, component, or arrangement of components (implemented, for example, by a processor-based computing platform and a set of programming instructions that transform the computing platform into a special-purpose device to implement specific functionality). Engines can also be implemented as a combination of both, where some functions are driven only by hardware, while other functions are driven by a combination of hardware and software.

In an example, the software may reside on a tangible machine-readable storage medium in executable or non-executable form. Software that resides in a non-executable form may be compiled, translated, or otherwise converted to an executable form either before or during runtime. In an example, the software, when executed by the engine's underlying hardware, causes the hardware to perform specified operations. Accordingly, an engine is physically constructed or specifically configured (eg, hardwired) or temporarily configured (eg, programmed) to operate in a specified manner or to perform some or all of any operations described herein with respect to that engine.

Considering the example of temporarily configuring engines, each of the engines may be instantiated at different times. For example, where an engine includes a general-purpose hardware processor core configured using software, the general-purpose hardware processor core may be configured at different times as corresponding different engines. Software may accordingly configure the hardware processor cores to, for example, constitute a particular engine at one instance of time and a different engine at a different instance of time.

In certain embodiments, at least a portion (and in some cases, all) of the engine may execute on the processor(s) of one or more computers that execute the operating system, system programs, and applications while also using multitasking, Multi-threaded, distributed (eg, cluster, peer-peer, cloud, etc.) processing (where appropriate), or other such technologies to implement the engine. As such, each engine may be implemented in various suitable configurations, and should generally not be limited to any particular implementation illustrated herein, unless such limitation is expressly indicated.

Furthermore, an engine itself may be composed of more than one sub-engine, each of which may be considered an engine itself. Furthermore, in the embodiments described herein, each of the various engines corresponds to a defined functionality; however, it should be understood that in other contemplated embodiments, each functionality may also be distributed to more than one engine. Likewise, in other contemplated embodiments, multiple defined functionality may be implemented by a single engine that performs those multiple functions, possibly in conjunction with other functions, or multiple defined functionality may be implemented as Ways different from those specified by the examples in this article are distributed across the engine collection.

As used herein, the term "model" includes its simple and ordinary meaning. In particular, a model may include one or more engines that receive input and compute output based on the input. The output can be a classification. For example, a video file may be classified as depicting a cat or not depicting a cat. Alternatively, image files may be assigned a numerical score indicating the likelihood that the image file depicts a cat, and image files with scores exceeding a threshold (eg, 0.9 or 0.95) may be determined to depict cats.

This document may refer to a specific number of things (eg "six mobile devices"). Unless expressly stated otherwise, the quantities provided are examples only and may be replaced by any positive integer, integer or real number as it makes sense for the given situation. For example, in an alternative embodiment, "six mobile devices" includes any positive integer number of mobile devices. Unless stated otherwise, references to something in the singular (eg "computer" or "the computer") may include one or more items (eg "the computer" may refer to one or more computers).

Figure 1 illustrates the training and use of a machine learning program in accordance with some example embodiments. In some example embodiments, a machine learning program (MLP) (also referred to as a machine learning algorithm or tool) is used to perform operations associated with a machine learning task such as image recognition or machine translation.

Machine learning is a field of study that gives computers the ability to learn without being explicitly programmed. Machine learning explores the study and construction of algorithms (also referred to herein as tools) that can learn from existing data and make predictions on new data. Such machine learning tools operate by building models from example training data 112 to make data-driven predictions or decisions, which are represented as outputs or evaluations 120 . Although the example embodiments are presented with respect to several machine learning tools, the principles presented herein may be applicable to other machine learning tools as well.

In some example embodiments, different machine learning tools may be used. For example, Logistic Regression (LR), Naive-Bayes, Random Forest (RF), Neural Network (NN), Matrix Factorization, and Support Vector Machine (SVM) tools can be used to classify or score job postings .

Two common types of problems in machine learning are classification problems and regression problems. Classification problems (also known as categorization problems) aim to classify items into one of several categorical values (for example, is this object an apple or an orange). Regression algorithms aim to quantify some items (for example, by providing real values). The machine learning algorithm utilizes the training data 112 to find correlations between the identified features 102 that affect the outcome.

A machine learning algorithm analyzes data using features 102 to generate an estimate 120 . A feature 102 is an individually measurable property of the observed phenomenon. The concept of features is related to the concept of explanatory variables used in statistical techniques such as linear regression. In pattern recognition, classification, and regression, the selection of informative, discriminative, and independent features is important for efficient operation of MLPs. Features can be of different types, such as numeric features, strings, and graphs.

In an example embodiment, features 102 may be of different types and may include one or more of the following items: message word 103, message concept 104, communication history 105, past user behavior 106, message subject 107 , other message attributes 108 , sender 109 and user data 110 .

The machine learning algorithm utilizes the training data 112 to find correlations between the identified features 102 that affect the outcome or evaluation 120 . In some example embodiments, training data 112 includes labeled data, which is known data about one or more identified features 102 and one or more outcomes, such as detecting communication patterns, detecting message meaning, generating message summaries , detect action items in messages, detect urgency in messages, detect relationship between user and sender, calculate score attributes, calculate message scores, etc.

Armed with the training data 112 and the identified features 102 , the machine learning tool is trained at operation 114 . The machine learning tool identifies the value of features 102 as they relate to training data 112 . The result of the training is the trained machine learning program 116 .

When the machine learning program 116 is used to perform the evaluation, new data 118 is provided as input to the trained machine learning program 116, and the machine learning program 116 produces an evaluation 120 as output. For example, when checking a message for action items, the machine learning program uses the message content and message metadata to determine whether there is a request for action in the message.

Machine learning techniques train the model to make accurate predictions about the data fed into the model (e.g., what the user said in a given utterance; whether the noun is a person, place, or thing; what the weather will be like tomorrow) . During the learning phase, the model is developed against the input training data set to optimize the model to correctly predict the output given the input. In general, the learning phase can be supervised, semi-supervised, or unsupervised; indicating a lower and lower level of providing the "correct" output corresponding to the training input. In the supervised learning phase, all outputs are given to the model, and the model is instructed to develop general rules or algorithms that map inputs to outputs. In contrast, in the unsupervised learning phase, the desired output is not provided for the input, allowing the model to develop its own rules for discovering relationships within the training data set. In the semi-supervised learning phase, a partially labeled training set is provided, where some outputs of the training dataset are known and some are unknown.

A model can be run for several epochs (e.g., iterations) against a training dataset, where the training dataset is repeatedly fed into the model to refine its results. For example, in the supervised learning phase, the model is developed to predict the output for a given set of inputs, and is evaluated over several epochs to provide more reliably for the training dataset the maximum number of inputs designated as corresponding to the given inputs Output. In another example, for the unsupervised learning phase, develop the model to cluster the data into n groups, and evaluate over several epochs how consistently it puts a given input into a given group , and how reliably it produces n desired clusters in each epoch.

Once an epoch is run, the models are evaluated and the values of their variables are adjusted in an iterative attempt to better refine the model. In various aspects, the evaluation is weighted toward false negatives, false positives, or equally weighted with respect to the overall accuracy of the model. Depending on the machine learning technique used, the values can be adjusted in several ways. For example, in a genetic or evolutionary algorithm, the values of the model that were most successful in predicting the desired output are used to develop values for the model to use during subsequent epochs, which may include random variation/mutation to provide additional data points. Those of ordinary skill in the art will be familiar with several other machine learning algorithms that can be applied with the present disclosure, including linear regression, random forests, decision tree learning, neural networks, deep neural networks, and the like.

Each model develops a rule or algorithm over several epochs by varying the value of one or more variables affecting the input to more closely map to the desired outcome, but since the training data set may vary, and preferably is very large, so perfect accuracy and precision may not be possible. Thus, the number of epochs that make up the learning phase can be set to a given number of trials or a fixed time/computational budget, or can be set at Terminated before reaching this amount/budget. For example, if the training phase is designed to run for n epochs and produce a model that is at least 95% accurate, and such a model was produced before the nth epoch, then the learning phase can end early and use the resulting Model for target accuracy threshold. Similarly, if a given model is not accurate enough to meet a random chance threshold (e.g. the model is only 55% accurate in deciding true/false outputs for a given input), then the learning phase of that model can be terminated early, however Other models in the learning phase can continue training. Similarly, when a given model continues to deliver similar accuracy, or results waver over multiple epochs (a performance plateau has been reached), the learning phase for that given model can be performed before the number of epochs/compute budget is reached termination.

Once the learning phase is complete, the model is finalized. In some example embodiments, the finalized model is evaluated against test criteria. In a first example, a test data set comprising inputs and their known outputs is fed into the finalized model to determine how accurate the model is on data it has not been trained on. In a second example, the false positive rate or false negative rate can be used to evaluate the finalized model. In a third example, the division between data clusters is used to select a model that produces the sharpest boundaries for its data clusters.

FIG. 2 illustrates an example neural network 204 in accordance with some embodiments. As shown, neural network 204 receives source domain data 202 as input. The input is passed through a plurality of layers 206, resulting in an output. Each layer 206 includes a plurality of neurons 208 . Neurons 208 receive inputs from neurons of the previous layer and apply weights to the values received from those neurons to produce neuron outputs. The neuron outputs from the final layer 206 are combined to produce the output of the neural network 204 .

As shown at the bottom of Figure 2, the input is a vector x . The input passes through a plurality of layers 206, where weights W ₁ , W ₂ , ..., W _i are applied to the input to each layer to derive f ¹ (x) , f ² (x) , ... , f ^i-1 ( x) until the final output f(x) is calculated.

In some example embodiments, a neural network 204 (eg, a deep learning, deep convolutional, or recurrent neural network) includes a series of neurons 208 (eg, long short-term memory (LSTM) nodes) arranged in a network. Neurons 208 are architectural elements used in data processing and artificial intelligence (particularly machine learning), which include memory that can decide when to "remember" based on the weights of the inputs provided to a given neuron 208 and when to "forget" the value held in that memory. As used herein, each of the neurons 208 is configured to accept a predetermined number of inputs from other neurons 208 in the neural network 204 to provide relational and subrelational outputs for the content of the frame being analyzed. Individual neurons 208 can be chained together in various configurations of neural networks and/or organized into tree structures to provide interaction and relational learning modeling of how each of the frames in an utterance is related to each other of.

For example, an LSTM node that is a neuron includes several gates to process input vectors (such as phonemes from utterances), memory cells, and output vectors (such as contextual representations). Input and output gates control the flow of information into and out of a memory cell, respectively, while forgetting gates selectively remove information from memory cells based on inputs from earlier connected cells in the neural network. The weight and bias vectors for the various gates are adjusted during the training phase, and once the training phase is complete, those weights and biases are finalized for normal operation. Those skilled in the art will understand that neurons and neural networks can be constructed programmatically (eg, via software instructions), or via specialized hardware that connects each neuron to form a neural network.

Neural networks use features to analyze data to produce an assessment (such as recognizing units of speech). A characteristic is an individually measurable property of an observed phenomenon. The concept of features is related to the concept of explanatory variables used in statistical techniques such as linear regression. Further, the deep features represent the output of the nodes in the hidden layers of the deep neural network.

A neural network (sometimes called an artificial neural network) is a computing system/device based on a biological neural network that takes into account the animal brain. Such systems/devices gradually improve their performance on performing tasks, which is called learning, and generally do not require task-specific programming. For example, in image recognition, a neural network can be taught to recognize images that contain an object by analyzing example images that have been labeled with the name of that object, and having learned the object and name, the analysis results can be used to identify unidentified objects. The object in the tagged image. Neural networks are based on collections of connected units called neurons, where each connection between neurons (called a synapse) can transmit a one-way signal whose activation strength varies with the strength of the connection. A receiving neuron can fire and propagate a signal to downstream neurons connected to it, generally based on whether the combined incoming signal from potentially many transmitting neurons is of sufficient strength, where strength is a parameter.

A deep neural network (DNN) is a stacked neural network, which consists of multiple layers. These layers are made up of nodes, which are where computation occurs, loosely modeled after a neuron in the human brain, which fires when it encounters enough stimuli. Nodes combine input from data with sets of coefficients, or weights, that amplify or suppress that input, assigning importance to the input for the task the algorithm is trying to learn. These input-weight products are summed, and their sum is passed through a so-called node activation function to determine whether and to what extent the signal affects the final result further through the network. DNNs use cascades of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. High-level features are derived from low-level features to form a hierarchical representation. The layers following the input layer may be convolutional layers that produce feature maps that are filtered results as input and used by the next convolutional layer.

In the training of DNN architectures, regression (structured as a collection of statistical procedures for estimating relationships between variables) can include the minimization of a cost function. A cost function can be implemented as a function that returns a number representing how well the neural network did at mapping training examples to the correct output. During training, if the cost function value is not within a predetermined range, then based on known training images, backpropagation is used, where backpropagation is a combination of training and optimization methods such as stochastic gradient descent (SGD) Common methods used with artificial neural networks.

Using backpropagation can include propagation and weight updates. When an input is submitted to a neural network, it propagates forward through the neural network layer by layer until it reaches the output layer. Then, using a cost function, the output of the neural network is compared to the desired output, and an error value is calculated for each of the nodes in the output layer. Error values are backpropagated from the output until each node has an associated error value that roughly represents its contribution to the original output. Backpropagation can use these error values to compute the gradient of the cost function with respect to the weights in the neural network. The computed gradients are fed to the chosen optimization method in order to update the weights in an attempt to minimize the cost function.

FIG. 3 illustrates the training of an image recognition machine learning program according to some embodiments. Machine learning programs can be implemented on one or more computers. Box 302 illustrates the training set, which includes multiple classes 304 . Each category 304 includes a plurality of images 306 associated with that category. Each category 304 may correspond to a type of object in image 306 (eg, numbers 0-9, man or woman, cat and dog rating). In one example, a machine learning program is trained to recognize images of US presidents, and each category corresponds to each president (e.g. one category corresponds to Barack Obama, one category corresponds to George Walker Bush (George W. Bush, one category corresponds to Bill Clinton, etc.). At block 308, a machine learning program is trained, eg, using a deep neural network. At block 310 , the trained classifier resulting from the training of block 308 recognizes image 312 , and at block 314 , recognizes the image. For example, if image 312 is a photograph of Bill Clinton, the classifier identifies the image at block 314 as corresponding to Bill Clinton.

Figure 3 illustrates the training of a classifier in accordance with some example embodiments. The machine learning algorithm is designed to recognize faces, and the training set 302 includes data that maps samples to categories 304 (eg, categories include all images of wallets). Categories can also be called tags. Although the embodiments presented herein are presented with reference to object recognition, the same principles can be applied to train a machine learning program to recognize any type of item.

The training set 302 includes a plurality of images 306 (eg, images 306 ) of each category 304 , and each image is associated with one of the categories (eg, categories) to be recognized. The machine learning program is trained 308 with the training data to generate a classifier 310 that can be used to recognize images. In some example embodiments, the machine learning program is a DNN.

When the input image 312 is to be recognized, the classifier 310 analyzes the input image 312 to identify a category (eg, category 314 ) corresponding to the input image 312 .

Figure 4 illustrates the feature extraction process and classifier training in accordance with some example embodiments. The training classifier can be divided into a feature extraction layer 402 and a classifier layer 414 . Each image is analyzed sequentially by a plurality of layers 406 - 413 in the feature extraction layer 402 .

With the development of deep convolutional neural networks, the focus of face recognition is to learn a good face feature space, in which the faces of the same person are close to each other, and the faces of different people are far away from each other. For example, verification tasks with LFW (Labeled Faces in the Wild) dataset are often used for face verification.

Many face recognition tasks (such as MegaFace and LFW) are based on the similarity comparison between images in the gallery set and query set, which is basically a K-nearest neighbor (KNN) method for estimating a person's identity. In an ideal situation, there is a good face feature extractor (inter-class distance is always greater than intra-class distance), and the KNN method is sufficient to estimate the identity of the person.

Feature extraction is the process of reducing the amount of resources required to describe large datasets. When performing analysis on complex data, a major problem arises from the number of variables involved. Analyzes with a large number of variables generally require large amounts of memory and computing power, and can cause classification algorithms to overfit to training samples and generalize poorly to new samples. Feature extraction is a general term that describes methods of constructing combinations of variables to bypass these large dataset problems while still describing the data with sufficient accuracy for the desired purpose.

In some example embodiments, feature extraction starts with an initial measurement dataset and builds derived values (features) that are intended to be informative and non-redundant, thereby facilitating subsequent learning and generalization steps. Further, feature extraction is related to dimensionality reduction, such as reducing large vectors (sometimes with very sparse data) into smaller vectors that capture the same or similar amount of information.

Determining the subset of initial features is called feature selection. The selected features are expected to contain relevant information from the input data such that the desired task can be performed by using this simplified representation instead of the full initial data. DNNs utilize stacks of layers, where each layer performs a function. For example, layers can be convolutions, nonlinear transformations, calculations of averages, etc. Finally, this DNN generates an output through a classifier 414 . In Figure 4, data travels from left to right, and features are extracted. The goal of training a neural network is to find the parameters of all layers such that they are sufficient for the desired task.

As shown in Figure 4, a "stride of 4" filter is applied at layer 406 and max pooling is applied at layers 407-413. The stride controls how the filter is convolved around the input volume. "Stride of 4" means that the filter performs convolutions around the input volume four units at a time. Max pooling refers to downsampling by selecting the maximum value in each max pooling region.

In some example embodiments, the structure of each layer is predefined. For example, a convolutional layer may contain small convolutional kernels and their corresponding convolutional parameters, and a summation layer may compute a sum or a weighted sum of two pixels of an input image. Training helps define the weight coefficients for the summation.

One way to improve the performance of DNNs is to identify newer structures for feature extraction layers, and another way is by improving the way in which parameters are identified at different layers to accomplish the desired task. The challenge is that for a typical neural network there may be millions of parameters to optimize. Trying to optimize all these parameters from scratch could take hours, days, or even weeks, depending on the amount of computing resources available and the amount of data in the training set.

Figure 5 is a block diagram illustrating a computer 500 in accordance with some embodiments. In some embodiments, components of computer 500 may store or be integrated with other components shown in the circuit block diagram of FIG. 5 . For example, a portion of computer 500 may reside within processor 502 and may be referred to as "processing circuitry." Processing circuitry may include processing hardware such as one or more central processing units (CPUs), one or more graphics processing units (GPUs), and the like. In alternative embodiments, computer 500 may operate as a standalone device, or may be connected (eg, networked) with other computers. In a networked deployment, the computer 500 can operate as a server, a client, or both in a server-client networking environment. In an example, computer 500 may act as a peer machine in a peer-to-peer (P2P) (or other decentralized) network environment. In this document, the terms P2P, device-to-device (D2D), and sidelink (sidelink) are used interchangeably. The computer 500 can be a dedicated computer, personal computer (PC), tablet computer, personal digital assistant (PDA), mobile phone, smart phone, network device (web appliance), network router, switch or bridge, or any A machine capable of executing instructions (sequentially or otherwise) that specify actions to be taken by the machine.

The examples described herein may comprise or may operate on logic or a plurality of components, modules or mechanisms. Modules and parts are tangible entities (such as hardware) that perform a specified operation and that can be configured or arranged in a certain manner. In an example, a circuit (eg, within itself or with respect to external entities such as other circuits) may be arranged in a specified manner as a module. In an example, all or part of one or more computer systems/devices (e.g., stand-alone, client or server computer systems) or one or more hardware processors may be implemented by firmware or software (e.g., instructions, application or application) configured as a module, which is used to perform specified operations. In an example, software may reside on a machine-readable medium. In an example, the software, when executed by the underlying hardware of the module, causes the hardware to perform specified operations.

Accordingly, the term "module" (and "part") is understood to include a tangible entity, whether physically constructed, specifically configured (e.g. hardwired) or temporarily (e.g. transient) configured (e.g. programmed) as An entity that operates in the specified manner or performs some or all of any operation described herein. Consider the example where modules are provisioned temporarily, each of the modules need not be instantiated at any one time. For example, where a module includes a general-purpose hardware processor configured using software, the general-purpose hardware processor may be configured at different times as corresponding different modules. Software may accordingly configure the hardware processor to, for example, constitute a particular module at one instance of time and a different module at a different instance of time.

Computer 500 may include a hardware processor 502 (e.g., a central processing unit (CPU), a GPU, a hardware processor core, or any combination of the foregoing), main memory 504, and static memory 506, some or all of which may be accessed via Interlinks (eg, bus bars) 508 communicate with each other. Although not shown, the main memory 504 may include any or all of removable and non-removable storage, volatile memory, or non-volatile memory. The computer 500 may further include a video display unit 510 (or other display units), an alphanumeric input device 512 (such as a keyboard), and a user interface (UI) navigation device 514 (such as a mouse). In an example, the display unit 510, the input device 512 and the UI navigation device 514 may be a touch screen display. The computer 500 may additionally include a storage device (e.g., a drive unit) 516, a signal generating device 518 (e.g., a speaker), a network interface device 520, and one or more sensors 521, such as a global positioning system (GPS) sensor, compass , accelerometer or other sensors. The computer 500 may include an output controller 528, such as a serial (e.g., universal serial bus (USB)), side-by-side, or other wired or wireless (e.g., infrared (IR), near field communication, etc.) A peripheral device (such as a printer, card reader, etc.) for communication or control.

Drive unit 516 (e.g., a storage device) may include a machine-readable medium 522 having stored thereon data structures or instructions 524 (e.g., software ) of one or more collections. Instructions 524 may also reside, completely or at least partially, within main memory 504 , static memory 506 , or hardware processor 502 during execution by computer 500 . In an example, one or any combination of hardware processor 502, main memory 504, static memory 506, or storage device 516 may constitute a machine-readable medium.

Although machine-readable medium 522 is illustrated as a single medium, the term "machine-readable medium" may also include a single medium or multiple media (such as centralized or distributed data) configured to store the one or more instructions 524. library, and/or associated cache and servers).

The term "machine-readable medium" may include any medium capable of storing, encoding, or carrying instructions for execution by the computer 500 and causing the computer 500 to perform any one or more of the techniques of the present disclosure, or any medium capable of storing, encoding, or or media carrying data structures used by or associated with such instructions. Non-limiting examples of machine-readable media can include solid-state memory and optical and magnetic media. Specific examples of machine-readable media may include: non-volatile memory, such as semiconductor memory devices (such as Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM) ) and flash memory devices; magnetic disks, such as internal and removable hard disks; magneto-optical disks; random access memory (RAM); and CD-ROM and DVD-ROM disks. In some examples, machine-readable media may include non-transitory machine-readable media. In some examples, machine-readable media may include machine-readable media that are not transitory propagated signals.

The instruction 524 may further utilize a transmission medium via the communication network 526 via a network interface device 520 utilizing multiple transmission protocols (such as frame forwarding, Internet Protocol (IP), Transmission Control Protocol (TCP), User Packet Protocol (UDP) ), Hypertext Transfer Protocol (HTTP), etc.) to transmit or receive. Example communication networks can include local area networks (LANs), wide area networks (WANs), packet data networks (such as the Internet), cellular networks (such as cellular networks), plain old telephone (POTS) networks, and wireless Data networking (such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 series of standards known as Wi-Fi®, the IEEE 802.16 series of standards known as WiMax®), the IEEE 802.15.4 series of standards, the Long Term Evolution (LTE) series of standards, Universal Mobile Telecommunications System (UMTS) series of standards, peer-to-peer (P2P) networks, and more. In an example, the network interface device 520 may include one or more physical jacks (such as Ethernet, coaxial, or telephone jacks) or one or more antennas to connect with the communication network 526 .

Substrates (eg, silicon) have been used as interposers disposed between electrical components (eg, printed circuit boards, integrated circuits, etc.). Metallized through-glass vias (TGVs) provide a path through the interposer, and power signals are passed between opposite sides of the interposer. Glass is a very favorable substrate material for electrical signal transmission because of its dimensional stability, adjustable coefficient of thermal expansion (CTE), low electrical loss at high frequencies, high thermal stability, and the ability to operate in various thicknesses and large panel sizes. ability to form. TGV production technology can utilize laser damage and etching (LD&E) processes to create microwells in thin glass substrates for various applications in the semiconductor and display industries. It can use a pulsed laser beam to create damage tracks in glass using a series of short picosecond pulses. The laser-damaged traces are then etched in an acid bath, allowing vias to be formed to achieve a specific surface diameter (e.g. between 10 and 200 µm) and a desired internal geometry (straight, tapered, or with hourglass shape of the target waist diameter). Three-dimensional (3D) via shape is critical for process development, product monitoring and control. TGVs with internal defects can not only affect the electrical performance of the final product, but also affect subsequent processes such as metallization. Although our large-scale LD&E process has achieved high-speed TGV formation, it is difficult for inspection and characterization of the final etched via geometry, such as 3D shape defects (bumps, chatter marks, blind vias, etc.) However, some major challenges remain.

Some schemes for characterizing TGV geometry include vision-based surface diameter or girdle diameter measurements, 3D computational topography (CT) scanning, or destructive testing, which may utilize cutting of the substrate to enable side-viewing optical microscopy. TGVs with internal via sidewall defects may still have inlet, outlet, or waist diameters that are within specification. In some cases, via sidewall defects can be identified on CT scans or through destructive testing, which is time-consuming, expensive, and can only be performed on small samples. Furthermore, it may be difficult to quantify the characteristics of via sidewall defects using these known techniques. Therefore, techniques to inspect the internal shape and sidewall defects of TGVs are desirable.

Some embodiments relate to a non-destructive method that combines machine vision with machine learning to inspect TGV samples for internal via shape and sidewall defects. The method, which can also characterize and map defects, can be easily scaled up for high-speed inline inspection of large TGV panel production.

In some embodiments, a computing device coupled to a camera takes images of individual vias from oblique inspection angles under appropriate lighting. Some embodiments use optimized illumination angles (eg, incidence from the back or top), preferential polarization orientation, and uniform illumination to improve contrast and sensitivity of TGV images. Traditional visual inspection methods include inspection with a microscope, either from the top or bottom of the via, or from the side after cutting the sample, both of which provide limited information on the internal shape of the via. Some embodiments may inspect all TGVs inside a substrate by detecting weak light deflection features of TGV shape defects and comparing the detected features to images of properly shaped vias. Individual TGV images are extracted from raw scan images along with their corresponding via locations, which are converted to XY part coordinates (or other coordinates) and matched to the nominal via pattern. Some embodiments include fully automated via characterization. Supervised deep learning (DL)/machine learning (ML) techniques can be used to characterize vias in terms of their appearance in images. In some cases, a collection of guide hole images is classified by the user as "good" or "defective" and then used to train a machine learning (ML) algorithm. The trained ML model enables automatic TGV classification. Some embodiments provide reclassification after classification using online learning. Some embodiments aim to obtain image label feedback after obtaining category category and score information. This enables users to make more insightful decisions on via images with low margin scores, new cases, or outliers. In addition, other measurements can also be used for via ML characterization. A global coordinate transfer and matching algorithm can be used to incorporate other measurements such as the top, bottom or waist diameter of the TGV corresponding to a particular via. This enables one to understand correlations between different measurements and improve classification accuracy. Some embodiments are also applicable to the inspection of metallized TGVs. Finally, line-scan imaging can be used to apply this approach to high-speed in-line applications.

Some embodiments relate to non-destructive, high-speed methods that can inspect millions of TGVs in flat substrates. Some embodiments enable quantitative accounting for different internal via defects in metallized or non-metallized TGVs in different transparent substrates such as glass, fused silica, and the like. Some embodiments provide methods of optimizing, calibrating and monitoring LD&E and TGV metallization processes. By being non-destructive and rapid, some embodiments provide significant cost and time savings for TGV production. Some embodiments relate to novel data fusion and machine learning techniques for obtaining correlations between measurements and LD&E parameters to fundamentally understand which aspects of LD&E interactions affect TGV measures.

FIG. 6 illustrates an example schematic 600 of TGV inspection using backlighting, in accordance with some embodiments. As shown in diagram 600 , camera 602 is pointed at object 604 . Item 604 includes TGV 606 and chip light emitting diodes (LEDs) 608 . Object 604 is illuminated by diffuse backlight 610 .

FIG. 7 illustrates an example TGV defect inspection system 700 in accordance with some embodiments. As shown, the TGV defect inspection system 700 includes an area camera 702 with a rotation unit 704 for rotation. Light source 706 powers backlight illuminator 708 of the surface configured for X/Y motion 710 . Area camera 702 is pointed at the surface illuminated by backlight illuminator 708 .

The TGV process can use a short single picosecond laser pulse to create a laser damage track in a glass substrate. The laser beam is formed into an extended focus or quasi-non-diffracted beam using optics that form Bessel-like or Bessel-Gaussian beams. Due to the quasi-non-diffracting nature of the beam, the light maintains a tightly focused intensity over a much longer range than the more commonly used Gaussian beam, enabling the entire thickness of the glass substrate to be damaged by a single short pulse. The diameter of the damage track can be 300-500 nm and its length can extend through the entire thickness of the glass (50 to 1000 µm). Laser damaged glass can be etched in an etchant bath containing either hydrofluoric acid (HF) or sodium/potassium hydroxide to form TGVs. Laser and etching processes are disclosed, for example, in US Patent No. 9,517,963, which is incorporated herein by reference in its entirety.

FIG. 8 illustrates an example TGV characterization image 800 via waist 802 , via inlet and outlet 804 , and via side view 806 . According to some embodiments, image 800 may be generated using an optical microscope.

A conventional characterization scheme for TGVs involves measuring inlet, outlet, and waist diameters with a 2D visual inspection tool, as shown at 802 and 804 in FIG. shown here. TGV diameter measurements cannot detect internal defects in the via shape as shown by the microscope's profile. However, side-view inspection with optical microscopy can be a destructive method and cannot meet the needs of in-line measurements. Another well-known measurement tool is the 3D CT scanner, which provides details inside the TGV with unparalleled resolution.

9 illustrates example CT scan images of a TGV showing defect-free vias and vias with internal defects, in accordance with some embodiments. However, CT scanning is slow, very expensive, and difficult to scale up for in-line applications.

10A-10B illustrate individual via images cropped from raw area scan images, their classified shapes/internal defects, and their positions transformed and mapped to part coordinates, according to some embodiments.

Figure 11 illustrates defective TGVs with different two-dimensional (2D) features depending on defect type and optical configuration, according to some embodiments.

FIG. 12 illustrates an example via CT scan image cross-section 1200 in accordance with some embodiments. Cross section 1200 includes chatter marks 1202 and protrusions 1204. 2D image 1200 was acquired at an oblique angle and using line scan imaging.

13 illustrates an example line scan image 1300 with backlighting including non-defective vias 1302 and defective vias 1304 and 1306, in accordance with some embodiments.

FIG. 14 illustrates example images 1400 of different types of vias in accordance with some embodiments. Image 1400 includes images of waisted vias 1402 , blind vias 1404 , raised vias 1406 , and good (defect-free) vias 1408 .

FIG. 15 illustrates an example TGV image 1500 of a non-metallized TGV 1502 and a metallized TGV 1504 in accordance with some embodiments. Image 1500 is acquired using a line scan camera and a backlight.

16 illustrates a first example via classification map 1600 measured before 1602 and after 1604 of rotating the part 90 degrees, in accordance with some embodiments.

FIG. 17 illustrates a second example via classification map 1700 measured from side A 1702 and side B 1704 in accordance with some embodiments.

Figure 18 illustrates example area scan imagery machine learning classification results in accordance with some embodiments.

Figure 19 illustrates an example TGV inspection system 1900 in accordance with some embodiments. System 1900 allows X movement 1902 and Y movement 1904 . System 1900 includes line scan time delayed integration (TDI) camera 1906 , LED backlight carrier 1908 and turntable 1910 .

Figure 20 illustrates an example relationship between internal defect rate and etch delay, in accordance with some embodiments. As shown in Figure 20, under the same laser damage conditions, the internal defect rate is related to the etch delay.

Figure 21 illustrates an example machine learning program 2100 in accordance with some embodiments. As shown, labeling tool 2102 and training data set 2108 provide data for machine learning (ML) classification 2104 . ML classification 2104 provides output to relabeling and selection tool 2106 , which validates or edits the classification categories and scores produced by ML classification 2104 . The confirmed or edited classification categories or scores are provided to the training dataset 2108 via data upload from the relabeling and selection tool 2106 .

Some embodiments relate to a high-speed, non-destructive, in-line capable, quantitative method for the detection and classification of internal via defects in transparent substrates. Some embodiments include inspection devices, which can be stand-alone or integrated into other TGV inspection systems. FIG. 19 shows the design concept of a stand-alone line-scan based TGV internal defect measurement system 1900 with backlighting built into the sample carrier (eg, LED backlight carrier 1908 ). The sample carrier is mounted on a rotary stage and a linear axis. Camera 1906 is mounted on another linear axis that is orthogonal to the vehicle's linear axis. The motion system described (eg, X motion 1902 and Y motion 1904) enables inspection of vias from different angles within the sample plane as desired based on the via pattern design.

Some techniques include taking a diffuse image of each individual TGV within a transparent substrate (eg, glass) at an oblique angle with an imaging device, as shown in Figure 6 or Figure 7 . The illumination source produces light that illuminates the pilot hole at a specific angle. These include but are not limited to top/back lighting, edge lighting or low angle darkfield lighting. The source can be a high intensity LED light with a selected spectrum or white light. Some embodiments inspect metallized or unmetallized vias with optimized illumination for optimal contrast. For example, a diffused top/backlight can reduce reflections from the inner surface of the TGV and increase image contrast.

In some cases, the camera is a detector that collects the signal from the via. To detect very weak signals deflected from internal defects in vias such as bumps or chatter marks, a TDI line scan camera can be used. To improve the speed of inspection, TDI cameras can be operated at high frame rates. The camera angle has to be optimized and is typically between 40 and 50 degrees relative to the substrate surface. The minimum spacing and camera angle between adjacent vias must be optimized to prevent vias from overlapping in the image. With TDI line scan imaging, all vias will be in focus, which makes image processing less complex and faster compared to area scan cameras.

In terms of optics, the camera objective can have a specific magnification, numerical aperture (NA) and depth of field, and is preferably telecentric to capture the top and bottom of the via when in focus. In some examples, lenses with a magnification of less than 3x and an NA of less than 0.1 provide a depth of field large enough to image through glass as thick as 500 µm. The magnification may be, for example, 1.5x-2.5x.

Some embodiments include a combination of software and hardware to normalize image pixel response and noise from optical aberrations and electrical sources.

Some embodiments include, at the computing device, extracting individual via images and correlating shape attributes with other measurement data such as diameter, roundness, LD&E process information, and other available operator classification information. Some embodiments use pattern matching or other image correlation algorithms to find individual TGVs, as shown in FIGS. 10A-10B . Some embodiments utilize a one-to-one via image correlated with other measurements or process parameters based on the global via pattern defined by the product design.

Some embodiments include, at the computing device, generating the machine learning model. Via imagery and other measurement data can be used in supervised machine learning to train ML algorithms to automatically classify via types. Images from different types of via shapes will first be manually sorted and classified for training purposes; for example, classified as "good (no defect)", "raised", or "blind", as shown in Figures 13 and 14 .

Machine learning/deep learning algorithms enable automatic via classification without requiring prior domain knowledge of via characteristics. A well-known and common ML/DL method is the supervised learning model, which utilizes labeled data to train the ML/DL model. ML experts design models to get better performance based on training data. However, some embodiments involve using better ML models to obtain better training datasets, as shown in FIG. 21 .

The cropped via images are collected and labeled (become the training dataset 2108) to train the ML model. The via image after the training procedure may not need to be marked, as shown in marking tool 2102 . The trained ML model classifies the via image at ML classification 2104 . After ML classification 2104 , the user is notified at relabeling and selection tool 2106 about the via defect condition and classification score. If the image has a strong inference score from the ML model, the user can re-label the via image, and the important via image data will be stored at the training dataset. Users can sort the via images with low marginal scores to examine the labeling and decide how to relabel and store it at the training dataset. Some embodiments enhance the integrity of the training data set, enabling ML models to perform well and improve incrementally over future training sessions.

Using the proposed inspection system and ML algorithms, certain internal via defects are linked to specific conditions in the LD&E process. Most vias with internal defects have normal surface properties (such as inlet and outlet diameters) and may not be identifiable via surface measurements alone, either from the top or bottom. FIG. 17 shows an example of a classification map 1700 measured from side A ( 1702 ) and side B ( 1704 ), respectively. Some via defects such as bumps and chatter marks have good planar symmetry. Furthermore, defect maps inspected from different viewing angles (i.e. after sample rotation) are well aligned. Figure 16 shows two via classification maps 1602 and 1604, one measured at zero degrees (1602) and one measured after a sample rotation of 90 degrees (1604). Figure 20 shows that for the same laser process but different etch conditions (eg, different etch delays), the percentage of internal via defects can vary significantly. The disclosed techniques can also be used to inspect vias after metallization. FIG. 15 shows a via image 1500 using a line scan and backlight configuration before 1502 and after 1504 metallization.

Figure 22 is a block diagram of a computing device 2200 for TGV defect inspection, according to some embodiments. Computing device 2200 may include one or more of the following: a server, a laptop, a desktop, a mobile phone, a tablet, a smart watch, a personal digital assistant (PDA), and the like. Computing device 2200 may include one or more components of computer 500 in addition to the components shown in FIG. 22 . Further, while a single computing device 2200 is shown, it should be noted that in some embodiments, components of computing device 2200 may be distributed across multiple physical or virtual machines. As shown, computing device 2200 includes processing circuitry 2210 (eg, one or more of processor 502, central processing unit, graphics processing unit, etc.) and memory 2220 (eg, main memory 504, static memory 506, drive unit 516, disk drive, hard disk drive, random access memory, etc.). The memory 2220 stores an image generation engine 2230 , a computer vision engine 2240 and a classification engine 2250 . Example operations of image generation engine 2230 , computer vision engine 2240 , and classification engine 2250 are discussed in connection with FIG. 23 . Engines 2230, 2240, and 2250 may be implemented using artificial intelligence, machine learning, supervised learning, online learning, and/or deep learning techniques, eg, as described in connection with FIGS. 1-4.

23 is a flowchart of a computer-implemented method 2300 for TGV defect inspection, according to some embodiments. Method 2300 may be implemented at a computing device (eg, computing device 2200 ) or at multiple computing devices, which may include one or more components of computer 500 .

At block 2310, the computing device controls an illumination source (e.g., LED backlit carrier 1908) and an imaging device (e.g., camera 1906) to generate an image of a TGV comprising an etched or metalized microvia in a transparent workpiece, the A transparent workpiece includes at least two surfaces. Controlling the illumination source and imaging device includes, for a plurality of TGVs, directing the illumination source to illuminate the TGVs in the workpiece; and directing the imaging device to detect scattered image signals from light scattered by the TGVs. The imaging axis of the imaging device extends a non-zero imaging angle relative to the axis of a given TGV from the plurality of TGVs. The entirety of a given TGV is within the field of view of the imaging device.

At block 2320, the computing device generates an image of the TGV based on the scattered image signal detected by the imaging device using an image generation engine at the computing device (eg, image generation engine 2230).

At block 2330, the computing device uses a computer vision engine (eg, computer vision engine 2240) at the computing device to calculate coordinates of the TGV within the image based on the scattered image signals detected by the imaging device.

At block 2340, the computing device uses a classification engine (e.g., classification engine 2250) at the computing device to generate a TGV type classification for one or more TGVs in the imagery based on the calculated coordinates, wherein the TGV type classification indicates whether the TGV has or Lack of defects and, if the TGV is defective, indicate the type of defect.

At block 2350, the computing device determines that a plurality of TGVs have defects associated with a given defect type based on the generated TGV type classification.

At block 2360, the computing device identifies manufacturing machine settings associated with the defect type.

At block 2370, the computing device transmits (eg, to the manufacturing machine) control signals to adjust manufacturing machine settings.

According to some embodiments, the imaging device includes a camera and an imaging lens arranged on the imaging axis, the camera includes an area scan camera or a line scan camera, and the non-zero imaging angle is between 30 degrees and 60 degrees.

According to some embodiments, the imaging lens includes a magnification factor and a numerical aperture that depend on the thickness of the transparent workpiece.

According to some embodiments, the thickness of the transparent workpiece is between 1 and 1000 microns, wherein the magnification factor is less than 3.

According to some embodiments, the control signals are transmitted directly to the manufacturing machine to adjust the manufacturing machine settings there.

According to some embodiments, the computing device accesses additional images of the TGV manufactured after adjusting the manufacturing machine settings. The computing device determines whether defects associated with a given defect type vary in frequency. The computing device transmits additional control signals to the manufacturing machines to further adjust manufacturing machine settings based on changes in frequency of a given defect type.

According to some embodiments, a control signal is transmitted to the user device to instruct the end user to adjust manufacturing machine settings.

According to some embodiments, the computing device transmits the generated TGV type classification to the user equipment. The computing device receives an indication from the user device that at least one TGV type classification is incorrect. The computing device provides further training to the TGV type classifier based on the indication.

According to some embodiments, the generated images are of a plurality of different image types, including one or more of the following: RGB images, black and white images, shaded images, and CT scan images.

According to some embodiments, the classification engine includes an artificial neural network (ANN) trained using online supervised learning.

According to some embodiments, when the depth of the pilot hole is greater than the depth of focus of the imaging system, multiple camera focuses may be performed at increasing pilot hole depths, or multiple inspection cycles may be performed at increasing depths.

According to some embodiments, the computer directs light from the illumination source onto the plurality of vias by directing the light from the illumination source through an edge of the transparent workpiece.

According to some embodiments, depending on the final via interior shape/geometry (cylindrical, ribbon, etc.), the illumination NA can vary from telecentric illumination to angle-controlled diffuse surfaces.

According to some embodiments, the computing device determines the characteristics of the laser processing system by determining the focal position of the defect-forming laser processing based on the guide hole image and the type.

According to some embodiments, the computing device determines the characteristics of the laser processing system by determining the laser power of the defect forming laser processing based on the guide hole image and the type.

According to some embodiments, the computing device determines the characteristics of the etching system by determining an etching delay before introducing ultrasonically enhanced etching based on the via image and type.

According to some embodiments, the computing device correlates or fuses other measurements including one or more of the following: measurements of via entry/exit and waist, laser damage, and etch processing, to improve ANN classification accuracy Data, guide hole metallization processing data.

Some embodiments include a machine coordinate matching algorithm for correlating individual via measurements on different inspection systems.

According to some embodiments, the ANN reduces the effects from optical imaging distortion as well as the cost of optical components and complex calibration.

In some embodiments, a computing device performs active learning on a training data set including a labeled data set by collecting a set of unlabeled data and user (eg, expert) feedback. The computing device provides a confidence score for each possible category and provides a marginal score, which is the difference between the highest score and the second highest score. Computing devices use user feedback to derive true labels. In addition to this, the computing device also provides other information about the quality of the via, including the diameter/roundness of the entrance and exit of the via shape, waist diameter and circularity, which provides support to the user. Supplementary information for decision-making. Data with low marginal scores are periodically provided to the user with supplementary information including inlet and outlet diameter/circularity, girdle diameter, and circularity of pilot hole shape to determine higher likelihood labels for unlabeled data . The newly labeled data items are used to train a classification engine (eg, classification engine 2250 ) to ascertain the user's knowledge and quality control level.

In some embodiments, the lighting setup directs the lighting source to shine light onto the surface of the transparent workpiece at a particular lighting angle. The angle of illumination is different from the angle of the imaging axis.

As noted above, the blocks of method 2300 are performed sequentially in the order given. However, these blocks can also be performed in any order, not necessarily the order presented. In some cases, two or more blocks may be executed in parallel.

Some embodiments are described as numbered examples (Examples 1, 2, 3, etc.). These are provided as examples only, and do not limit the techniques disclosed herein.

Example 1 is a method comprising the steps of: controlling, by a computing device, an illumination source and an imaging device to generate images of through-glass vias (TGVs), wherein the TGVs include etched or metalized microvias in a transparent workpiece , the transparent workpiece includes at least two surfaces, wherein controlling the illumination source and the imaging device includes performing the following steps on a plurality of TGVs: directing the illumination source to illuminate the TGVs in the workpiece; and directing the imaging device to detect measuring scattered image signals from light scattered by the TGVs, wherein the imaging axis of the imaging device extends a non-zero imaging angle relative to the axis of a given TGV from the plurality of TGVs, wherein the entirety of the given TGV is within the field of view of the imaging device; using an image generation engine at the computing device to generate images of the TGVs based on the scattered image signals detected by the imaging device; using computer vision at the computing device an engine that calculates coordinates of the TGVs in the images based on the scattered image signals detected by the imaging device; for one or more TGVs in the images and based on the calculated coordinates, using A classification engine at the computing device to generate a TGV type classification, wherein the TGV type classification indicates whether the TGV has or lacks a defect, and if the TGV is defective, indicates the type of defect; based on the generated TGV type classification, determining a plurality of The TGV has a defect associated with a given defect type; at the computing device, identifying a manufacturing machine setting associated with the given defect type; and from the computing device, transmitting a control signal to adjust the manufacturing machine setting.

In Example 2, the subject matter of Example 1 includes: wherein the imaging device comprises a camera and an imaging lens disposed on the imaging axis, wherein the camera comprises an area scan camera or a line scan camera, wherein the non-zero imaging angle is between 30 degrees and 60 degrees.

In Example 3, the subject matter of Example 2 includes wherein the imaging lens includes a magnification factor and a numerical aperture that are dependent on a thickness of the transparent workpiece.

In Example 4, the subject matter of Example 3 includes wherein the thickness of the transparent workpiece is between 1 and 1000 microns, wherein the magnification factor is less than 3.

In Example 5, the subject matter of Examples 1-4 includes wherein the control signal is transmitted directly to the manufacturing machine to adjust the manufacturing machine settings there.

In Example 6, the subject matter of Example 5 includes: accessing additional images of TGVs manufactured after adjusting the manufacturing machine settings; determining whether the defect associated with the given defect type has changed in frequency; and from the calculation The apparatus transmits additional control signals to the manufacturing machine to further adjust the manufacturing machine settings based on a frequency change of the given defect type.

In Example 7, the subject matter of Examples 1-6 includes wherein the control signal is transmitted to user equipment to instruct an end user to adjust the manufacturing machine setting.

In Example 8, the subject matter of Examples 1-7 includes: transmitting to user equipment the generated TGV type classification; receiving from the user equipment an indication that at least one TGV type classification is incorrect; and reporting to the TGV type classifier based on the indication Provide further training.

In Example 9, the subject matter of Examples 1-8 includes wherein the generated images are of a plurality of different image types including one or more of the following: RGB images, black and white images, shaded images and CT scan images.

In Example 10, the subject matter of Examples 1-9 includes wherein the classification engine comprises an artificial neural network (ANN) trained using online supervised learning.

Example 11 is at least one machine-readable medium comprising instructions that, when executed by the processing circuitry, cause the processing circuitry to perform operations to implement any of Examples 1-10.

Example 12 is a component comprising the means for implementing any of Examples 1-10.

Example 13 is a system for implementing any of Examples 1-10.

Example 14 is a method for implementing any of Examples 1-10.

Although embodiments have been described with reference to specific example embodiments, it will be apparent that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the disclosure. Accordingly, the specification and drawings are to be viewed in terms of illustration rather than limitation. The accompanying drawings, which form a part hereof, show, by way of illustration and not limitation, specific embodiments in which the subject matter may be practiced. The illustrated embodiments are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other embodiments may be utilized and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Accordingly, the present "embodiments" should not be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims and all equivalents to which such claims are entitled.

Although specific embodiments have been illustrated and described herein, it should be understood that any arrangement which achieves the same purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all adaptations or variations of various embodiments. Combinations of the above-described embodiments, and other embodiments not specifically described herein, will be apparent to those of skill in the art upon reviewing the above description.

In this document, use of the term "a(a)" or "an" as is common in patent documents includes one or more than one, as opposed to "at least one" or "one or more" Any other instances or usages are irrelevant. In this document, unless otherwise indicated, the term "or" is used to denote a non-exclusive or such that "A or B" includes "A but not B", "B but not A" and "A and B ". In this document, the terms "including" and "in which" are used as the plain English equivalents of the corresponding terms "comprising" and "wherein". Also, in the following claims, the terms "including" and "comprising" are open ended, that is, a system, user equipment (UE), article, composition, formulation or process, if Inclusion of elements other than those listed after such term in the claim is still considered to be within the scope of the claim. Furthermore, in the following claims, the terms "first", "second" and "third" etc. are used only as labels and are not intended to impose numerical requirements on their objects.

An "Abstract" of this disclosure is provided to comply with patent law requirements that require an abstract that will enable the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the requested items. In addition, in the foregoing "Description of Embodiments," it can be seen that various features are grouped in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter may lie in less than all features of a single disclosed embodiment. Therefore, the following claims are hereby incorporated into the "embodiments", wherein each claim exists independently as a separate embodiment.

102: Features 103: message word 104:Message concept 105: Communication history 106: Past User Behavior 107: Message subject 108: Other message attributes 109: sender 110:User information 112: Training data 114: Operation 116:Machine Learning Programs 118: New information 120: Evaluation 202: source domain data 204: Neural Networks 206: layer 208: Neurons 302: block 304: category 306: Image 308: block 310: block 312: Image 314: block 402: Feature extraction layer 406: layer 407: layer 408: layer 409: layer 410: layer 411: layer 412: layer 413: layer 414: Classifier layer 500: computer 502: Processor 504: main memory 506: static memory 508: cross-link 510: Video display unit 512: Alphanumeric input device 514: User interface (UI) navigation equipment 516: storage device 518: Signal generating equipment 520: Network interface equipment 521: sensor 522: Machine-readable media 524: instruction 526: Communication network 528: Output controller 600: Schematic diagram 602: camera 604: Object 606:TGV 608: chip light emitting diode 610: Diffuse backlight 700: TGV defect inspection system 702: Area camera 704: Rotation unit 706: Optical power supply 708: backlight illuminator 710: X/Y movement 800: Image 802: waist 804:Entrance and exit 806: side view 1200: cross section 1202: quiver mark 1204: Image 1300: Image 1302: No defect guide hole 1304: Defective guide hole 1306: Defective guide hole 1400: Image 1402: narrow waist guide hole 1404: blind guide hole 1406: raised guide hole 1408: Good (no defect) via 1500: Image 1502: Image 1504: Image 1600: Guide hole classification diagram 1602: Guide hole classification diagram 1604: Guide hole classification diagram 1700: Guide hole classification diagram 1702: Guide hole classification diagram 1704: Guide hole classification diagram 1900: System 1902: X Movement 1904:Y movement 1906: Camera 1908: LED backlit vehicle 1910: Turntable 2100: Machine Learning Programs 2102: Marking Tool 2104: ML Classification 2106: Remark and select tools 2108: training data set 2200: Computing equipment 2210: Processing Circuitry 2220: Memory 2230: Image generation engine 2240: Computer Vision Engine 2250: classification engine 2300: method 2310: block 2320: block 2330: block 2340: block 2350: block 2360: block 2370: block

Figure 1 illustrates the training and use of a machine learning program in accordance with some embodiments.

Figure 2 illustrates an example neural network in accordance with some embodiments.

FIG. 3 illustrates the training of an image recognition machine learning program according to some embodiments.

Figure 4 illustrates the feature extraction process and classifier training in accordance with some embodiments.

Figure 5 is a block diagram of a computing machine in accordance with some embodiments.

6 illustrates an example schematic diagram of glass via (TGV) inspection using a backlight, in accordance with some embodiments.

Figure 7 illustrates an example TGV defect inspection system in accordance with some embodiments.

8 illustrates example TGV characterizations via the waist, via the inlet and outlet, and via the side view using an optical microscope, in accordance with some embodiments.

9 illustrates example computed topography (CT) scan images of a TGV showing defect-free vias and vias with internal defects, in accordance with some embodiments.

10A-10B illustrate individual via images cropped from raw area scan images, their classified shapes/internal defects, and their positions transformed and mapped to part coordinates, according to some embodiments.

Figure 11 illustrates defective TGVs with different two-dimensional (2D) features depending on defect type and optical configuration, according to some embodiments.

Figure 12 illustrates a cross section of an example via CT scan image in accordance with some embodiments.

13 illustrates example line scan images using backlighting including non-defective vias and defective vias, in accordance with some embodiments.

Figure 14 illustrates example images of different types of vias in accordance with some embodiments.

15 illustrates example TGV images of non-metallized and metallized TGVs at oblique angles, in accordance with some embodiments.

Figure 16 illustrates a first example via classification diagram in accordance with some embodiments.

Figure 17 illustrates a second example via classification diagram in accordance with some embodiments.

Figure 18 illustrates example area scan imagery machine learning classification results in accordance with some embodiments.

Figure 19 illustrates an example TGV inspection system in accordance with some embodiments.

Figure 20 illustrates an example relationship between internal defect rate and etch delay, in accordance with some embodiments.

Figure 21 illustrates an example machine learning program in accordance with some embodiments.

Figure 22 is a block diagram of a computing device for TGV defect inspection, according to some embodiments.

23 is a flowchart of a computer-implemented method for TGV defect inspection, according to some embodiments.

Domestic deposit information (please note in order of depositor, date, and number) none

Overseas storage information (please note in order of storage country, organization, date, and number) none

102: Features

103: message word

104:Message concept

105: Communication history

106: Past User Behavior

107: Message subject

108: Other message attributes

109: sender

110:User information

112: Training data

114: Operation

116:Machine Learning Programs

118: New information

120: Evaluation

Claims (20)

A method comprising the steps of: An illumination source and an imaging device are controlled by a computing device to generate images of through glass vias (TGVs), wherein the TGVs comprise etched or metalized microvias in a transparent workpiece comprising at least The two surfaces, wherein controlling the illumination source and the imaging device include performing the following steps on a plurality of TGVs: directing the illumination source to shine light onto the TGVs in the workpiece; and directing the imaging device to detect a scattered image signal from light scattered by the TGVs, wherein an imaging axis of the imaging device extends a non-zero imaging angle relative to an axis of a given TGV from the plurality of TGVs, wherein the entirety of the given TGV is within a field of view of the imaging device; using an image generation engine at the computing device to generate images of the TGVs based on the scattered image signals detected by the imaging device; using a computer vision engine at the computing device to calculate the coordinates of the TGVs within the images based on the scattered image signals detected by the imaging device; for one or more TGVs in the images and based on the calculated coordinates, using a classification engine at the computing device to generate a TGV type classification, wherein the TGV type classification indicates whether the TGV has or lacks a defect, and if the TGV has a defect, indicate a defect type; determining a plurality of TGVs having a defect associated with a given defect type based on the generated TGV type classifications; at the computing device, identifying a manufacturing machine setting associated with the given defect type; and From the computing device, a control signal is transmitted to adjust the manufacturing machine settings. The method as claimed in claim 1, wherein the imaging device includes a camera and an imaging lens arranged on the imaging axis, wherein the camera includes an area scan camera or a line scan camera, wherein the non-zero imaging angle is between 30 degrees and 60 degrees. The method of claim 2, wherein the imaging lens includes a magnification factor and a numerical aperture that depend on a thickness of the transparent workpiece. The method of claim 3, wherein the thickness of the transparent workpiece is between 1 and 1000 microns, and wherein the magnification factor is less than 3. The method of claim 1, wherein the control signal is transmitted directly to a manufacturing machine to adjust settings of the manufacturing machine there. The method as described in claim item 5, further comprising: access additional images of TGVs manufactured after adjusting the manufacturing machine settings; determining whether the defect associated with the given defect type has changed in frequency; and Additional control signals are transmitted from the computing device to the manufacturing machine to further adjust the manufacturing machine settings based on a change in frequency of the given defect type. The method of claim 1, wherein the control signal is transmitted to a user device to instruct an end user to adjust the manufacturing machine settings. The method as described in claim item 1, further comprising: transmitting the generated TGV type classifications to a UE; receiving an indication from the UE that at least one TGV type classification is incorrect; and Further training is provided to the TGV type classifier based on this indication. The method of claim 1, wherein the generated images are of a plurality of different image types including one or more of the following: RGB images, black and white images, shadow images, and CT scan images . The method of claim 1, wherein the classification engine comprises an artificial neural network (ANN) trained using online supervised learning. A non-transitory machine-readable medium storing instructions that, when executed by processing circuitry, cause the processing circuitry to: controlling an illumination source and an imaging device to generate images of through glass vias (TGVs), wherein the TGVs comprise etched or metalized microvias in a transparent workpiece comprising at least two surfaces, wherein Controlling the illumination source and the imaging device includes performing the following steps on a plurality of TGVs: directing the illumination source to shine light onto the TGVs in the workpiece; and directing the imaging device to detect a scattered image signal from light scattered by the TGVs, wherein an imaging axis of the imaging device extends a non-zero imaging angle relative to an axis of a given TGV from the plurality of TGVs, wherein the entirety of the given TGV is within a field of view of the imaging device; using an image generation engine to generate images of the TGVs based on the scattered image signals detected by the imaging device; using a computer vision engine to calculate the coordinates of the TGVs within the images based on the scattered image signals detected by the imaging device; for one or more TGVs in the images and based on the calculated coordinates, using a classification engine to generate a TGV type classification, wherein the TGV type classification indicates whether the TGV has or lacks a defect, and if the TGV has a defect, indicating a defect type; determining a plurality of TGVs having a defect associated with a given defect type based on the generated TGV type classifications; identifying a manufacturing machine setup associated with the given defect type; and A control signal is transmitted to adjust the manufacturing machine settings. The machine-readable medium of claim 11, wherein the imaging device includes a camera and an imaging lens disposed on the imaging axis, wherein the camera includes an area scan camera or a line scan camera, wherein the non-zero imaging The angle is between 30 degrees and 60 degrees. The machine-readable medium of claim 12, wherein the imaging lens includes a magnification factor and a numerical aperture that depend on a thickness of the transparent workpiece. The machine-readable medium of claim 13, wherein the thickness of the transparent workpiece is between 1 and 1000 microns, wherein the magnification factor is less than 3. The machine readable medium of claim 11, wherein the control signal is transmitted directly to a manufacturing machine to adjust settings of the manufacturing machine there. The machine-readable medium of claim 15, further storing instructions that, when executed by the processing circuitry, cause the processing circuitry to: access additional images of TGVs manufactured after adjusting the manufacturing machine settings; determining whether the defect associated with the given defect type has changed in frequency; and Additional control signals are transmitted to the manufacturing machine to further adjust the manufacturing machine settings based on a change in frequency of the given defect type. The machine readable medium of claim 11, wherein the control signal is transmitted to a user device to instruct an end user to adjust the manufacturing machine settings. A system comprising: processing circuitry; and A memory storing instructions that, when executed by the processing circuitry, cause the processing circuitry to: controlling an illumination source and an imaging device to generate images of through glass vias (TGVs), wherein the TGVs comprise etched or metalized microvias in a transparent workpiece comprising at least two surfaces, wherein Controlling the illumination source and the imaging device includes performing the following steps on a plurality of TGVs: directing the illumination source to shine light onto the TGVs in the workpiece; and directing the imaging device to detect a scattered image signal from light scattered by the TGVs, wherein an imaging axis of the imaging device extends a non-zero imaging angle relative to an axis of a given TGV from the plurality of TGVs, wherein the entirety of the given TGV is within a field of view of the imaging device; using an image generation engine to generate images of the TGVs based on the scattered image signals detected by the imaging device; using a computer vision engine to calculate the coordinates of the TGVs within the images based on the scattered image signals detected by the imaging device; for one or more TGVs in the images and based on the calculated coordinates, using a classification engine to generate a TGV type classification, wherein the TGV type classification indicates whether the TGV has or lacks a defect, and if the TGV has a defect, indicating a defect type; determining a plurality of TGVs having a defect associated with a given defect type based on the generated TGV type classifications; identifying a manufacturing machine setup associated with the given defect type; and A control signal is transmitted to adjust the manufacturing machine settings. The system of claim 18, wherein the imaging device includes a camera and an imaging lens disposed on the imaging axis, wherein the camera includes an area scan camera or a line scan camera, wherein the non-zero imaging angle is between 30 degrees and 60 degrees. The system of claim 19, wherein the imaging lens includes a magnification factor and a numerical aperture that depend on a thickness of the transparent workpiece.

Images