JP4728968B2 - Data analysis method and apparatus - Google Patents

Description

(Refer to related applications)
This application
U.S. Patent Application No. 10 / 817,750 (filed April 2, 2004, partly pending application of "METHODS AND APPARATUS FOR ANALYZING TEST DATA"), which is U.S. Patent Application No. 10/730 , 388 (filed on Dec. 7, 2003, the title of the invention “METHODS AND APPARATUS FOR DATA ANALYSIS”), which is a co-pending application of US Patent Application No. 10 / 367,355 (February 14, 2003). No. 10 / 154,627 (filed on May 24, 2002, name of invention “METHODS AND APPARATUS”), which is a partially pending application of the title “METHODS AND APPARATUS FOR DATA ANALYSIS”. FOR SEMICONDUCTOR TESTING ", which is a partially pending application of US Patent Application No. 09 / 872,195 (filed May 31, 2001, part of the invention" METHODS AND APPARATUS FOR DATA SMOOTHING ") US Patent Application No. 60 / 293,577 (May 24, 2001, title of invention "METHODS AND APPARATUS FOR DATA SMOOTHING", US Provisional Application No. 60 / 295,188 (2001) Filed on May 31, entitled “METHODS AND APPARATUS FOR TEST DATA CONTROL AND ANALYSIS”, and US Provisional Patent Application No. 60 / 374,328 (filed April 21, 2002, It claims the benefit of the Ming, entitled "METHODS AND APPARATUS FOR TEST PROGRAM ANALYSIS AND ENHANCEMENT"
US Provisional Patent Application No. 60 / 542,459 (filed on Feb. 6, 2004, claiming the benefit of EVOLVING NEURAL NETWORKS USING SWARM INTELLIGENCE FOR BINGMAP CLASSIFICATION,
US Provisional Patent Application No. 60 / 546,088 (filed on Feb. 19, 2004, claiming the benefit of the invention name “DYNAMIC OUTLIER ALGORITHM SELECTION QUALITY IMPROVEMENT” AND TEST PROGRAM OPTIMIZATION,
This application uses the disclosure of each application as a reference. However, in the event of a conflict with any application to which this disclosure refers, the present disclosure shall prevail.

(Field of Invention)
The present invention relates to data analysis.

(Background of the Invention)
The semiconductor company tests the component to make sure it works properly. The test data not only determines whether this component functions properly, but also determines whether defects are pointed out in the manufacturing process. Thus, many semiconductor companies analyze data collected from several different components to identify and correct problems. For example, a company may collect test data for multiple chips on each wafer from several different lots. Test data can come from various sources such as parametric electrical tests, optical inspection, scanning electron microscopy, energy dispersive x-ray spectroscopy, and focused ion beam processes for defect analysis and fault isolation. This data is used to identify common defects or defect patterns, or to identify parts that may exhibit quality or performance problems, as well as to identify or classify user-defined “good parts” Can be analyzed. Steps are then taken to correct this problem. Testing is typically performed before device packaging (at the wafer level) and after assembly is complete (final test).

  Collecting and analyzing test data is expensive and time consuming. The automatic tester applies a signal to the component and reads the corresponding output signal. The output signal can be analyzed to determine if the component is operating correctly. Each tester generates a large amount of data. For example, each tester may perform 200 tests on a single component, and each test may be repeated 10 times. As a result, a test on a single component will yield 2000 results. Because each tester tests more than 100 components per hour and several testers may be connected to the same server, a huge amount of data must be stored. In addition, for data processing, servers typically store test data in a database to facilitate data manipulation and analysis. However, storage in a conventional database requires additional storage capacity and time to organize and store the data.

  Furthermore, obtaining test data is a complex and laborious process. A test engineer prepares a test program that instructs the tester to generate an input signal to the component and receive the output signal. This program tends to be very complex in order to ensure that the components work full and accurate. As a result, a moderately complex integrated circuit test program produces a large amount of tests and results. In order to obtain a satisfactory solution, the program preparation requires extensive design and changes, and program optimization involves, for example, removing redundant tests or minimizing test time, etc. More effort is required.

  Analysis of the collected data is also difficult. The amount of data can require significant processing power and time. As a result, data is typically not analyzed at product runtime, but instead is typically analyzed between test runs or batches. To alleviate some of these burdens, some companies only sample data from the tester and dispose of others. However, an analysis that does not analyze all of the data ensures that the resulting analysis cannot be sufficient in completeness or accuracy. As a result, sampling is far from complete understanding of the test results.

  Furthermore, even if a complete set of test data generated by the tester is maintained, the vast amount of test data creates difficulties in analyzing the data and extracting meaningful results. Test data may include meaningful information regarding devices, test processes, and manufacturing processes, and this information may also be used to improve manufacturing, reliability, and testing. However, given the amount of data, separating and presenting information to a user or other system is challenging.

  In addition, many data interpretations are performed manually by engineers. Based on their own experience and knowledge in the manufacturing and testing process, the engineer reviews the data and makes inferences about the testing and manufacturing process. Although manual analysis is often useful, engineers tend to understand the manufacturing system and the test system separately and therefore tend to reach different subjective conclusions based on the same data. Another problem arises when skilled personnel leave the company or otherwise become inconvenient. The knowledge and understanding of a skilled person's manufacturing and testing system cannot be easily passed on to other persons.

  A more complete understanding of the present invention can be derived with reference to the detailed description and claims when considered in conjunction with the following illustrative drawings. These drawings may not be at a constant magnification. Like reference numerals refer to like elements throughout the drawings.

  Elements in the figures are drawn for simplicity and clarity and are not necessarily divided at a constant magnification. For example, the connections and steps performed by some of the elements in the figure are exaggerated or omitted compared to other elements to help better understand the embodiments of the present invention. There is also.

Detailed Description of Exemplary Embodiments
The present invention may be described in terms of functional block components and various process steps. Such functional blocks and steps may be implemented by any number of hardware or software configured to perform specific functions. For example, the present invention may use various testers, processors, storage systems, processes, and algorithms such as statistical engines, memory elements, signal processing elements, neural networks, pattern analysis, logic elements, programs, and the like. They can perform various functions under the control of one or more testers, microprocessors or control devices. Further, the present invention can be implemented with any number of test environments. Also, each described system is only one example application for the present invention. Furthermore, the present invention may use any number of conventional techniques such as data analysis, component interfacing, data processing, component handling and the like.

  With reference to FIG. 1, methods and apparatus in accordance with various aspects of the present invention operate in conjunction with a test system 100 having a tester 102, such as an automatic test equipment (ATE) for semiconductor testing. In the present embodiment, the test system 100 includes a tester 102 and a computer system 108. Test system 100 may be configured to test any component 106 such as a semiconductor device, circuit board, package device, or other electrical or optical system on a wafer. In this embodiment, component 106 comprises a number of integrated circuit die (die) or packaged integrated circuits or devices formed on a wafer. Component 106 is formed using a fabrication process. The process may comprise any suitable manufacturing process for generating component 106 and may include a test process that may comprise any suitable process for testing the operation of component 106.

  Tester 102 suitably comprises any test device that tests component 106 and generates output data relating to the test, and may comprise a number of devices or other data sources. Tester 102 may comprise a conventional automatic tester such as a Teradyne tester and will work properly with other devices that facilitate the test. Tester 102 is selected and configured according to the particular component 106 to be tested and / or any other suitable criteria.

  Tester 102 may operate with computer system 108. The computer system 108 performs, for example, programming of the tester 102, loading and / or execution of a test program, data collection, provision of instructions to the tester 102, test data analysis, tester parameter control, and the like. In this embodiment, the computer system 108 receives tester data from the tester 102 and executes various data analysis functions independently of the tester 102. The computer system 108 may implement a statistical engine that analyzes data from the tester 102 and a diagnostic system 216 for identifying potential problems in the manufacturing and / or testing process based on the test data. The computer system 108 may comprise a separate computer such as a personal computer or workstation. The computer is connected to or networked with the tester 102 to exchange signals with the tester 102. In alternative embodiments, the computer system 108 may be omitted from or incorporated into other components of the test system 100. Various functions may be performed by other components, such as a tester 102 or element connected to the network.

  In the exemplary system, computer system 108 includes a processor 110 and a memory 112. The processor 110 may be any suitable processor (conventional Intel, Motorola, for example) that operates in conjunction with any suitable operating system (eg, Windows XP®, Unix®, or Linux®). Or an Advanced Micro Devices processor). Similarly, the memory 112 may comprise any suitable memory that is accessible to the processor 110 for data storage (eg, random access memory (RAM) or other suitable storage system). In particular, the memory 112 of the present system includes a high-speed access memory that stores and receives information and is suitably configured to have sufficient capacity to facilitate the operation of the computer 108.

  In the present embodiment, the memory 112 includes a capacity for storing the output result received from the tester 102 and facilitating analysis of the output test data. The memory 112 is configured to store and retrieve analysis test data at high speed. In various embodiments, the memory 112 is configured to store elements of a dynamic data log. The element suitably comprises a set of information selected by the test system 100 and / or the operator according to criteria and analysis selected based on the test results.

  For example, the memory 112 suitably stores a component identifier for each component 106 (eg, xy coordinates corresponding to the position of the component 106 on the wafer map relative to the tested wafer). Each xy coordinate in memory 112 may be associated with a particular component 106 at the corresponding xy coordinate on the wafer map. Each component identifier has one or more fields. Each field corresponds to, for example, a specific test performed on the component 106 at the corresponding xy location on the wafer, statistics associated with the corresponding component 106, or related data. The memory 112 may be configured to include any data identified by the user based on any criteria or rules as desired.

  The computer 108 of this embodiment also provides appropriate access to the storage system (eg, other memory (or a portion of the memory 112), hard drive array, optical storage system, or other suitable storage system). Have The storage system may be local, such as a hard drive dedicated to the computer 108 or tester 102, or remote, such as a hard drive array associated with the server to which the test system 100 is connected. . The storage system may store programs and / or data used by computer 108 or other components of test system 100. In this embodiment, the storage system includes a database 114 that can be used via the remote server 116. The remote server 116 includes, for example, a main manufacturing server for production facilities. The database 114 stores tester information (for example, a tester data file, a master data file for operating the test system 100, and its components, test programs, downloadable instructions for the test system 100, etc.). Further, the storage system may comprise a complete tester data file, such as a history tester data file retained for analysis.

  Test system 100 may include additional devices that facilitate testing of component 106. For example, the test system 100 includes a device interface 104. The device interface 104, like a conventional device interface board and / or device handler or prober, handles the component 106 and provides an interface between the component 106 and the tester 102. In one embodiment, the device interface comprises a multi-site device interface configured to test multiple sites on a single wafer simultaneously. Test system 100 may include other components, software, etc. to facilitate testing of test component 106, depending on the particular configuration, application, environment, or other relevant factors of test system 100, or these May be connected. For example, in the present embodiment, the test system 100 may use a suitable communication medium, such as a local area network, an intranet, or a global network such as the Internet, to send information to other systems such as a remote server 116. Connected to.

  Test system 100 may include one or more testers 102 and one or more computers 108. For example, one computer 108 may be connected to an appropriate number (eg, up to 20 or more) of testers 102 depending on various factors such as the system processing capability and configuration of the computer 108. Further, the computer 108 may be separate from the tester 102 or may be incorporated into the tester 102 using, for example, one or more processors, memory, clock circuits, etc. of the tester 102 itself. In addition, various functions may be performed by different computers. For example, the first computer performs various pre-analysis tasks, then several computers receive data and perform data analysis, and another set of computers perform dynamic data logging and / or other output analysis And prepare a report.

  Test system 100 in accordance with various aspects of the present invention tests component 106 and provides extended analysis and test results. For example, extended analysis can identify inaccurate, questionable, or abnormal results, repeated tests, and / or tests that are relatively likely to be defective. Test system 100 may also analyze multiple data sets (eg, data obtained from multiple wafers and / or lots of wafers) to generate a databased composite in multiple databases. Various data also diagnose characteristics in manufacturing, testing and / or other processes (eg, problems, inefficiencies, potential hazards, instabilities or other aspects that can be identified via test data) Therefore, it can also be used by the test system 100. An operator (eg, a product engineer, test engineer, production engineer, device engineer, or other staff using test data and analysis) then validates and / or improves the test system 100 and / or the manufacturing system to determine whether the component 106 The result can be used to classify.

  Test system 100 in accordance with various aspects of the present invention performs an extended test process for testing component 106, collecting and analyzing test data. Test system 100 works properly with software applications running on computer 108. Referring to FIG. 2, the software application of this embodiment includes a number of elements (including a configuration element 202, a supplemental data analysis element 206, and an output element 208) for implementing an extended test process. The test system 100 may also include a composite analysis element 214 for analyzing data from more than one data set. Further, the test system can include a diagnostic system 216 to identify characteristics and potential problems using the test data.

  Each element 202, 206, 208, 214, 216 suitably comprises a software module running on the computer 108 that performs various tasks. In general, the component element 202 prepares the test system 100 for testing and analysis. In supplemental data analysis element 206, the output test data from tester 102 is automatically analyzed, suitably at runtime, to generate supplemental test data. The supplemental test data is then transmitted to the operator or other system, such as composite analysis element 214, diagnostic system 216, and / or output element 208.

  The configuration element 202 configures the test system 100 to test the component 106 and analyze the test data. The test system 100 appropriately uses information from an operator to configure the test system 100 with a predetermined initial parameter set and, if necessary. In order to minimize operator attendance to the test system 100, the test system 100 is initially suitably configured with predetermined or default parameters. Adjustments are made to the configuration as needed, for example, by the operator via the computer 108.

Referring to FIG. 3, the exemplary configuration process 300 performed by the configuration element 202 begins with an initialization procedure (step 302) that sets the computer 108 to an initial state. The configuration element 202 then obtains application configuration information from, for example, the computer 108 and the database 114 for the tester 102 (step 304). For example, the configuration element 202 may access a master configuration file for the extended test process and / or a tool configuration file related to the tester 102. The master configuration file may include data related to the exact configuration for the computer 108 and other components of the test system 100 to perform the extended test process. Similarly, the tool configuration file suitably includes data relating to the configuration of the tester 102 (eg, connection, directory, IP address, tester node identification, manufacturer, flag, prober identification, or any other information associated with the tester 102). Included.
23 The configuration element 202 may then configure the test system 100 according to the data contained in the master configuration file and / or the tool configuration file (step 306). Additionally, the configuration element 202 may use configuration data that further derives relevant information from the database 114 (eg, an identifier of the tester 102 that associates data such as logistic status for the tester data with the tester 102) (step 308). The test system 100 information also suitably includes one or more default parameters that may be approved, rejected or adjusted by the operator. For example, the test system 100 information may include global statistical process management (SPC) rules and goals. These rules and goals are presented to the operator according to installation, configuration, power-up, or other appropriate time required for approval and / or modification. The test system 100 information may also include default wafer maps or other files for products, wafers, components 106 or other items, respectively. These items can affect or be affected by the test system 100. Configuration algorithms, parameters, and any other criteria can be stored in the recipe file for easy access to correlation with specific products and / or tests and for traceability.

  When the initial configuration process is complete, the test system 100 starts a test run according to the test program, eg, along with a series of conventional tests. The tester 102 applies a signal connected to the component 106 and appropriately executes a test program that reads output test data from the component 106. The tester 102 may perform a number of tests on each component 106 on the wafer or on the wafer itself, and each test may be repeated several times on the same component 106. Tests can include, but are not limited to, continuity, supply current, leakage current, parametric static, parametric dynamic, and functionality and stress tests. Test data from the tester 102 is stored for quick access and supplemental analysis once the test data is obtained. Data may be stored in long-term memory for later analysis and use.

  Each test produces at least one result for at least one component. Referring to FIG. 9, an exemplary set of test results for a single test of a number of components is characterized by a first test result set having statistically similar values and values that deviate from the first set. Contains a second test result set to be attached. Each test result can be compared to an upper test limit and a lower test limit. If a particular result of a component exceeds any limit, the component may be classified as a “bad part” or may be classified according to tests and / or test results.

  Some of the second set of test results that deviate from the first set may or may not exceed the control limits. For this purpose, these test results that fall outside the first set but do not exceed the control limits or otherwise are not detected are called “outliers”. Outliers in test results can be identified and analyzed for any suitable purpose, such as, for example, identifying potentially unreliable components. Outliers can also be used to identify various potential problems and / or improvements in testing and production processes.

  When tester 102 generates test results, output test data for each of the component, test, and iteration is stored by tester 102 in a tester data file. The output test data received from each component 106 is analyzed by the tester 102 to classify the performance of the component 106 (e.g., classify into a specific bin classification, etc., by comparing the upper and lower test limits). . The classification result is also stored in the tester data file. The tester data file may include additional information such as logistic data and test program identification data. The test data file is then provided to the computer 108 in an output file, such as a standard tester data format (STDF) file, and stored in memory. The tester data file may also be stored in the storage system for long-term storage for later analysis (eg, performed by the composite analysis element 214).

  When the computer 108 receives the tester data file, the supplemental data analysis element 206 analyzes to provide an extended output result. The supplemental data analysis element 206 may provide any suitable analysis of the tester data to achieve any suitable purpose. For example, supplemental data analysis element 206 may implement a statistical engine. The engine analyzes the output test data at runtime to identify the data of interest to the operator and its data characteristics. The identified data and characteristics can be stored, while the unidentified data can be disposed of, such as discarded.

  The supplemental data analysis element 206 may calculate statistical figures according to, for example, a set of data and statistical configuration data. The statistical configuration data can be any suitable type of analysis, such as statistical process management, outlier identification and classification, signature analysis, and data correlation, according to the needs of the test system 100 and / or the operator. You can direct. In addition, the supplemental data analysis element 206 performs analysis appropriately at run time (ie, in seconds or minutes before the next test data generation). The supplemental data analysis element 206 may also perform analysis automatically with minimal interference from operators and / or test engineers.

  In the present test system 100, after the computer 108 receives and stores the tester data file, the supplemental data analysis element 206 performs various preliminary tasks. This is because the computer 108 prepares the analysis of the output test data and facilitates the generation of supplementary data and the preparation of the output report. 4A to 4C will be described here. In this embodiment, the supplemental data analysis element 206 first copies the tester data file to the tool input directory corresponding to the associated tester 102 (step 402). The supplemental data analysis element 206 also retrieves configuration data to prepare the computer 108 for supplemental analysis of the output test data.

  The configuration data suitably includes a set of logistic data that can be retrieved from the tester data file (step 404). The supplemental data analysis element 206 also generates logistic criteria (step 406). The logistic criteria may include tester 102 information, such as tester 102 information derived from the tool configuration file. In addition, logistic criteria are assigned to identification.

  The configuration data may also include an identifier for the test program that generated the output test data. The test program may be identified in any suitable manner, such as by collating it 408 in the database 114, in conjunction with the identification of the tester 102, or reading it from the master configuration file. If the test program identification cannot be established (step 410), the test program identification can be generated and combined with the tester identification (step 412).

  If the configuration data is less than all of the wafers, the wafers in the test run are further identified for processing by the supplemental data analysis element 206. In this embodiment, supplemental data analysis element 206 accesses a file indicating which wafers are to be analyzed (step 414). If no instructions are provided, the computer 108 does not perform the analysis properly on all wafers in the test run.

  If the wafer for the current tester data file is to be analyzed (step 416), the supplemental data analysis element 206 proceeds to perform supplemental data analysis of the tester data file for the wafer. Otherwise, the supplemental data analysis element 206 waits for or accesses the next tester data file (step 418).

  The supplemental data analysis element 206 may establish one or more section groups to be analyzed for the various wafers to be tested (step 420). In order to identify the appropriate section group to apply to the output test data, supplemental data analysis element 206 properly identifies the appropriate section group definition. For example, it is identified by a test program and / or tester identification. Each section group includes one or more section arrays, and each section array includes one or more sections of the same section type.

  Section types include various types of groups of components 106 located in a predetermined area of the wafer. For example, referring to FIG. 5, section types may include rows 502, columns 504, stepper fields 506, annular bands 508, radial zones 510, quadrants 512, or any other desired grouping of components. Different section types may be used depending on the order of the components being processed, the configuration of the components, such as the sections of the tube. Such groups of components 106 are analyzed together, for example, to identify common defects or characteristics that may be associated with the group. For example, if a particular portion of the wafer does not conduct heat like other portions of the wafer, the test data for a particular group of components 106 may reflect common characteristics or defects that are linked to uneven heating of the wafer. .

  Once the section group for the current tester data file is identified, supplemental data analysis element 206 retrieves any additional relevant configuration data, such as test limits and / or enable flags for tester 102 (step 422). In particular, supplemental data analysis element 206 appropriately retrieves the desired set of statistics or calculations associated with each section array within the section type (step 423). The desired statistics and calculations can be specified by the operator or in any manner, such as retrieval from a file. In addition, the supplemental data analysis element 206 may also identify one or more signature analysis algorithms for each relevant section type or other suitable variation for the wafer (step 424), and similarly retrieve the signature algorithm from the database 114. Can be taken out.

  All of the configuration data may be provided by default or automatically accessed by the configuration element 202 or supplemental data analysis element 206. Furthermore, the configuration element 202 and the supplemental data analysis element 206 of the present embodiment allow the operator to appropriately change the configuration data according to the operator's request or the test system 100 request. If configuration data is selected, the configuration data may be associated with relevant criteria and saved for future use as default configuration data. For example, if the operator selects a section group for a particular type of component 106, the computer 108 may automatically use the same section group for all such components 106. However, this does not apply when other instructions are given by the operator.

  The supplemental data analysis element 206 also provides the configuration and storage of tester data files and additional data. Supplemental data analysis element 206 appropriately allocates memory, such as a portion of memory 112, for the data to be stored (step 426). Allocate appropriate memory for all data to be stored (including output test data from tester data files, statistical data generated by supplemental data analysis element 206, management parameters, etc.) by supplemental data analysis element 206 To provide. The amount of memory allocated may be calculated, for example, depending on the number of tests performed on the component 106, the number of section group arrays, control limits, statistical calculations to be performed by the supplemental data analysis element 206, and the like.

  When all the configuration data is ready to perform supplemental analysis, as soon as output test data is received, supplemental data analysis element 206 loads the relevant test data into memory (step 428) and outputs test data. Perform supplementary analysis. The supplemental data analysis element 206 may perform any number and type of data analysis depending on the component 106, the configuration of the test data system 100, the operator's desires, or other relevant criteria. The supplemental data analysis element 206 is for selected features that identify potentially defective components 106 and patterns, trends, or other features in the output test data that may indicate production concerns or deficiencies. May be configured to parse the section.

  The supplemental data analysis element 206 identifies data and / or components 106 corresponding to various criteria, for example, by calculating and analyzing various statistics based on the output test data. The supplemental data analysis element 206 may also classify and contrast output test data to provide information to operators and / or test engineers associated with the component 106 and the test system 100. For example, the supplemental data analysis element 206 may, for example, compare output data to identify potentially relevant or redundant tests, and outlier occurrence rates to identify tests that frequently generate outliers. Analysis can be performed.

  The supplemental data analysis element 206 can include a smoothing system that first processes the tester data to smooth the data and to help identify outliers (step 429). In an alternative embodiment, the smoothing system and process may be omitted and the data is processed without smoothing. The smoothing system can also identify significant changes in data, trends, etc., and these changes can be provided to the operator by output element 208. The smoothing system is suitably implemented as a program that runs on the computer system 108, for example. The smoothing system suitably comprises a number of phases for smoothing the data according to various criteria. The first phase may include a basic smoothing process. The supplemental phase conditionally provides test data for enhanced tracking and / or additional smoothing.

  The smoothing system works properly by first adjusting the initial value of the selected tester data by the first smoothing technique and supplementarily adjusting the value by the second smoothing technique. , Where the initial value and the first adjusted value meet the threshold. The first smoothing technique tends to smooth the data. The second smoothing technique also tends to smooth the data and / or tends to improve the tracking of the data, but by a different method than the first smoothing technique. In addition, the threshold may include any suitable criteria for determining whether to apply supplemental smoothing. The smoothing system appropriately compares a plurality of pre-adjusted data with a plurality of previous raw data to generate a comparison result, which is selected depending on whether the comparison result meets the first threshold. A second smoothing technique is applied to the selected data to adjust the value of the data. In addition, the smoothing system appropriately calculates the expected value of the selected data and applies a third smoothing technique to the selected data, and is selected depending on whether the expected value meets the second threshold. The reference value can be adjusted.

  Referring to FIG. 8, the first smoothed test data point is appropriately set to be equal to the first raw test data point (step 802). The smoothing system then proceeds to the next raw test data point (step 804). Prior to performing a smoothing operation, the smoothing system first determines whether smoothing is appropriate for the data point and, if appropriate, performs a basic smoothing operation on the data. To determine whether smoothing is appropriate, any criteria can be used, such as the number of data points received, the deviation of the data point value from the selected value, or the comparison of each data point value with a threshold. May be applied. In this embodiment, the smoothing system performs a comparison with a threshold value. Comparison with the threshold determines whether data smoothing is appropriate. If deemed appropriate, the initial smoothing process is suitably configured to proceed to the initial smoothing of the data.

More specifically, in this embodiment, the process begins with the first raw data point _R0 . R ₀ is also designated as the first smoothed data point S ₀ . As additional data points are received and analyzed, the difference between each raw data point (R _n ) and the previously smoothed data point (S _n−1 ) is calculated and the threshold (T _1). ) (Step 806). If the difference between the raw data point R _n and the previously smoothed data point S _n−1 exceeds the threshold T ₁ , the excess threshold corresponds to a considerable divergence from the smoothed data; It is assumed to indicate a data shift. Thus, the occurrence of a threshold crossing may be recorded. The data points _{S n} to be currently smoothing is set equal to the raw data points _{R n} (step 808). No smoothing is performed and the process proceeds to the next raw data point.

The difference between the raw data point, the preceding to the smoothed data points, if the threshold is not exceeded T _1, the process, in conjunction with the first smoothing process to compute the data points S _n to be currently smoothed ( Step 810). The first smoothing process provides basic smoothing of the data. For example, in the present embodiment, the basic smoothing process include, for example, a conventional exponential smoothing process, the following _{S n = (R n S n} -1) * M 1 + S n-1
According to the equation Here, M ₁ is a smoothing coefficient such as 0.2 or 0.3.

The initial smoothing process appropriately utilizes a relatively low factor M ₁ to provide a significant amount of smoothing for the data. The initial smoothing process and coefficients may be selected according to any criteria and may be configured in any manner. However, it follows the application of the smoothing system, the processed data, the requirements and capabilities of the smoothing system, and / or any other criteria. For example, the first smoothing process can be random, random walk, moving average, simple exponential function, linear exponential function, seasonal exponential function, exponential function weighted with moving average, or data is first smoothed Any suitable other type of smoothing may be used.

  The data may be further analyzed for smoothing and / or subject to further smoothing. Supplemental smoothing may be performed on the data to enhance smoothing of the data and / or improve tracking of the smoothed data relative to the raw data. Multiple phase supplemental smoothing may be considered and may be applied where appropriate. The various phases can be independent, interdependent, or complementary. In addition, the data may be analyzed to determine whether supplemental smoothing is appropriate.

  In this embodiment, the data is analyzed to determine whether to perform one or more additional phase smoothings. The data is analyzed based on any suitable criteria to determine whether supplemental smoothing can be applied (step 812). For example, the smoothing system identifies a trend in data, such as by comparing a plurality of adjusted data points with raw data points for preceding data and generating a comparison result. The comparison results are based on whether substantially all of the preceding adjusted data shares a common relationship (such as smaller, larger, or equal) with substantially all of the corresponding raw data. Done.

Smoothing system of the present embodiment compares the selected number P ₂ pieces raw data points, the same number of smoothed data points. P ₂ pieces of raw data points all values, corresponding exceeds the smoothed data point (or equal to), or if all raw data points are less than the corresponding smoothed data points (or equivalent) if The smoothing system may determine that the data shows a certain trend and should be tracked more closely. Thus, its occurrence can be recorded. Then, the smoothing applied to the data may change by applying supplemental smoothing. On the other hand, if none of these criteria are met, the currently smoothed data points remain originally calculated and no relevant supplemental data smoothing is applied.

In this embodiment, the criteria for comparing the smoothed data with the raw data is selected to identify trends in the data. The smoothed data may lag behind the trend. Accordingly, the number of points P _2, to the trend of change in the raw data, depending wishes sensitivity of the system to where, may be selected.

  Supplemental smoothing changes the overall smoothing effect according to the data analysis. Any suitable supplemental smoothing may be applied to the data in order to more effectively perform data smoothing or data trend tracking. For example, in this embodiment, if the data trend indicated by the data analysis indicates that it should be tracked more closely, supplemental smoothing is performed first so that the smoothed data tracks the raw data more closely. Applied to reduce the degree of smoothing applied to (step 814).

In the present embodiment, the degree of smoothing is reduced by recalculating the values for the currently smoothed data points using a reduced degree of smoothing. Any suitable smoothing system may be used for more efficient data tracking or otherwise responding to data analysis results. In this embodiment, another conventional exponential smoothing process uses the higher coefficient M ₂ to add S _n = (R _n −S _n−1 ) * M ₂ + S _{n−1 to the data.}
Applies.

The coefficients M ₁ and M ₂ can be selected according to the desired sensitivity of the system, both when there is no trend in the raw data (M ₁ ) and when there is (M ₂ ). In various applications, for example, the value of M ₁ may be higher than the value of M _2.

  Supplemental data smoothing may also include additional phases. The additional phase of data smoothing may similarly parse the data in some way to determine whether additional data smoothing should be applied. Any number of data smoothing phases and types may be applied or considered based on data analysis.

For example, in this embodiment, the data can be analyzed and potentially smoothed for noise control. For example, using a predictive process based on the slope or trend of the smoothed data. The smoothing system is based on selected P ₃ data points that have been smoothed prior to the current data point by any suitable process such as linear regression, N-points centered, etc. To calculate the inclination (step 816). In this embodiment, the data smoothing system uses a “least squares fit through line” process to establish the slope of the previously smoothed P ₃ data points.

The smoothing system predicts the value of the currently smoothed data point from the calculated slope. The system then compares the difference between the previous calculated value for the current smoothed data point (S _n ) and the predicted value for the current smoothed data point to the range number (R ₃ ). (Step 818). If the difference is greater than the range R _3, the appearance is recorded, the data points that are currently smoothed is not adjusted. If the difference is within range R ₃ , then the current smoothed data point is the calculated current smoothed data point (S _n ) and the predicted value (S _n−pred ) for the current smoothed data point. Is set to be equal to the difference between and multiplied by a _third multiplier M3 plus the original value of the current smoothed data point (step 820). The expression is _{S n = (S n-pred} -S n) * M 3 + S n
It is.

Thus, the current smoothed data point is set according to the modified difference between the originally smoothed data point and the predicted smoothed data point. However, it is reduced by a certain amount (if M ₃ is less than 1). Applying predictive smoothing tends to reduce the point-to-point noise sensitivity of relatively flat (or non-trending) parts of the signal. Because the predictive smoothing process on smoothed data points has limited applicability, the mean calculated based on the slope is when a significant change appears in the raw data, that is, when the raw data signal is not relatively flat It is guaranteed that it does not affect the smoothed data.

The supplemental data analysis element 206 may perform any suitable analysis of tester data (including raw tester data, smoothed tester data, or otherwise filtered or processed data). For example, the supplemental data analysis element 206 may filter the tester data to extract the information regarding the nature of the distribution by choosing the right technique to detect defects, warnings, tail thresholds. The supplemental data analysis element 206 may perform statistical process management (SPC) calculations and analysis on the output test data. More specifically, referring again to FIGS. 4A-C, supplemental data analysis element 206 may calculate and store desired statistics for a particular component, test, and / or section. (Step 430). The statistics may include any statistics useful to the operator or test system 100 (eg, mean, standard deviation, minimum, maximum, sum, count, Cp, Cpk, or any other suitable statistic. ).

  The supplemental data analysis element 206 also suitably performs signature analysis to dynamically and automatically identify trends and anomalies in the data (step 442). For example, according to a section, based on a combination of other data such as test results and / or historical data for that section. Signature analysis identifies a signature based on the identification of any suitable data or defects, such as test data, and applies a weighting system appropriately configured by an operator. Signature analysis may cumulatively classify trends and anomalies that may correspond to problem areas or other features in the wafer or manufacturing process. Signature analysis can be performed for any desired signature, such as noise peaks, waveform variations, mode shifts, and noise. In the present embodiment, the computer 108 appropriately performs signature analysis on the output test data for each desired test in each desired section.

In this embodiment, the signature analysis process may be performed along with a smoothing process. As the smoothing process analyzes the tester data, the results of the analysis indicating trends or anomalies in the data are stored to display changes or outliers in the data that may be significant to the operator and / or test engineer. For example, if a trend is displayed by comparison of data sets in a smoothing process, the occurrence of the trend can be recorded and stored. Similarly, if the data point exceeds the threshold value T ₁ of the data smoothing process, the occurrence can be recorded stored for inclusion of analysis and / or output reports after. Alternatively, the smoothing process can be omitted.

  For example, referring to FIGS. 6A-B, signature analysis process 600 may first calculate a count for a particular set of test data and control limits corresponding to a particular section and test (step 602). The signature analysis process then applies an appropriate signature analysis algorithm to the data points (step 604). Signature analysis is performed for each of the desired signature algorithms and then for each test and section to be analyzed. Errors identified by the signature analysis, trend results, and signature results are also stored (step 606). This process is repeated for each signature algorithm (step 608), test (step 610), and section (step 612). Upon completion, supplemental data analysis element 206 records the error (step 614), trend result (step 616), signature result (step 618), and any other desired data in the storage system.

  As soon as each relevant data point is identified, such as outliers and other important data identified by supplemental analysis, each relevant data point may be associated with a value that identifies the relevant characteristic (step 444). For example, each related component or data point can be appropriately represented in hexadecimal and associated with a series of values corresponding to the results of supplemental analysis for that data point. Each value can act as a flag or other designator for a particular characteristic. For example, if a particular data point completely fails a particular test, a first flag may be set to the corresponding hex value. If the particular data point is the beginning of a trend in the data, other flags can be set. Other values in hex may include information about the trend, such as the duration of the trend in the data.

  The supplemental data analysis element 206 may also be configured to classify and associate data (step 446). For example, the supplemental data analysis element 206 may utilize hexadecimal information associated with data points to identify defects, outliers, trends, and other features of the data. The supplemental data analysis element 206 suitably applies correlation techniques to conventional data to identify, for example, potentially redundant tests or related tests.

  Computer 108 may perform additional analysis functions on the generated statistics and output test data such as outliers that are automatically identified and classified (step 432). Analyzing each relevant data according to the selected algorithm, the outliers are properly identified. If a particular algorithm is inappropriate for the data set, the supplemental data analysis element 206 can be configured to automatically stop the analysis and select a different algorithm.

  The supplemental data analysis element 206 can be operated in any suitable way of specifying outliers, such as by comparison with selected values and / or according to data processing of a data smoothing or filtering process. For example, an outlier identification element in accordance with various aspects of the present invention first automatically calibrates its sensitivity to outliers based on a selected statistical relationship for each relevant data (step 434). Some of these statistical relationships can then be used to define threshold limits for relative outliers, then thresholds, or other reference points, such as data mode, mean or median, or a combination thereof. To be compared. In this embodiment, the statistical relationship is scaled by 1, 2, 3 and 6 times the standard deviation of the data to define different outlier amplitudes (step 436). The output test data may then be compared to outlier threshold limits to identify and classify the output test data as outliers (step 438).

  The supplemental data analysis element 206 stores the resulting statistics and outliers in memory and an identifier (eg, xy wafer map coordinates associated with any such statistics and outliers) (step 440). Selected statistics, outliers, and / or defects also trigger notification events (eg, electronic message transmission to operator, light tower trigger, tester 102 outage, or notification to server). obtain.

  In this embodiment, supplemental data analysis element 206 includes a scaling element 210 and an outlier classification element 212. The scaling element 210 is configured to dynamically scale selected coefficients and other values according to the output test data. Outlier classification element 212 is configured to identify and / or classify various outliers of the data according to a selected algorithm.

  More specifically, the scaling element of this embodiment suitably uses various statistical relationships to dynamically scale outlier sensitivity and to smooth coefficients for noise filtering sensitivity. The scaling factor is suitably calculated by the scaling element and used to modify the selected outlier sensitivity value and the smoothing factor. Any suitable criterion, such as an appropriate statistical relationship, may be used for scaling. Alternatively, scaling may be omitted from the process.

  Outlier classification element 212 is suitably configured to identify and / or classify outliers, output test data, and / or analysis results from any suitable algorithm at component 106. Additionally, the outlier classification element 212 is configured to utilize a number of candidate outlier identification algorithms to identify one or more algorithms suitable for identifying outliers from the output test data. obtain. Different outlier tests generate different population distributions, such that an outlier identification algorithm that adapts to one test may not adapt to other tests. Outlier classification element 212 is suitably configured to recognize differences between different data populations and automatically select one or more outlier identification algorithms based on the data population type of the current data. Automatic selection may select from any suitable set of candidate outlier identification algorithms and may perform selection according to any suitable criteria and analysis.

  For example, referring to FIG. 30, the outlier classification element 212 may be configured to automatically perform an outlier identification algorithm selection process. Outlier classification element 212 suitably comprises a preprocessing engine 3010 and a classification engine 3012. The preprocessing engine 3010 generates data appropriately to facilitate selection of the associated outlier identification algorithm. The classification engine 3012 appropriately selects one or more related outlier identification algorithms and identifies outliers accordingly.

  For example, output test data, such as data obtained from a particular test, may initially be used to analyze output test data that is compatible with a variety of candidate outlier identification algorithms. Provided to. The data can be analyzed in an appropriate manner to identify an appropriate algorithm to identify outliers in the output test data. For example, in this embodiment, the preprocessing engine 3010 receives the output test data and retrieves an available outlier identification algorithm, such as by retrieving an available outlier identification algorithm from an outlier identification algorithm library stored in memory. Prepare the algorithm. The preprocessing engine 3010 analyzes the output test data for outliers using some of the available algorithms. In this embodiment, the preprocessing engine 3010 is preprocessing data (eg, outliers identified by all algorithms, as well as various values such as minimum, maximum, average, median, standard deviation, CPK, CPM, etc. Output test data is analyzed using each of the algorithms specified by the user or other suitable algorithm selection.

  The algorithm may be based on industry standards (eg, IQR, median ± N × σ, etc.) and / or proprietary, custom, or user-defined outlier identification techniques. The outlier identification algorithm library is suitably configured so that the user can add, remove or edit outlier identification algorithms, for example, according to the particular product under test or the test characteristics to be performed. obtain. Different algorithms may be adapted to different statistical population types such as normal, lognormal, bimodal, clump or low CPK data population. Candidate outlier identification algorithms may comprise any suitable algorithm for various types and distributions of data. They are: interquartile range (IQR) normal distribution, 3σ; IQR normal distribution, 6σ; IQR log normal, 3σ; IQR log normal, 6σ; bimodal algorithm; clamp algorithm; low capacity algorithm; 3σ, 6σ or nσ Based custom algorithms; and algorithms such as proprietary algorithms with various sensitivities. The preprocessing engine 3010 may also analyze test data that generates features associated with the output test data. For example, the preprocessing engine 3010 can calculate various statistical characteristics of the output test data.

  The result of the preprocessing algorithm is dynamically selected for outlier detection. In this embodiment, outlier classification element 212 analyzes the test results to identify the most useful or applicable outlier identification algorithm. Data from the selected outlier identification algorithm can be retained until the remaining data is discarded. For example, in this embodiment, the classification engine 3012 receives the result of the preprocessing analysis generated by each of the available outlier identification algorithms. The classification engine 3012 analyzes the preprocessing data according to any suitable criteria such as predetermined rules and / or user-defined recipe driven rules. This is to determine whether the preprocessing data satisfies various criteria.

  The rule can be any suitable rule. For example, using statistical ratios or statistics, such as comparing statistics such as minimum, maximum, average, median, standard deviation, CPK and CPM with various thresholds or other criteria. For example, the classification engine 3012 may skip the outlier detection process in some situations, such as when there are too few test results, or when there is a very narrow or bimodal distribution between test results. The rules can be preselected and / or adjusted or added by the user to adapt to specific conditions of the product and test environment. Further, the classification engine 3012 may be configured to apply a particular algorithm to certain types of tests. For example, when a test result is known to have a certain result. Other rules may determine whether a particular test is applicable. For example, the classification engine 3012 may compare CPK to a threshold value. If the CPK is below the threshold, an IQR normal outlier identification algorithm may be used. In this system, the results from the algorithm that satisfies the rules are used for outlier identification. Other algorithm results for that test are appropriately ignored.

  Outlier classification element 212 may also identify and analyze selected outliers and component 106 according to test output test results and information generated by supplemental data analysis element 206. For example, the outlier classification element 212 is suitably configured to classify the component 106 into the critical / marginal / good category. This configuration includes, for example, user-defined criteria, user-defined good / bad spatial pattern recognition, data classification related to tester data compression, in-situ sensitivity qualification and analysis of test settings, tester yield leveling analysis, Together with dynamic wafer map and / or test strip mapping for part placement, and dynamic retest or test program optimization analysis. Outlier classification element 212 may classify data according to conventional SPC management rules such as Western Electric rules or Nelson rules to characterize the data.

  The outlier classification element 212 uses the selected set of classification limit calculation methods to properly classify the data. Any suitable classification method can be used to characterize the data according to the needs of the operator. The outlier classification element 212 classifies outliers, for example, by comparing the output test data with a selected threshold. The selected threshold is, for example, a value corresponding to one, two, three and six times the standard deviation statistically scaled from a threshold such as the data mean, mode and / or median. Such a threshold. Outlier identification by such a method tends to normalize any outlier identified in any test, regardless of data amplitude or relative noise.

  Outlier classification element 212 analyzes and correlates raw data points based on normalized outliers and / or user-defined rules. The rules can include any suitable technique for part and / or pattern classification. The supplemental data analysis element 206 may be configured to perform additional analysis of the output test data and information generated by the supplemental data analysis element 206. For example, the supplemental data analysis element 206 may identify tests with a high frequency of defects or outliers. For example, outliers, defects, or by comparing the total or average number of defects in a particular classification with one or more threshold values.

  The supplemental data analysis element 206 can also be configured to correlate data from different tests to identify trend analogies. The analogy identification may be performed, for example, by comparing cumulative counts, outliers, and / or outliers between wafers or other data sets. The supplemental data analysis element 206 can also analyze and correlate data from different tests to identify and / or classify potential critical and / or marginal and / or good portions on the wafer. The supplemental data analysis element 206 can also analyze and correlate data from different tests to identify user-defined good and / or bad patterns on a series of wafers for dynamic test time reduction purposes. .

  The supplemental data analysis element 206 is also suitable for analyzing and correlating data from different tests to identify user-defined related raw data for the purpose of dynamic compression of test data into memory. Configured. The supplemental data analysis element 206 may also analyze and correlate statistical anomalies and test data results for qualification and sensitivity analysis of test node in situ settings. Further, the supplemental data analysis element 206 can contribute to test node yield leveling analysis. This is done, for example, by identifying whether a particular test node has not been improperly calibrated or otherwise produced improper results. Further, the supplemental data analysis element 206 can be analyzed and correlated for the purpose of test program optimization. This optimization includes, but is not limited to, automatic identification of redundancy tests that use correlated results and outlier analysis to provide additional data for use in the analysis. Also, the supplemental data analysis element 206 can identify critical tests by identifying, for example, tests that regularly fail, tests that fail almost, tests that rarely fail, and / or tests that exhibit very low Cpk. Appropriately configured to identify tests.

  Supplemental data analysis may also provide for the identification of test sampling candidates in tests that rarely fail, or that do not fail at all, or that do not detect any outliers. The supplemental data analysis element 206 also relates to the analysis and correlation of identified outliers and / or other statistical anomalies, the number of defects, critical tests, longest / shortest tests, or test defects. In combination with basic functional issues, it may also provide the identification of optimal dimension test sequences based on association techniques such as conventional association techniques.

  Supplemental data analysis may also provide identification of critical, marginal, and good parts, as defined by the sensitivity parameters in the recipe configuration file. Part identification provides pre-packaging and / or pre-shipment processing / classification of parts that may indicate reliability risk and / or test time reduction by dynamic probe mapping of defective and good parts in wafer probes Can do. The identification of these parts can be done in any suitable way, for example, as a good part and a bad part, to a dynamically generated prober control map (for dynamic mapping), to a wafer map used in offline printing equipment. It can be displayed and output in a test strip map for strip testing in a test, in a results file, and / or in a database results table.

  When supplementary data analysis is performed at the cell controller level, the quality control in the probe tends to improve, and therefore the final test yield also tends to improve. Additionally, quality issues can also be identified during runtime rather than after product run-time. In addition, supplemental data analysis and signature analysis tend to improve out-of-order data quality by providing outlier identification to downstroke and offline analysis tools, as well as to test engineers or other staff. For example, the computer 108 indicates a fault in the production process on a wafer map that identifies the group of components undergoing signature analysis. In this way, the signature analysis system can identify potential defects that cannot be detected using conventional test analysis.

  Referring now to FIG. 10, an array of semiconductor devices is located on the wafer. In this wafer, the general resistivity of resistive components in semiconductor devices varies across the wafer. The reason is, for example, that the material and processing of the wafer are made uneven. The resistance of any particular component, however, can be within the control range of the test. For example, the targeted resistance value for a particular resistive component may be 1000Ω ± 10%. Near the edge of the wafer, the resistance values of most resistors approach the normal distribution range of 900Ω and 1100Ω but do not exceed that range (FIG. 11).

  Components on the wafer can include defects due to, for example, contaminants or imperfections in the manufacturing process. Due to the defect, the resistance value of the resistor located at the low resistivity end of the wafer can increase to 1080Ω. The resistance value is expected to be well above 1000 Ω for devices near the wafer center, but is still well within the normal distribution range.

  With reference to FIGS. 12A-B, the raw test data for each component may be plotted. The test data shows considerable variation. Part of that is due to the resistivity changing between components on the wafer when the prober indexes while passing horizontally or vertically through the device. Devices affected by defects are not easily identified based on visual inspection of test data or comparison with test limits.

  When test data is processed in accordance with various aspects of the present invention, the device affected by the defect can be tied as an outlier in the test data. Test data is largely limited to a range of values, however, the data associated with defects is not similar to the data of surrounding components. Thus, the data indicates a deviation from the value associated with the surrounding device without defects. The outlier classification element 212 can identify and classify outliers according to the magnitude of the deviation of the outlier data from the surrounding data.

  The output element 208 properly collects data from the test system 100 at run time and provides output reports to a printer, database, operator interface, or other desired destination. Any format such as graphical format, numbers, text, print format or electronic format can be used to display the output report for use or for later analysis. Output element 208 may provide any selected content including output test data selected from tester 102 and the results of supplemental data analysis.

  In this embodiment, the output element 208 suitably provides data selection from supplemental data to the product run time via output test data directed by the operator and a dynamic data log. Referring to FIG. 7, output element 208 first reads the sampling range from database 114 (step 702). The sampling range identifies predetermined information that should be included in the output report. In this embodiment, the sampling range identifies the component 106 on the wafer that has been selected by the operator for consideration. The predetermined component can be selected according to any criteria. For example, data for various perimeter zones, radial zones, random components, or individual stepper fields. The sampling range includes a set of xy coordinates corresponding to the position of a given component on the wafer, or an identified portion of available components in the batch.

  The output element 208 may also be configured to include information regarding outliers or other information generated or identified by the supplemental data analysis element of the dynamic data log (step 704). When configured in this way, identifiers such as xy coordinates for each of the outliers are also assembled as well. The coordinates for the component selected by the operator and the coordinates for the outliers are merged into the dynamic data log (step 706). This data log is in the format of the native tester data output format in this embodiment. By combining the resulting data into the dynamic data log, the amount required for data storage is reduced without compromising data integrity for later customer analysis, and compression of the original data into summary statistics, And it becomes easier to compress critical raw data into smaller native tester data files. The output element 208 retrieves selected information, such as raw test data and one or more data from the supplemental data analysis element 206, into each entry of the combined xy coordinate array of the dynamic data log (step 708).

  The retrieved information is then appropriately stored in the appropriate output report (step 710). The report may be prepared in any suitable format or method. In this embodiment, the output report includes a dynamic data log having a wafer map showing selected components on the wafer and their classification. Further, the output element 208 may superimpose wafer map data corresponding to outliers on the wafer map of preselected components. Additionally, the output element may only include outliers from the wafer map or batch as the sampled output. The output report may also include a series of graphical representations of the data to highlight outlier occurrences and correlations in the data. The output report may further include advice and advice support data. For example, if two tests appear to produce defects and / or outliers in the same set, the output report should indicate that the test is redundant and one of the two tests should be omitted from the test program. May include suggestions to advise. The advice may include a graphical representation of the data indicating that the two tests are the same result.

  The output report may be provided in any suitable manner (eg, output to a local workstation, sent to a server, alarm triggered, or any other compatible method) (step 712). In one embodiment, the output report may be provided off-line so that the output does not affect system operation or transfer to the main server. In this configuration, the computer 108 copies a data file, performs analysis, and generates a result, for example, for proof or verification purposes.

  Also, in accordance with various aspects of the present invention, in addition to supplemental analysis of each wafer's data, the test system 100 can perform a combined analysis of the data and generate additional data. This is to identify patterns and trends across multiple data sets, such as using multiple wafers and / or lots. Composite analysis is suitably performed to identify selected properties such as patterns and irregularities between multiple data sets. For example, multiple data sets can be analyzed because of common characteristics that can appear in patterns, trends or irregularities across two or more data sets.

  Composite analysis includes any analysis that is compatible with test data analysis for common characteristics between data sets, and can be implemented in any suitable manner. For example, in the present test system 100, the composite analysis element 214 performs a composite analysis of data obtained from multiple wafers and / or lots. Test data for each wafer, lot, or other grouping forms a data set. The composite analysis element 214 is suitably implemented as a software module that runs on the computer 108. However, the composite analysis element 214 may be implemented in any compatible configuration (eg, a hardware solution or a combined hardware and software solution). Further, the composite analysis element 214 may be executed on a remote computer such as the test system computer 108 or an independent workstation or a third party separate computer system. Following the generation of one or more complete data sets, or based on the collection of data (including historical data) generated well before the combined analysis, the combined analysis can be performed at runtime.

  A composite analysis may use any data from more than one data set. Composite analysis is a number of sets of input data (including raw data, filtered data and / or smoothed data) for each data set, such as performing a number of configurations via a classification engine. Can receive. Once received, the input data is appropriately filtered using a set of user-defined rules that can be defined by mathematical expressions, formulas, or any other criteria. The data is then analyzed to identify patterns or irregularities in the data. The composite data can be merged with other data, such as raw data or parsed data, to generate an expanded entire data set. The composite analysis element 214 can then provide an output report suitable for use to improve the testing process. For example, the output report may provide information regarding issues in the production and / or test process.

  In this system, the composite analysis element 214, along with user representation or any other suitable process, and spatial analysis, together with spatial analysis, constructs and establishes a composite map showing significant patterns and trends. Analyze the set. Composite analysis can receive several different data sets and / or composite maps for any one set of wafers by performing multiple user configurations on the set of wafers.

  Referring to FIG. 13, in this embodiment operating in a semiconductor testing environment, the composite analysis element 214 receives data from multiple data sets, such as data from multiple wafers or lots (1310). The data may include any suitable data for analysis (eg, raw data, filtered data, smoothed data, historical data from previous test runs, or data received from a tester at run time). . In this embodiment, the composite analysis element 214 receives raw data and filtered data at runtime. Filtered data may include any suitable data for analysis (eg, smoothed data and / or signature analysis data). In this embodiment, the composite analysis element 214 includes the raw data set and supplemental data generated by the supplemental data analysis element 206 (eg, filtered data, smoothed data, defect identification, outlier identification, signature analysis). Data and / or other data).

  After receiving the raw data and supplemental data, the composite analysis element 214 generates composite data for analysis (1312). Composite data comprises data representative of information from more than one data set. For example, composite data may be the number of defects and / or outliers in a particular test that occur in response to test data in different data sets (eg, data for components at the same or similar locations on different wafers or different lots). Information on However, the composite data can be any suitable data (eg, data relating to outliers or areas where defects are concentrated, data relating to wafer locations that produce a significant number of outliers or defects, or from two or more data sets. Other derived data).

  Composite data is suitably generated by comparing data from the various sets to identify patterns and irregularities from the various data sets. For example, the composite data can be generated by an analysis engine configured to provide and analyze the composite data according to any suitable algorithm or process. In this embodiment, the composite analysis element 214 includes an approximation engine configured to generate one or more composite masks based on the data set. The composite analysis element 214 may also process composite mask data, for example, to narrow down or enhance information in the composite mask data.

  In this embodiment, the approximation engine receives multiple datasets from a file, memory structure, database or other data store, performs spatial analysis on these datasets (1320), and returns the results in the form of a composite mask. Output. The approximation engine may generate composite mask data, such as a composite image for the entire data set, according to any suitable process or technique using any suitable method. In particular, the approximation engine appropriately merges the composite data with the original data (1322) and generates an output report for use by the user or other system (1324). The approximation engine also analyzes data for recurrent characteristics in the data set, or removes data that does not meet the selected criteria to narrow down the composite mask data for analysis such as spatial analysis, or It can also be configured for expansion.

  In this embodiment, the approximation engine performs composite mask generation (1312). It can also be configured to determine a removal area (1314), perform approximate weighting (1316), and detect and filter (1318) clusters. The approximation engine may also provide approximation adjustments or other actions using user-defined rules, criteria, thresholds and priorities. The result of the analysis is a composite mask of the input data set, which shows the spatial trends and / or patterns found throughout the given data set. The approximation engine can utilize any suitable output method or output medium, including memory structures, databases, other applications, and file-based data stores such as text or XML files that output composite masks.

  The approximation engine uses any suitable technique (including cumulative square method, N-point method, Western electric rules, or user-defined criteria or rules) to generate composite mask data. obtain. In this embodiment, composite mask data can be thought of as multiple datasets as a global view or a “stacked” view. The approximation engine collects and analyzes corresponding data from multiple data sets to identify potential relationships or common characteristics in the data for a particular set of corresponding data. The analyzed data can be any suitable data (raw data, filtered data, smoothed data, signature analysis data, and / or any other suitable data).

  In this embodiment, the approximation engine analyzes data corresponding to positions on a number of wafers. Referring to FIG. 14, each wafer has a device at a corresponding location that can be specified using a suitable identification system, such as an xy coordinate system. Thus, the approximation engine compares data for devices at corresponding positions or data points (points such as positions 10, 12 shown in FIG. 14) to identify patterns in the composite set of data.

  The approximation engine of this embodiment uses at least one of two different techniques (cumulative square method and formula-based method) for generating composite mask data. The approximation engine compares the data with a selection threshold or a calculation threshold to identify interesting data. In one embodiment, the approximation engine compares data points at corresponding locations on various wafers and / or lots with thresholds to determine whether the data exhibits a potential pattern across the data set. The approximation engine compares each data with one or more thresholds. Each of the thresholds may be selected in any suitable manner, such as a predetermined value, a calculated value based on current data, or a calculated value from historical data.

  For example, the first embodiment of the approximation engine implements a cumulative square method that compares data to a threshold. In particular, referring to FIG. 15, the approximation engine is a first data set (data set such as the result of a specific test for a specific device on a specific wafer in a specific lot) (1510). A data point is appropriately selected (1512) and the value of the data point is compared with a count threshold (1514). The threshold may comprise any suitable value and any type of threshold (eg, range, upper limit, lower limit, etc.) and may be selected according to any suitable criteria. If the value of the data point exceeds the threshold (ie, it may be lower than threshold, higher than threshold, within threshold limits, or any specific qualification relationship), the absolute value counter is incremented (1516). ), A summary value corresponding to the data point is generated.

  The value of the data point is also compared with a cumulative value threshold (1518). When the value of a data point exceeds the cumulative value threshold, the data point is added to the cumulative value for that data point (1520) to generate another aggregate value for that data point. The approximation engine repeats the process for each corresponding data point (1522) in each associated data set (such as each wafer in a lot or other selected group of wafers) (1524). Any other desired test or comparison may be performed on the data points as well as the data set.

  When all of the relevant data points in the population have been processed, the approximation engine may calculate a value based on the selected data (eg, data that exceeds a threshold). For example, the approximation engine may calculate an overall cumulative threshold for each set of corresponding data based on the cumulative value for the associated data point (1526). The overall cumulative threshold can be calculated in any suitable way of identifying the desired or related feature (eg, a method of identifying a corresponding set of data points that produces a relationship with the threshold). For example, the overall cumulative threshold (Limit) is the following equation:

に従って、定義され得る。ここで、Ａｖｅｒａｇｅは、データの複合母集団のデータの平均値で、Ｓｃａｌｅ Ｆａｃｔｏｒは、累積二乗法の感度を調整するために選択される値または変数で、Ｓｔａｎｄａｒｄ Ｄｅｖｉａｔｉｏｎは、データの複合母集団におけるデータポイント値の標準偏差で、（Ｍａｘ−Ｍｉｎ）は、データの完全な母集団におけるデータポイントの最大値と最小値との差である。一般に、全体の累積閾値は、特定のデータセットにおける所定のデータポイントを同定するための比較値を確立するために、定義される。

Can be defined according to Here, Average is the average value of data in the composite population of data, Scale Factor is a value or variable selected to adjust the sensitivity of the cumulative square method, and Standard Deviation is in the composite population of data. The standard deviation of data point values, (Max-Min) is the difference between the maximum and minimum values of data points in the complete population of data. In general, the overall cumulative threshold is defined to establish a comparison value for identifying a given data point in a particular data set.

  After calculating the overall cumulative threshold, the approximation engine determines whether to include each data point in the composite data, for example, comparing the count value and cumulative value against the threshold. The approximation engine of the present embodiment appropriately selects the first data point (1528), squares the total accumulated value for the data point (1530), and dynamically calculates the accumulated value squared for the data point. (1532). If the squared cumulative value exceeds the overall cumulative threshold, the data point is included in the composite data (1534).

  The absolute counter value for each data point is also the overall count threshold (e.g., a pre-selected threshold, or a threshold calculated based on, for example, the ratio of the number of wafers or other data sets in the population). ) (1536). If the value of the absolute counter exceeds all count thresholds, the data point can again be designated to be included in the composite data (1538). The process is performed appropriately at each data point (1540).

  The approximation engine may also generate composite mask data using other additional or alternative techniques. The approximation engine may also utilize a formula-based system to generate composite mask data. A formula reference system according to various aspects of the present invention uses variables and formulas or expressions to define a composite wafer mask.

  For example, in the exemplary formula criteria system, one or more variables may be user defined according to any suitable criteria. A variable may be appropriately defined for each data point in the relevant group. For example, the approximation engine may analyze each value at a particular data point. For example, to calculate a value at a data point, or to count the number of times a calculation provides a particular result. A variable can be calculated for each data point in the data set for each defined variable.

  After calculating the variables, the data points can be analyzed, for example, to determine if the data points meet user-defined criteria. For example, user-defined equations can be solved using calculated variable values. If the expression is equal to a certain value or range of values, the data point may be specified to be included in the composite mask data.

  Thus, the approximation engine may generate a composite mask data set according to any suitable process or technique. The resulting composite mask data includes a set of data corresponding to the results of the data population at each data point. As a result, characteristics for the data points can be identified across multiple data sets. For example, in this embodiment, the composite mask data may indicate specific device locations that share characteristics on multiple wafers (eg, highly variable test results or high defect rates). Such information can indicate issues or characteristics in the production or design process and thus can be used to improve and manage production and testing.

  The composite mask data can also be analyzed to generate additional information. For example, composite mask data can identify spatial trends and / or patterns in a data set and / or identify or filter significant patterns (eg, reduce confusion from relatively isolated data points) Filtering, expansion or narrowing of areas with specific characteristics, or filtering of data with known characteristics). The composite mask data of the present embodiment may be subjected to, for example, a spatial analysis in order to smooth the composite mask data and complete a pattern in the composite mask data. The selected exclusion zone may be subjected to specific processing such as composite mask data being removed, ignored, expanded, imparted with low significance, or otherwise distinguished from other composite mask data. The cluster detection process may also remove or downgrade the importance of data point clusters that are relatively insignificant or unreliable.

  In this embodiment, the approximation engine identifies specific designated zones in the composite mask data so that data points from the designated zones are subject to special designation processing in various analyzes or are ignored. Can be configured as follows. For example, in FIG. 16, the approximation engine may establish exclusion zones at selected locations on the wafer (eg, individual devices, device groups, device bands near the wafer perimeter). Exclusion zones may provide a mechanism that excludes certain data points from affecting other data points with approximate analysis and / or weighting. Data points are designated to be excluded in any suitable manner, such as assigning values that are out of range in subsequent processes.

  The relevant zone can be identified in any suitable manner. For example, the specification of excluded data points can be done by selecting a specific band width in the vicinity using the file list of device identifications or coordinates (eg, xy coordinates) or related zones in the composite data Can be done in other suitable processes to define. In this embodiment, the approximation engine causes the approximation engine to ignore the band of excluded devices on the wafer or otherwise specialize the data points that are in the range of the end of the user-defined data set. Can be defined using simple calculations. For example, all devices in this range or in the list of files are thus subject to the selected exclusion criteria. If the exclusion criteria are met, data points in the exclusion zone or devices that meet the exclusion criteria are excluded from one or more analyses.

  The approximation engine in this embodiment is suitably configured to perform additional analysis based on the composite mask data. The additional analysis can be configured for any suitable purpose, such as augmentation of the desired data, unwanted data removal, or identification of selected properties within the composite mask data. For example, the approximation engine is suitably configured to perform an approximate weighting process, such as based on a point weighting system, to smooth and complete the pattern in the composite mask data.

  Referring to FIGS. 17A-17B and FIG. 18, the approximation engine searches across all data points in the data set. The approximation engine selects the first point (1710) and checks the value of the data point against a criterion such as a threshold or a range of values (1712). If the data point exceeds the selected threshold or is found to be within the selected range, the approximation engine searches for values at data points around the main data point (1714). The number of data points around the main data point can be any selected number and can be selected according to any suitable criteria.

  The approximation engine searches the surrounding data points for data points that exceed the influential value or that satisfy any other criteria that indicate that the data point is a data point to be weighted (1716). If the data point exceeds the influential value, the approximation engine appropriately assigns a weight to the main data point according to the value of the surrounding data points. Further, the approximation engine may adjust the weight according to the relative position of surrounding data points. For example, the amount of weight imparted to surrounding data points can be determined according to whether the surrounding data points are adjacent (1718) or diagonal (1720) relative to the main data point. The total weight may also be adjusted 1722 if the data point is at the edge of the wafer. When data points around the main data point are checked (1724), the main data point is assigned an overall weight, for example by adding a weight factor from the surrounding data points. The weight for the main data point may then be compared (1726) to a threshold, such as a user-defined threshold. If the weight meets or exceeds the threshold, the data point is designated as such (1728).

  The composite mask data can also be further analyzed to filter the data. For example, in this embodiment, the approximation engine may be configured to identify a group of data points that are smaller than a threshold, such as a user-specific threshold, and appropriately remove it. 19 and 20, the approximation engine of this embodiment can be configured to define groups, measure their size, and remove smaller groups. To define the group, the approximation engine searches for data points that satisfy the criteria and searches through each data point in the composite mask data. For example, data points in the composite mask data can be partitioned into value ranges and assigned index values. The approximation engine begins by searching the composite mask data for data points that match a particular index (1910). Upon encountering a data point that fits the specified index (1912), the approximation engine designates the found point as the primary data point, and other data points that fall within the same index, or alternatively, substantially A recursive program is started (1914) that searches for data points that have the same value and also exceed the threshold, or that meet other desired criteria, in all directions from the main data point.

  As an example of the recursive function of the present embodiment, the approximation engine starts searching for a data point having a specific value (eg, 5). If a data point with a value of 5 is found, the recursive program searches all data points around the main device until it finds another data point with a value of 5. If other qualified data points are found, the recursive program selects the encountered data point as the main data point and repeats the process. Thus, the recursive process analyzes and marks all the data points that have matching values and are adjacent or diagonal to each other and therefore form a group. If the recursive program finds that all devices in the group have a particular value, the group is assigned a unique group index, and the approximation engine again searches through the composite mask data. When all of the data values are searched, the composite mask data is completely partitioned into groups of adjacent data points having the same group index.

  The approximation engine can determine the size of each group. For example, the approximation engine may count the number of data points in the group (1916). The approximation engine then compares the number of data points in each group to a threshold and removes groups that do not meet that threshold (1918). Groups may be excluded from the grouping analysis in any suitable manner, such as resetting the index values for the related groups to default values (1920). For example, if the threshold number of data points is 5, the approximation engine changes the group index value for each group with data points less than 5 to a default value. As a result, the only groups that remain sorted by different group indexes are the groups that have 5 or more data points.

  The approximation engine may perform any suitable additional operations to generate and refine the composite mask data. For example, composite mask data (including additional filtering, processing, and analysis results of the original composite mask data) may include information related to multiple data sets and the original source of the data (eg, device or manufacturing process on the wafer) ) Can be used. The data can be provided to the user or it can be used in any suitable manner. For example, the data can be combined with other data for further analysis (eg, to generate a data set that shows trends and patterns in the raw data, user-defined rules, composite mask data, raw The data and any other suitable data merging operation, etc.) may be provided to other systems. Further, the data can be provided to the user via a suitable output system such as a printer or visual interface.

  In this embodiment, for example, the composite mask data is combined with other data and provided to the user for review. The composite mask data can be combined with any other suitable data in any suitable manner. For example, the composite mask data can be merged with signature data, raw data, hardware bin data, software bin data, and / or other data. Data set coalescing may be performed in any suitable manner, such as with various user-defined rules (including representations, thresholds, preferences).

  In this system, the composite analysis element 214 uses an appropriate process to combine the composite mask data with the original map of composite data (eg, composite raw data, composite signature data, or composite bin data map). Use the process to execute the coalescing process. For example, referring to FIG. 21, composite analysis element 214 may combine the composite mask data with the original individual wafer data using an absolute coalescing system that fully combines the composite mask data with the original data map. As a result, the composite mask data merges with the original data map regardless of the overlapping or surrounding of existing patterns. When only one composite mask shows a pattern among many composite masks, the pattern is included in the entire composite mask.

  Alternatively, composite analysis element 214 may merge the data with additional analysis. The composite analysis element 214 may filter data that may be irrelevant or meaningless. For example, referring to FIG. 22, the composite analysis element 214 may only merge data in composite mask data that overlaps data in the original data map or other composite mask. This tends to highlight potentially relevant information.

  The composite analysis element 214 may alternatively evaluate the composite mask data and the original data to determine whether a particular threshold number, percentage, or other value of the data point overlaps between maps. Depending on the configuration, the merged data is a sufficiently overlapping area between the composite mask data that matches the threshold required for duplicate data points and the original data (in this case, data points corresponding to devices) May only be included. Referring to FIG. 23, the composite analysis element 214 may only include composite data patterns that overlap with testabin defects (ie, defective devices) in the original data at a sufficient rate (eg, 50% of the composite data overlaps with testabin defect data). Can be configured. Thus, if an amount less than the minimum amount of composite data overlaps with the original data, the composite data pattern can be ignored. Similarly, referring to FIG. 24, the composite analysis element 214 can compare two different sets of composite data, such as data from two different recipes, and the overlap between the two recipes is selected. It can be judged whether or not the standard is satisfied. Only duplicate data and / or data that meets the minimum criteria are merged.

  The combined data may be provided to output element 208 for output to a user or other system. The combined data can be passed as input to other processes such as a production error identification process or a large trend identification process. The merged data can also be output in any sort or format (eg, memory structure, database table, flat text file or XML file).

  In this embodiment, the merged data and / or wafer map is provided to an ink map generation engine. The ink map engine creates a map for an offline ink device. In addition to offline ink maps, the merged data results produce binning results for the inkless assembly of parts, or for any other process or application that utilizes this type of result. Can be used.

  Test system 100 may be configured to use test data that identifies characteristics and / or problems associated with manufacturing processes (including production processes and / or test processes). For example, the test system 100 analyzes test data from one or more sources and automatically characterizes the test data to identify known problems, issues, or characteristics (eg, on a pad) in the manufacturing and testing process. Residue, probing failure, conductive bridge, contamination, scratch, parametric variation, and / or stepper or reticle problems). If the characteristics of the test data do not correspond to a known issue, after diagnosing the issue, the test system 100 may receive and store information regarding the issue. This is so that the test system 100 is updated to diagnose new test data characteristics as they are encountered.

  In particular, the diagnostic system 216 is suitably configured to automatically identify test data characteristics and / or classify such characteristics according to their possible sources or causes. The test system 100 may also automatically provide warnings upon failure detection (eg, immediate defect classification and notification in runtime and / or later output reports). Information about the source or cause of various test data characteristics can be stored and compared with the data. Classification criteria and procedures can be configured such that when different test data characteristics are associated with different problems, that information is appropriately provided to the diagnostic system 216 for use in subsequent analysis. Stored information facilitates a configurable knowledge base that can be changed or updated depending on the particular data environment. The stored information also facilitates the recognition and classification of known scenarios for objective analysis and classification to report possible issues based on test frameworks, rules and symptoms. As new patterns are linked to specific problems, the diagnostic system 216 is updated so that the diagnostic system 216 facilitates understanding and retention of product engineering knowledge, generates a historical database, and is a consistent and repeatable analysis. Provide methodology.

  For example, the test data diagnostic system 216 can be configured to diagnose a derived problem at least in part by the test data. The data can be received and analyzed at runtime, retrieved from the storage system after completion of one or more test runs, and / or includes historical data. The diagnostic system 216 may receive test data from any suitable source (eg, parametric testing, metrology, process management, microscopy, spectroscopy, defect analysis, and fault isolation data). Diagnostic system 216 also provides additional data (eg, bin results, SPC data, spatial analysis data, outlier data, composite data, and data generated based on smoothed data, filtered composite data, test data. Processed data such as signatures may also be received.

  For example, referring to FIG. 25, the diagnostic system 216 of the present embodiment is configured to analyze multiple types of data. The diagnostic system 216 includes raw electronic wafer sort (EWS) data 2512 and EWS bin signature data 2514, bin map and / or yield pattern data 2518, outlier signature data 2520, and process management or electrical testing for each wafer. (ET) Data 2516 is analyzed. The EWS bin signature data 2514 may include any suitable classification data based on EWS results, such as may be generated by the supplemental data analysis element 206, for example. In this embodiment, the EWS bin signature data 2514 corresponds to each device on the wafer, and the size of the device defect (if the device fails) (critical, fairly small, as determined by the supplemental data analysis element 206). Data indicating the classification).

  The diagnostic system 216 also receives process management data 2516, such as data regarding electrical characteristics at various points on the wafer and / or component 106. In addition, the diagnostic system 216 may also receive bin map data 2518 for the wafer indicating the pass / fail bin classification of the component 106. Additionally, the diagnostic system 216 may also receive an outlier signature bin map 2520, such as, for example, data generated by the outlier classification element 212. For example, each outlier in the data can be categorized as minor, minor, medium, critical, etc., depending on the selected criteria.

  The diagnostic system 216 can be configured in any suitable manner to analyze the received data and identify process characteristics such as problems or challenges in the production or test process. Process characteristics can be identified by any suitable criteria or process. For example, referring to FIG. 26, the diagnostic system 216 of this embodiment includes a rule-based analyzer 2610 for identifying process characteristics according to predetermined criteria. Additionally or alternatively, the diagnostic system 216 may comprise a pattern recognition system 2612 for identifying process characteristics based on test data recognition patterns.

  In particular, the rule-based analyzer 2610 may analyze test data of specific characteristics for a specific problem based on a clear set of rules. The particular characteristic suitably includes any known data set corresponding to a particular test or production challenge. Rule-based analyzer 2610 is suitably configured to analyze data for a selected type of data and generate a corresponding signal. For example, if a test corresponding to a particular output node on a number of components does not produce any results, the diagnostic system 216 may indicate that (a) the output node is not functioning or (b) the test probe is A notification of normal contact with the output node may be generated.

  The pattern recognition system 2612 is suitably configured to receive data from various sources and identify patterns in the data. The pattern recognition system 2612 also matches the identified pattern with known issues associated with such a pattern, for example by specifying the likelihood of a specific cause based on the identified pattern. Is also configured appropriately. For example, a cluster of devices with similar reject bin results or outliers located at the same location on different wafers may indicate a particular problem in the production process. Pattern recognition system 2612 identifies and analyzes patterns in the data that may indicate such challenges in the production and / or test process.

  The pattern recognition system 2612 can be configured in any manner to identify patterns in various test data and analyze patterns that correspond to potential production or test challenges. In this embodiment, the pattern recognition system 2612 comprises an intelligent system configured to recognize patterns (eg, clustered defects or outlier spatial patterns) in test data. In particular, the pattern recognition system 2612 of this embodiment includes a spare processor 2614 and a classifier 2616. Spare processor 2614 processes the data handled by classifier 2616 and / or performs pattern identification that may correspond to issues within the received data. Classifier 2616 classifies the identified patterns or other data into various known or unknown categories.

  Spare processor 2614 may perform different operations on different types of data. For example, spare processor 2614 may not process any particular data at all, such as when no EWS bin signature analysis data or other data is needed by classifier 2616 to be pre-processed. Other test results can be used for statistical analysis. For example, spare processor 2614 may perform a major component analysis on the EWS test results that generate one or more eigenvectors or eigenvalues. Major component analysis may be based on the covariance of any suitable variables (eg, wafer location, test type, and wafer sequence). In this embodiment, spare processor 2614 may select the eigenvector associated with the larger eigenvalue, or select only the eigenvector associated with the largest eigenvalue, or ignore the less meaningful component. possible.

  Other data may be summarized or reduced. For example, process control data may be summarized as a sum of a series of test values corresponding to a wafer test structure. Thus, spare processor 2614 sums the values of each test for each test structure. The total number (and thus the magnitude of the vector) is therefore the same as the number of test structures.

  Other data may also be filtered and further processed by the reserve processor 2614. For example, spare processor 2614 may filter noise or other unwanted data from the data set and / or identify patterns from within test data (eg, bin map data and outlier signature bin maps). In addition, it can be configured in any suitable manner. For example, the reserve processor 2614 may filter data sets that do not show patterns, generate data associated with patterns in the data, and select specific patterns or other features for classification. Configured. In this embodiment, the reserve processor 2614 includes a pattern filter 2618, a feature extractor 2620, and a feature selector 2622. Pattern filter 2618 determines whether a data set, such as data for a particular wafer, contains a pattern. Feature extractor 2620 presents information for the data set specified by pattern filter 2618 and generates features suitable for analysis by classifier 2616. Feature selector 2622 analyzes the generated features according to selection criteria for selecting features for classification.

  Pattern filter 2618 may be configured to identify patterns in the data set in any suitable manner. For example, the pattern filter 2618 suitably comprises a software module configured to process the received data and detect if there are any patterns in the data. In order to maintain the integrity of the test data, the pattern filter properly processes the data without losing information from the original data. The pattern filter 2618 may leave only the data set with the detected pattern and discard the data set without the pattern. The pattern filter 2618 can be configured to analyze various types of data independently or in combination. The pattern filter 2618 may identify any suitable pattern, such as bulls-eye, hot spots, rings, and other patterns.

  In this embodiment, the pattern filter 2618 analyzes according to a pattern mining algorithm and with one or more masks associated with a known or theoretical pattern. For example, referring to FIG. 28, the pattern filter may be configured to perform median filtering of test data using a two-dimensional e-bin map with a pattern mask. The pattern mask may comprise any suitable mask that determines which devices in the e-bin map should be selected by performing median filtering. For example, the pattern may be defined from an existing scenario, defined from a real environment scenario, or defined by a simulation generated from a domain expert. The pattern mask is suitably similar to the pattern to be identified by the classifier 2616. For example, the pattern mask may utilize information generated by the composite analysis element 214 (eg, composite mask data from various data sets or combined composite mask data). However, any suitably simulated theoretical pattern can be used.

  Median filtering is performed near each value selected by the mask in the original e-bin map data. In particular, each data point of the data set and selected data points around each data point are compared to each selected mask. For example, around each value selected by the mask in the test data, the median value is calculated taking into account the neighborhood of size n × n (eg, a 3 × 3 window). Any of these datasets that do not show a pattern are ignored. Those datasets that contain patterns are provided to the feature extractor 2620. Alternative filtering techniques (eg, approximate analysis pattern separation techniques disclosed herein) may also be applied.

  In this embodiment, the pattern filter 2618 also reduces data set noise, for example, by removing intermittent noise from the test data. The pattern filter 2618 may use any suitable system for filtering noise (eg, spatial filtering, median filtering, or a convolution process on test data). In one exemplary embodiment, the pattern filter 2618 is used to reduce noise, such as “salt-and-pepper” noise due to outliers in the data. Spatial filtering such as median filtering is used.

  A data set having patterns can be analyzed to match the identified patterns to a particular task. However, under certain circumstances, the raw data used by the pattern filter 2618 may not be appropriate for analysis by the classifier 2616. As a result, feature extractor 2620 generates data that can be used by classifier 2616 based on the test data.

  In this embodiment, feature extractor 2620 generates features that indicate information from the data set specified by pattern filter 2618. Features can be particularly useful in situations where the original data is difficult or impossible to use. The features can then be analyzed by a classifier 2616 to identify the pattern types that are analyzed by the pattern filter 2618. For example, feature extractor 2620 may be configured to encode related information in the original data, such as by calculating values for a set of variables based on the data set. The feature is used by the classifier 2616 to efficiently encode the relevant information present in the original data and to classify the corresponding pattern in the data set into a defect classification. Configured appropriately. Features are also calculated appropriately from any dataset, thus becoming independent of the particular component under test for the nature of the test itself.

  A feature may include any suitable information extracted from the data. In this embodiment, the feature extractor 2620 calculates several features. These features are substantially independent of the characteristics of the particular device being tested or the data set, such as mass, center of gravity, geometric set of moments, and Hu's seven moments from the test data. Shows the standardized and / or compressed data. Mass provides information about the size of the distribution within the dataset of interest. The center of gravity provides a position such as an xy coordinate corresponding to the center of mass distributed within the die. A geometric moment (eg, a full set of 15 moments) produces a representation equivalent to the data set. The seven moments of Hu comprise moments that are invariant under translation, scaling and rotation actions.

  Various features can be determined for any data set. In this embodiment, the features are calculated based on bin data, outlier signature data, or other test data for the wafer. Thus, the mass generally corresponds to the size or other value of the die distribution within a given bin without giving any information about the location of the distribution. When the test value at coordinates x and y is f (x, y), the mass M is the following equation:

に従って、適切に計算される。ここで、Ｎはテストデータセットにおけるデータポイントの総数である。質量は、データポイントの数（例えば、ウェハー上のダイの個数）が異なるデータセット間で整合性があるように、規格化される。

According to Where N is the total number of data points in the test data set. The mass is normalized so that there is consistency between data sets with different numbers of data points (eg, the number of dies on the wafer).

The centroid can be defined by spatial coordinates such as x _c and y _c . The centroid measures the center of mass of the die distribution and provides position information. The center of gravity of the data is the following equation:

のような任意の適切な方法で計算され得る。

Can be calculated in any suitable manner, such as

  The geometric moment of order (p = 0 ... 3, q = 0 ... 3) is given by the following equation:

に従って計算され得る。

Can be calculated according to

  The information provided by this set of moments provides an equivalent representation to the data set in the sense that the bin map can be constructed from all moments of that order. Thus, each moment coefficient conveys a specific amount of information present in the bin map.

  In the present embodiment, Hu's seven moments are also considered (Hu, MK, “Visual Pattern recognition by moments innovations”, IRE Transactions on Information Theory, Vol. 8 (2), pp. 179-187. (See 1962). The seven moments of Hu are invariant under translation, scaling and rotation actions. The moment of Hu is the following equation:

を用いて、計算され得る。ここで、η_ｐｑは、全てのｐ、ｑに対する中心モーメントであり、以下：

Can be used to calculate. Where η _pq is the central moment for all p and q, and

によって、定義される。

Defined by

  The first six of these moments are unchanged under the action of symmetrical movement, but the last moment changes sign. These quantity values can be very large or different. To avoid a strict problem, the logarithm of the absolute value can be adopted and passed as a feature by the classifier 2616. Because these features are invariant, it is advantageous when bin maps or other data sets are analyzed with signature classification independent of scale, position, or angular position.

  A representative set of 25 features is appropriately extracted from each bin map or other data set specified by the pattern filter 2618. All or a portion of this feature may be provided directly to the classifier 2616 for classification. In this embodiment, fewer than all features may be provided to the classifier 2616, for example, to reduce the number of features to be analyzed and thus also reduce the dimension of the analysis process. Reducing the number of features tends to reduce computational complexity and redundancy. Further, the generalization characteristics required for classifier 2616 may require that the number of features be limited. For example, in the present classifier 2616, the generalized characteristic of the classifier 2616 may correspond to the ratio of the number of free classifier parameters to the training parameter N. A large number of features corresponds to a large number of classifier parameters such as synaptic weight. For training parameter N, which is a finite and usually limited number, fewer features tend to improve the generalization of classifier 2616.

  The system's feature selector 2622 analyzes the generated features according to the criteria selected for feature selection for classification. In particular, the feature selector 2622 is suitably adapted to select specific features to be analyzed and to minimize errors induced by providing less than all of the features to the classifier 2616. Composed. Feature selector 2622 may be configured in any suitable manner for selecting features for transfer to classifier 2616.

  In this embodiment, the feature selector 2622 implements a genetic algorithm to select features. The genetic algorithm suitably includes a parallel search process. This parallel search process tends to maintain a large number of solutions, remove suspicious solutions, and improve good solutions. Genetic algorithm analysis is properly applied over and over to various features and the output of the algorithm is the optimal solution found in the evolving process.

  Referring to FIG. 27, in this embodiment, to implement a genetic algorithm, feature selector 2622 first defines values for GA parameters (2710), starts a generation counter (2712), Starting with the initial population formation at random (2714). The population includes a group of coded solids, each individual representing a selected feature. The solid sequence in the initial population is randomly generated, for example, by an automated computer program. Any suitable parameter can be used. For example, the number of epochs, the number of individuals in the population, chromosome size, cost function, selection ratio, cross / replication ratio, mutation rate, etc. In this system, each population has 10 different solids, which represent whether there are specific features in the optimal solution. In other words, each individual is coded in binary on a chromosome representing a set of features (ie, a 25-bit (number of features) string). Here, “1” indicates a specific feature considered in the classification, and “0” means that the feature at that position is not used.

  The feature selector 2622 then evaluates (2716) the initial population, applies crossings and mutations to the population (2718, 2720), and increments the generation counter (2722). When the generation counter reaches a preselected limit (eg, maximum number of generations) (2724), feature selector 2622 terminates the analysis and provides the selected features to classifier 2616 (2726). . If the limit has not yet been reached, the feature selector 2622 repeatedly evaluates the child population (2728) and applies crossings and mutations to the population until the limit is reached.

  The classifier 2616 classifies the identified patterns into different known categories or unknown categories. The classifier 2616 may comprise any suitable classification system, such as a Bayes classifier or a maximum likelihood classifier, for patterning identified by the reserve processor 2614. The classifier is a supervised non-parametric classifier and / or a supervised or unsupervised rule-based classifier. The classifier 2616 of this embodiment is configured to classify patterns based on analysis of features selected by the feature selector 2622.

  Classifier 2616 receives data for analysis such as features obtained by feature extractor 2620 and selected by feature selector 2622. The data is processed to compare the data with data for a known pattern. If a known pattern matches the data, a task or characteristic corresponding to the known pattern is recorded. For example, referring to FIGS. 31A-B, classifier 2616 may associate a particular characteristic in the input data with a particular problem source, such as by looking up a lookup table. The classifier 2616 may also assign a possibility of a specific task or characteristic corresponding to the pattern. If none of the known patterns match the data, it is also recorded that it did not match. The resulting information can then be provided to output element 208.

  Referring to FIG. 32, in this embodiment, the classifier 2616 includes a two-stage classifier 3208. A first stage 3210 receives data from individual data sources and classifies the data, for example, according to a set of rules or patterns in the data, to address potential problems. The second stage 3212 receives data from the various first stage 3210 classifiers and classifies all the data to identify the most likely problem source or problem source characterized by the data. And combine the results.

  The classifier in the two stages 3210, 3212 may comprise any suitable system, such as an intelligent system. Such systems include, for example, neural networks, particle swarm optimization (PSO) systems, genetic algorithm (GA) systems, radial basic function (RBF) neural networks, multilayer perceptual (MLP) neural networks. , RBF-PSO system, MLP-PSO system, MLP-PSO-GA system, or other types of classifiers, or combinations thereof. A particular system may be selected depending on the performance of the classifier for a particular data set.

This embodiment includes a linear neural network such as an RBF neural network and / or a feedforward network configured to analyze the features selected by the feature selector 2622. Referring to FIG. 29, an RBF neural network 2910 according to various aspects of this embodiment is configured using an evolutionary algorithm technique such as PSO to function as a classifier. The RBF network suitably comprises three layers that play different roles. Input layer 2912 comprises a source node that connects the RBF network to feature selector 2622 to receive the selected features. A second layer 2914, suitably equipped with a hidden layer, applies a non-linear deformation from the input space to the hidden space. In this hidden layer, the nerve activation function (h _i (x)) is a radial basic function (RBF Φ). Gaussian functions are commonly used, but Cauchy functions, multiquadratic functions, and inverse-multiquadratic functions can be used. In this embodiment, each hidden nerve calculates the distance from its input to the nerve center point c and applies that distance to the RBF. The output layer 2916 (o _i (x)) nerve performs a weighted sum between the output of the hidden layer and the weighted link connecting both the output layer and the hidden layer. For example, the following formula:

に従う。ここで、ｘは入力、ΦはＲＢＦ、Ｃ_ｉはｉ番目の隠れ神経の中心、ｒ_ｉはその半径、ｗ_ｉｊは隠れ神経番号ｉと出力神経番号ｊを接続する重みリンク、そして、Ｗ_０は出力神経のバイアスである。

Follow. Where x is the input, Φ is the RBF, C _i is the center of the i th hidden nerve, r _i is its radius, w _ij is the weight link connecting the hidden nerve number i and the output nerve number j, and W ₀ Is the output nerve bias.

  In other words, the hidden layer 2914 nerve applies a high-dimensional nonlinear deformation from the input space to the hidden space, and the output layer 2916 performs a linear deformation from the hidden unit space to the output space. The justification for this arrangement is that pattern classification problems injected non-linearly into a high-dimensional space are more likely to be linearly separable than in a low-dimensional space.

  The parameters of the classifiers in the two stages 3210, 3212 (eg, the number of basis functions and their respective centers and widths for the RBF network) determine the impact of evolution according to evolutionary algorithms such as PSO. Easy to receive. In this embodiment, each particle of the algorithm indicates a center and radius value. Referring to FIG. 33, the width of the basic function is initialized randomly within the range [0, 1], the center of which is randomly selected between the inputs in the training set (3310).

  With respect to the evolution of the number of fundamental functions in the hidden layer, this approach is characterized as a constructive method. For example, at the start of evolution, the size of each network in the population seems to be minimal (eg, only one nerve in the hidden layer). As the evolutionary process progresses, the network grows, adding more nerves to the hidden layer. All the networks are initialized with a single primitive function of the hidden layer, and the networks are formed incrementally. For each optimization iteration, the overall performance of the swarm (ie, the mean square error of the best particle in that iteration) is calculated (3312). If the overall error increases relative to the performance of the swarm in the previous iteration (3314), one basic function is added to the hidden layer of each neural network (3316).

  Particles can be structured (local model) in the lbest neighborhood in a circle topology. Here, each particle is affected only by its immediate neighbors. In this type of topology, the neighborhoods are tightly connected while some of the population can be quite far from each other. In this way, one segment of the population can concentrate on local optimal conditions, while other segments concentrate on different optimal conditions or continue the search. In this topology, if the influence spreads from neighborhood to neighborhood and the optimal value is the best value found by any part of the population, this optimal value will eventually bring all the particles in Pull in.

  In the PSO algorithm, the inertia weight affects the locus of particles. The non-zero inertia coefficient gives the particles a priority that keeps moving in the same direction. The inertia coefficient can typically be adjusted to about 0.9 to 0.4 to remove instability, such as the inertia weight decreases over time and becomes unstable. Alternatively, the coefficients may have a two-dimensional representation that decays parabolically rather than decaying linearly. The effect of the time-decreasing factor tends to narrow the search, leading to a shift from exploratory mode to exploitative mode.

  The following table summarizes an exemplary set of coefficient values.

分類器２６１６は、また、その対応する特性に基づいて、示唆された修正アクションも提供し得る。特に、分類器２６１６は、データベース１１４のようなメモリにアクセスするように構成され得る。それは、様々な製造および／またはテストプロセスの特性に応答して、修正アクション候補のセットを同定するためである。例えば、同定されたパターンに合致する特性が、コンポーネントが製造プロセスの特定ポイントで過度な加熱に曝されたことを示したら、分類器２６１６は、その課題を是正する特徴に対応する潜在的な修正アクション（例えば、特定の製造ポイントで、ウェハーの曝される温度降下または時間短縮などのアクション）を求めて、データベース１１４をチェックし得る。

Classifier 2616 may also provide suggested corrective actions based on its corresponding characteristics. In particular, the classifier 2616 can be configured to access a memory such as the database 114. It is for identifying a set of corrective action candidates in response to various manufacturing and / or test process characteristics. For example, if a characteristic that matches the identified pattern indicates that the component has been exposed to excessive heating at a particular point in the manufacturing process, the classifier 2616 may identify potential corrections that correspond to features that correct the challenge. The database 114 may be checked for actions (eg, actions such as a temperature drop or a reduction in time the wafer is exposed to at a particular manufacturing point).

  The pattern recognition system 2612 may also be configured to learn additional information regarding the identified pattern and the corresponding task. For example, the pattern recognition system 2612 can be configured to receive diagnostic feedback information after pattern identification. The diagnostic feedback information appropriately corresponds to the actual problem identified in the production or test process that caused the identified pattern. The pattern recognition system may then use the diagnostic feedback information for future data analysis to identify recurrence of the problem.

  Various functions and elements of the test system 100 may be configured to process multi-site test data and conventional single-site data. Unlike conventional single-site data, multi-site data can be received simultaneously from different parts of the wafer and received using different hardware and software resources. As a result, data from different sites may differ due to factors other than device differences (eg, probe hardware differences, etc.). Accordingly, test system 100 minimizes potential problems associated with multi-site testing and / or identifies problems that may be associated with multi-site testing and analyzes test data for multi-site testing. Can be configured to.

  For example, in one embodiment, supplemental data analysis system 206, composite analysis system 214, and / or diagnostic system 216 can analyze test data for each individual site and test data from each probe can be a separate tester 102. Can be run independently as if they were generated by As a result, inconsistencies between various sites do not cause problems in data analysis.

  In other embodiments, the supplemental data analysis system 206, the composite analysis system 214, and / or the diagnostic system 216 can independently analyze data from different sites in one calculation, and two in another calculation. Some calculations may be performed using both the data from the sites combined and the data from the independent sites and the combined data. For example, the statistical calculations associated with the median tester value may be calculated independently for each site. However, to perform approximate analysis, test system 100 may use a combined data set using data from all sites. The data can be handled in any suitable manner so that variations caused by multi-site testing are recognized and / or raised.

  In operation, data is received from various sources. The data is first analyzed, for example by rule-based diagnosis, to ensure that the problem is known, like the corresponding data. The diagnostic system 216 generates an output that indicates a particular problem identified using rule-based analysis. The data is then provided to a pattern recognition system 2612 that analyzes the data to identify patterns in the data. The pattern recognition system 2612 can then analyze the identified pattern corresponding to a particular production or test task. The pattern recognition system 2612 appropriately assigns the likelihood of being associated with a particular problem based on the pattern. The diagnostic system 216 may also recommend corrective action based on the patterns identified in the data. The diagnostic system 216 may then generate an output report that informs various identified issues and suggested corrective actions. After the reported issue is raised, the pattern recognition system 2612 may receive diagnostic feedback information. The diagnostic feedback information is stored in the pattern recognition system 2612 for use in future analysis.

  The specific implementations shown and described are merely illustrative of the invention and its best modes and are not intended to limit the scope of the invention in any way. For simplicity, conventional signal processing, data transfer, and other functional aspects of the system (and the individual components that operate the components of the system) may not be shown in detail. Further, the connecting lines shown in the various drawings are intended to represent exemplary functional relationships and / or physical connections between the various elements. There can be many alternative or additional functional relationships or physical connections in the actual system. The present invention has been described above with reference to preferred embodiments. However, changes and modifications can be made without departing from the scope of the invention. These and other changes or modifications are intended to be included within the scope of the present invention as expressed in the following claims.

FIG. 1 is a block diagram of a test system and related functional components in accordance with various aspects of the present invention. FIG. 2 is a block diagram of elements that operate the test system. FIG. 3 shows a flow diagram for the component elements. FIG. 4A shows a flow diagram for the supplemental data analysis element. FIG. 4B shows a flow diagram for the supplemental data analysis element. FIG. 4C shows a flow diagram for the supplemental data analysis element. FIG. 5 is an illustration of various sections of the wafer and sectioning techniques. FIG. 6A further shows a flow diagram for the supplemental data analysis element. FIG. 6B further shows a flow diagram for the supplemental data analysis element. FIG. 7 shows a flow diagram for the output element. FIG. 8 is a flow diagram for operation of an exemplary data smoothing system in accordance with various aspects of the invention. FIG. 9 is a plot of test data for multiple component tests. FIG. 10 is a diagram of a wafer with multiple devices and a resistivity profile for that wafer. FIG. 11 is a graph of resistance versus resistance population for various devices on the wafer of FIG. FIG. 12A is a general plot and detailed plot of raw test data and outlier detection trigger, respectively, for the various devices of FIG. FIG. 12B is a general and detailed plot of raw test data and outlier detection trigger, respectively, for the various devices of FIG. FIG. 13 is a flow diagram of a complex analysis process in accordance with various aspects of the present invention. FIG. 14 is a diagram of representative data point locations on three representative wafers. FIG. 15A is a flow chart and chart for the cumulative square composite data analysis process. FIG. 15B is a flowchart and chart for the cumulative square composite data analysis process. FIG. 15C is a flow chart and chart for the cumulative square composite data analysis process. FIG. 16 is a diagram of exclusion zones defined on the wafer. FIG. 17A is a flowchart of the approximate weighting process. FIG. 17B is a flowchart of the approximate weighting process. FIG. 18 is a diagram of a set of data points subjected to approximate weighting. FIG. 19 is a flow diagram of the cluster detection and filtering process. FIG. 20 is an illustration of a set of clusters that have been detected and filtered. FIG. 21 is a diagram of a set of data points that have been merged using an absolute merge process. FIG. 22 is a diagram of a set of data points merged using a heavy complex process. FIG. 23 is an illustration of a set of data points merged using a percentage multiplex complex process. FIG. 24 is a diagram of a set of data points merged using a percentage multiplex complex process. FIG. 25 is a block diagram of a system for identifying process characteristics using test data. FIG. 26 is a block diagram of the diagnostic system. FIG. 27 is a flowchart of the classification process. FIG. 28 is a diagram of a pattern filtering process. FIG. 29 is a diagram of a neural network. FIG. 30 is a diagram of a system that automatically selects one or more outlier identification algorithms. FIG. 31A is a chart showing the relationship between different types of input data characteristics and possible causes of those characteristics. FIG. 31B is a chart showing the relationship between different types of input data characteristics and possible causes of those characteristics. FIG. 32 is a diagram of the classification process. FIG. 33 is a flow diagram of the particle swarm optimization process.

Claims (19)

A tester configured to test a set of components and generate test data for the set of components;
Collect test data comprising: a diagnostic system configured to receive the test data from the tester and automatically analyze the test data to identify problems associated with the interior of the component manufacturing process And a test system for testing a semiconductor chip for analysis ,
The diagnostic system is configured to recognize a pattern in the test data and match the recognized pattern with the problem, and the diagnostic system is configured to classify the pattern using an evolutionary algorithm Test system, including a classifier.   The test system of claim 1, wherein the evolutionary algorithm comprises a particle swarm optimization algorithm. The diagnostic system comprises at least two stages;
The first stage includes a plurality of classifiers configured to receive different types of test data and generate first stage data based on the different types of test data;
A second stage is configured to receive the first stage data from the plurality of classifiers and classify the pattern using an evolutionary algorithm based on the first stage data; The test system according to claim 1.   4. The test system of claim 3, wherein the first stage and the second stage comprise a self-adaptive system, and the first stage and the second stage are independently trainable.   The test system of claim 1, wherein the evolutionary algorithm comprises at least one of a particle swarm optimization algorithm and a genetic algorithm.   The test system of claim 1, wherein the diagnostic system comprises at least one of a particle swarm optimization system, a genetic algorithm system, a radial basis function neural network, and a multilayer sensory neural network.   The test system of claim 1, wherein the diagnostic system is configured to automatically select an outlier identification algorithm for analyzing the test data.   8. The diagnostic system is configured to select the outlier identification algorithm based on at least one of a type of data population of the test data and a type of test generated by the tester. Test system as described in.   The test system of claim 7, wherein the diagnostic system is configured to analyze the test data using a plurality of outlier identification algorithms.   The test system of claim 7, wherein the diagnostic system is configured to select the outlier identification algorithm from a configurable algorithm library.   The test system of claim 1, further comprising a configurable knowledge base, wherein the diagnostic system is configured to retrieve stored information from the knowledge base to analyze the test data.   The test system of claim 11, wherein the diagnostic system comprises a self-adaptive system configured to learn from a set of historical test data.   The test system of claim 12, wherein the self-adaptive system is configured to learn from the test data generated by the tester.   The test system of claim 1, wherein the diagnostic system is configured to generate supplemental data based on the test data, the supplemental data being independent of the type of component and the type of test data. .   The test system of claim 14, wherein the diagnostic system is configured to filter the test data.   The test system of claim 14, wherein the diagnostic system is configured to identify a trend in the test data.   The tester is configured to generate the test data using a multi-site test, and the diagnostic system is configured to normalize the test data to offset the effects of the multi-site test The test system of claim 1, wherein:   The test system of claim 1, wherein the test data includes at least one of historical data and real-time data.   The test system of claim 1, wherein the diagnostic system is configured to automatically provide a corrective action based on the test data analysis.

Images