TW201135474A - Method for sampling workpiece for inspection and computer program product performing the same - Google Patents

Method for sampling workpiece for inspection and computer program product performing the same Download PDF

Info

Publication number
TW201135474A
TW201135474A TW99110416A TW99110416A TW201135474A TW 201135474 A TW201135474 A TW 201135474A TW 99110416 A TW99110416 A TW 99110416A TW 99110416 A TW99110416 A TW 99110416A TW 201135474 A TW201135474 A TW 201135474A
Authority
TW
Taiwan
Prior art keywords
workpiece
historical
value
workpieces
gsi
Prior art date
Application number
TW99110416A
Other languages
Chinese (zh)
Other versions
TWI427487B (en
Inventor
Chi-An Kao
Ying-Lin Chen
Fan-Tien Cheng
Original Assignee
Foresight Technology Company Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Foresight Technology Company Ltd filed Critical Foresight Technology Company Ltd
Priority to TW99110416A priority Critical patent/TWI427487B/en
Publication of TW201135474A publication Critical patent/TW201135474A/en
Application granted granted Critical
Publication of TWI427487B publication Critical patent/TWI427487B/en

Links

Landscapes

  • General Factory Administration (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

A method for sampling a workpiece for inspection and a computer program product performing the method are disclosed. The method calculates a reliance index (RI) and a RI threshold (RIT) associated with a virtual measurement value of a workpiece; and a global similarity index (GSI) and a GSI threshold (GSIT) associated with the process data of the workpiece by analyzing process data applied on a unit of production equipment. When the RI of the workpiece is smaller than the RIT or the GSI thereof is greater than the GSIT, the workpiece can be chosen for measurement.

Description

201135474 六、發明說明: 【發明所屬之技術領域】 本發明是有關於-種工件抽樣檢驗的方法,特別是有 關於-種可有效地抽檢出不良卫件的方法及其電腦程式產 品。 【先前技術】 目前大部分半導體及TFT-LCD廠對於生產機台之產 • 品或工件的品質監測方法係採取抽測的方式,其中此工件 可為半導體業之晶圓或TFT-LCD業之玻璃基板^當生產機 台完成若干個工件(Workpiece)的加工處理後,此些工件會 被置放於 ^匣或晶圓傳送盒(Front Opening Unified Pod ; FOUP中,以傳送至量測機台來檢測工件的品質。一 般而言’量測機台會從整個卡匣之複數個工件(例如:25 片)中固定地抽選一個工件為樣本來進行量測,例如:卡匣 中之第一個工件。此種習知之抽樣檢驗的方法係假設生產 • 機台的製程品質不會突然發生異常,因而可使用被抽測之 產品或工件的量測結果來推斷同一卡匣或晶圓傳送盒内之 所有產品的品質。然而,習知之抽樣檢驗的方法只能得知 此實際被抽測之工件的品質,而此實際被抽測之工件並不 一定是具有潛在風險的工件,故常會產生漏偵測(Miss Detection ; MD)的情形。此外,若生產機台在兩次抽測之 間發生異常’習知方法便無法及時發現,因而導致許多不 . 良品的產生,並造成可觀的成本損失。 理論上,若能對同一卡匣或晶圓傳送盒内之所有工件 201135474 均進行量測,則可避免前述之漏偵測的情形,更可及時發 現生產機台發生異常。然而,對同一卡匣或晶圓傳送盒内 之每一個工件均進行實際量測相當曠曰費時,需耗費許多 人力物力。況且,對具有數百道製程之晶圓或TFT_LCD廠 而言,欲對每一道製程之每—個工件進行實際量測更幾乎 是件不可能的任務。 因此,為避免上述問題發生,必須要提供一種工件抽 樣檢驗的方法及其電腦程式產品,藉以有效地抽選出合適的 工件來進行量測,俾便在生產機台發生異常時能及時發現。 【發明内容】 因此,本發明之一態樣就是在提供一種工件抽樣檢驗 的方法及其電腦程式產品’藉由判斷工件之信心指標 (Reliance Index ; RI)值是否小於信心指標門檻值;或工^ 之GSI(Global Similarity Index ;整體相似度指標)值是否大 於GSI門檻值(GSIT) ’來有效地抽選出合適的工件以進行 量測’而it免發生漏债測的情形,並能及時發現生產機台 異常。 口 根據本發明之上述目的,提出一 法。在本發明之-實施例中,首先:抽樣檢驗的方 歷史製程參數資料,並從量測機“生f機台之複數組 值,其中此些歷史量測值分別為根據歷史量測 之工件的量測值。接著,使用歷史製程=參數所生產 測值來建立-推估模式與—參考广資料和歷史量 立係根據-推估演算法,參考模式;建:中推估模式的建 哽立係根據一參考演 201135474 算法’推估演算法與參考演篡 程參數至推估模絲R。然後’輸入歷史製 量測值式 > 考模式,而計算出複數個歷史虛擬 複數個歷史參考_值1著,分料算歷史虛201135474 VI. Description of the Invention: [Technical Field] The present invention relates to a method for sampling inspection of a workpiece, and more particularly to a method for effectively sampling a defective guard and a computer program product thereof. [Prior Art] At present, most semiconductor and TFT-LCD factories adopt the sampling method for the quality monitoring methods of the products or workpieces of the production machine. The workpiece can be the wafer of the semiconductor industry or the glass of the TFT-LCD industry. Substrate ^When the production machine completes the processing of several workpieces, the workpieces are placed in the Front Opening Unified Pod; FOUP for transmission to the measuring machine. Detecting the quality of the workpiece. Generally speaking, the measuring machine will randomly select one workpiece from the entire number of workpieces (for example: 25 pieces) for measurement, for example: the first one in the cassette The conventional sampling method is based on the assumption that the process quality of the production machine will not suddenly be abnormal, so the measurement results of the product or workpiece being sampled can be used to infer the same cassette or wafer transfer box. The quality of all products. However, the method of sampling inspection can only know the quality of the workpiece that is actually being tested, and the workpiece that is actually sampled is not necessarily a potential risk. The workpiece often causes Miss Detection (MD). In addition, if the production machine is abnormal between the two tests, the conventional method cannot be found in time, resulting in many defects. And cause considerable cost loss. In theory, if all the workpieces 201135474 in the same cassette or wafer transfer box can be measured, the above-mentioned leak detection can be avoided, and the production machine can be found in time. Abnormal. However, actual measurement of each workpiece in the same cassette or wafer transfer cassette is quite time consuming and requires a lot of manpower and material resources. Moreover, for wafers or TFT_LCD plants with hundreds of processes It is almost impossible to perform actual measurement for each workpiece of each process. Therefore, in order to avoid the above problems, it is necessary to provide a method of sample sampling inspection and computer program products, thereby effectively The appropriate workpiece is selected for measurement, and it can be found in time when an abnormality occurs in the production machine. [Invention] Therefore, one of the present inventions This is to provide a method of sample sampling inspection and its computer program product 'by determining whether the confidence index of the workpiece (Reliance Index; RI) is less than the confidence threshold value; or GSI (Global Similarity Index; overall similarity) Whether the value of the indicator is greater than the GSI threshold value (GSIT) 'to effectively select the appropriate workpiece for measurement' and it is free from the occurrence of the leak test, and can find the abnormality of the production machine in time. Objective, a method is proposed. In the embodiment of the present invention, first: sampling the historical process parameter data of the test, and from the measuring machine "the complex array value of the machine, wherein the historical measurements are respectively The measured value of the workpiece based on historical measurements. Then, using the historical process=parameters to produce the measured values to establish-estimate the model and the reference-wide data and historical quantity basis--estimation algorithm, reference mode; The reference 201135474 algorithm 'infers the algorithm and the reference parameters to the estimated mode R. Then enter 'the historical measurement value> > test mode, and calculate a plurality of historical virtual plural historical reference _ value 1, the distribution is historically virtual

門的2的分配⑽㈣此011)與歷史參考預測值的分配之 面積而產生複數個敎信心指標值,其中當重疊 、積愈大’則心指標值愈高,代表所對應至歷史虛擬量 測值的可信度愈高。然後’根據歷史虛擬㈣值、歷史參 考預測值和歷史量測值來計算出—信指標門檀值⑻T)。 二後收集生產機台所送出之卡匣内之複數個工件的製程 參數資料,並輸入每一個工件的製程參數資料至推估模式 和參考模式,而計算出每一個工件之虛擬量測值和參考預 測值。接著,計算每一個工件之虛擬量測值的分配與參考 預測值的分配之間的重疊面積而產生每一個工件之信心指 標值,其中當重疊面積愈大,則信心指標值愈高,代表所 對應至其虛擬量測值的可信度愈高。然後,自此些工件中 選取其信心指標值小於信心指標門檻值之至少一個第一工 件’並將第一工件送至量測機台以進行檢測。 依據本發明之又一實施例’在工件抽樣檢驗的方法 中’首先獲取生產機台之複數組歷史製程參數資料。接著, 使用此些組歷史製程參數,並根據一統計距離演算法,來 建立一統計距離模式。然後,以此些組歷史製程資料及此 些歷史量測值,並應用交互驗證(Cross Validation)中的留一 法(Leave-One-Out ; LOO)原理來重建統計距離模式,並計 算出相對應的GSI值,以計算出一 GSI門檻值(GSIT)。接 著,輸入每一個工件的製程參數資料至統計距離模式,而 201135474 ·«+算出母一個工件之虛擬量測值所對應之製程參數資料的 GSI值。然後,自此些工件中選取其GSI值大於Gsi門檻 值之至少一個第二工件,並將第二工件傳送至一量測機台 以進行檢測。 根據本發明之上述目的,另提出一種内儲用於工件抽 樣檢驗之電腦程式產品,當電腦載入此電腦程式產品並執 行後,可完成如上述之工件抽樣檢驗的方法。 因此,應用本發明,可藉由某工件之製程參數資料來 φ 評估其品質是否可能有異常,以有效地抽選出合適的工件 來進行量測,而避免發生漏偵測的情形,並能及時發現生 產機台異常。 【實施方式】 清參照第1圖,其係纟會示實施本發明之工件抽樣檢驗 的方法的系統架構示意圖。本發明提供全自動化型虚擬量 測(Automatic Virtual Metrology ; AVM)系统 90 於生產機台 φ 20與量測機台30之間,藉以使用卡匣80内之所有工件82 的製程參數資料22來辅助量測機台30抽選合適的工件來 進行量測。在一實施例中,AVM系统90先通知製造執行 系統(Manufacturing Execution System ; MES)(未繪示)被抽 選出之工件的代號,製造執行系統再根據此被抽選出之工 件的代號下指令給量測機台30,以對此被抽選出之工件進 行量測。此外,在一實施例中,AVM系统90係嵌入在量 測機台30中。在又一實施例中,AVM系统90係嵌入在生 產機台20中。當然,AVM系统90亦可獨立地執行工件抽 201135474 樣檢驗方法,故本發明並不在此限。 請參照第2圖,其係繪示根據本發明之實施例之AVM 系统的架構示意圖。本實施例之AVM系统90至少包括: 製程參數資料前處理模組10、量測資料前處理模組12、推 估模式60、信心指標模組40和相似度指標模組50。製程 參數資料前處理模組10係針對來自生產機台20之原始製 程參數資料進行整理及標準化,刪除異常資料並筛選出重 要參數,將不重要參數排除,以避免產生干擾作用,而影 響預測精度。量測資料前處理模組12係針對來自量測機台 30之量測資料進行篩選,以去除其中之異常值。推估模式 60可利用推估演算法來推估卡匣80中之複數個工件82的 第一階段虛擬量測值(VMi),亦可選擇性地利用雙階段運算 機制62及推估演算法來推估卡匣80中之複數個工件82的 第二階段虛擬量測值(VM„)。可能選用的推估演算法有: 迴歸演算法、類神經網路演算法等各式預測演算法。信心 指標模組40係用來評估虛擬量測值的可信度,而產生信心 指標(RI)。相似度指標模組50係用來評估目前輸入之製程 參數資料與推估模式60内用來訓練建模之所有參數資料 的相似程度,而產生製程參數的相似度指標(GSI),此相似 度指標係用以輔助信心指標來判斷虛擬量測系統的信心 度。 在推估模式60運作之前,須將從生產機台20所獲得 的製程參數資料(歷史製程參數資料)與從量測機台30所取 得的品質量測資料(歷史量測值)分別傳送至製程參數資料 前處理模組10和量測資料前處理模組12,以進行資料前 201135474 處理。這些經前處理及標準化後 ί *;Γ(νΜ,)" 好^ mT、i 指標值(RI)和整體相似 又“值()。所謂「第二階段」虛擬 信心指標和相似度指標縣在從_機台取得工;;^ == 將二件82的製程參數資料和實際量測值加 入歷史製&參數讀及歷史量龍,來重新訓 估模式60、信心指標模組4〇之參考模式和相似度指標= 組50統計距離模式,再重新計算出卡匣8〇内之每一 ==虛擬量測值(VMlI)與其伴隨的信心指標和整 以下’說明推估模式、信心指標值(參考模式)和整體 相似度指標值(統計距離模式)相關的理論基礎。 推估模式與信心指標(參者槿式、 如表1所示,假設目前E集到n組量測的資料,包含 製程資料(^=/,2,...,„)及其對應的實際量測值資料(乃^^ 2,…,π) ’其中每組製程資料包含有p個參數(自參數1至 參數p),即\ = [\而,···,〜卜此外,亦蒐集到(㈣)筆實際生 產時製程資料,但除少„+1外,並無實際量測值資料,即在: 筆實際生產的工件中,僅抽測例如第一筆工件進行實際量 測,再以其實際量測八+1來推斷其他(w_w")筆工件的。質。 201135474 表1原始資料範例 樣本資 料點 參數 1 參數 2 參數 P 實際量測 值 1 ^1,2 • · · xhp 少1 2 χ2,\ x2,2 • * « Χ2,Ρ yi … … … … … … η 〜,1 心,2 • · · XfhP yn η+1 ^/7+1,1 ^m+1,2 • · · ^n+\yp yn+\ η+2 χη+2,\ χη+2,2 # # · ^n+2,p Zip … … … • · · … … m ^m,\ •V2 … Xm,p ZipThe assignment of the gate 2 (10) (4) This 011) and the area of the historical reference prediction value are generated to generate a plurality of confidence index values, wherein when the overlap, the larger the product, the higher the heart index value, the representative corresponds to the historical virtual measurement. The higher the credibility of the value. Then, based on the historical virtual (four) value, the historical reference predicted value, and the historical measured value, the -indicator threshold value (8)T is calculated. Secondly, collect the process parameter data of the plurality of workpieces in the cassette sent by the production machine, and input the process parameter data of each workpiece to the estimation mode and the reference mode, and calculate the virtual measurement value and reference of each workpiece. Predictive value. Then, calculating the overlap area between the allocation of the virtual measurement value of each workpiece and the allocation of the reference prediction value to generate a confidence index value of each workpiece, wherein the larger the overlap area, the higher the confidence index value, the representative office The higher the credibility corresponding to its virtual measurement. Then, at least one first workpiece whose confidence index value is less than the confidence threshold value is selected from the workpieces and the first workpiece is sent to the measuring machine for detection. According to still another embodiment of the present invention, in the method of sample sampling inspection, the data of the complex array history of the production machine is first obtained. Then, using these group history process parameters, and based on a statistical distance algorithm, a statistical distance mode is established. Then, using these historical process data and these historical measurements, and applying the Leave-One-Out (LOO) principle in Cross Validation to reconstruct the statistical distance pattern and calculate the phase The corresponding GSI value is used to calculate a GSI threshold (GSIT). Then, the process parameter data of each workpiece is input to the statistical distance mode, and 201135474 · «+ calculates the GSI value of the process parameter data corresponding to the virtual measurement value of the parent workpiece. Then, at least one second workpiece whose GSI value is greater than the Gsi threshold value is selected from the workpieces, and the second workpiece is transferred to a measuring machine for detection. According to the above object of the present invention, a computer program product for storing sample sampling inspection is provided, and when the computer is loaded into the computer program product and executed, the method of sampling inspection of the workpiece as described above can be completed. Therefore, by applying the invention, it is possible to evaluate whether the quality of the workpiece may be abnormal by using the process parameter data of a workpiece, so as to effectively select a suitable workpiece for measurement, and avoid the occurrence of leak detection, and timely Found that the production machine is abnormal. [Embodiment] Referring to Fig. 1, there is shown a system architecture diagram showing a method for performing sample inspection of a workpiece of the present invention. The present invention provides a fully automated Virtual Virtual Metrology (AVM) system 90 between the production machine φ 20 and the measuring machine 30 for assisting the process parameter data 22 of all workpieces 82 in the cassette 80. The measuring machine 30 draws the appropriate workpiece for measurement. In an embodiment, the AVM system 90 first notifies the Manufacturing Execution System (MES) (not shown) the code of the selected workpiece, and the manufacturing execution system then gives instructions to the selected workpiece according to the code. The measuring machine 30 measures the workpiece to be selected. Moreover, in an embodiment, the AVM system 90 is embedded in the measurement machine 30. In yet another embodiment, the AVM system 90 is embedded in the production machine 20. Of course, the AVM system 90 can also independently perform the workpiece sampling method of the 201135474, so the present invention is not limited thereto. Please refer to FIG. 2, which is a schematic diagram showing the architecture of an AVM system according to an embodiment of the present invention. The AVM system 90 of the present embodiment includes at least: a process parameter data pre-processing module 10, a measurement data pre-processing module 12, an estimation mode 60, a confidence indicator module 40, and a similarity index module 50. The process parameter data pre-processing module 10 collates and standardizes the original process parameter data from the production machine 20, deletes the abnormal data and filters out the important parameters, and excludes the unimportant parameters to avoid the interference effect and affect the prediction accuracy. . The measurement data pre-processing module 12 filters the measurement data from the measurement machine 30 to remove the abnormal value therein. The estimation mode 60 can use the estimation algorithm to estimate the first stage virtual measurement value (VMi) of the plurality of workpieces 82 in the cassette 80, and can also selectively utilize the two-stage operation mechanism 62 and the estimation algorithm. The second stage virtual measurement value (VM„) of the plurality of workpieces 82 in the cassette 80 is estimated. The possible estimation algorithms are: regression algorithm, neural network algorithm and other prediction algorithms. The confidence indicator module 40 is used to evaluate the credibility of the virtual measurement value, and generates a confidence indicator (RI). The similarity indicator module 50 is used to evaluate the currently input process parameter data and the estimation mode 60. The similarity degree of all parameter data of the training model is trained, and the similarity index (GSI) of the process parameter is generated, which is used to assist the confidence index to judge the confidence of the virtual measurement system. Before the estimation mode 60 operates The process parameter data (historical process parameter data) obtained from the production machine 20 and the product quality measurement data (historical measurement value) obtained from the measuring machine 30 are respectively transmitted to the process parameter data pre-processing module. 10 and measurement Before the material processing module 12, for 201,135,474 treatment before information on these ί after pre-treatment and standardization *;. Γ (νΜ,) " good ^ mT, i index values (RI) and the overall similarity and "value (). The so-called "second stage" virtual confidence indicator and similarity indicator county are getting jobs from the machine;; ^ == adding the process parameters and actual measurement values of the two pieces of 82 to the historical system & parameter reading and historical volume dragon , to re-evaluate mode 60, confidence indicator module 4〇 reference mode and similarity index = group 50 statistical distance mode, and then recalculate each of the card = 8 虚拟 virtual value (VMlI) and its The accompanying confidence indicator and the following theoretical basis for the description of the estimation model, the confidence index value (reference model) and the overall similarity index value (statistical distance model). Estimation mode and confidence index (as shown in Table 1, assume that the current E set to n sets of measurement data, including process data (^=/, 2,..., „) and their corresponding The actual measured value data (is ^^ 2,...,π) 'where each set of process data contains p parameters (from parameter 1 to parameter p), ie \ = [\,,···,~b, It also collects ((4)) the process data of the actual production of the pen, but there is no actual measurement data except for „+1, that is, in the actual workpiece produced by the pen, only the first workpiece is actually measured for actual measurement. Then, based on its actual measurement of eight +1, the other (w_w") pen workpiece is inferred. 201135474 Table 1 original data sample sample data point parameter 1 parameter 2 parameter P actual measurement value 1 ^1,2 • · · Xhp less 1 2 χ2, \ x2,2 • * « Χ2,Ρ yi ... ... ... ... ... ... η 〜,1心,2 • · · XfhP yn η+1 ^/7+1,1 ^m+1, 2 • · · ^n+\yp yn+\ η+2 χη+2,\ χη+2,2 # # · ^n+2,p Zip ... ... ... · · ... ... m ^m,\ •V2 ... Xm , p Zip

在表1中,乃、h、...、Λ為歷史量測值,Λ+1為正 在生產中之工件批貨中之第一個工件的實際量測值。通 常,一組實際量測值(乃,ζ·=1,2,…,η)為具有平均數//,標 準差σ的常態分配,即乃〜Λ^,σ2)。 針對樣本組(兄_,/=1,2,…,4之平均數與標準差將所有 實際量測值資料標準化後,可得到\、Ζ,2、...、(亦稱 為Ζ分數(z Scores)),其中每一個Ζ分數之平均數為0,標 準差為1,即\〜#(〇,1)。對實際量測資料而言,若4愈接近 〇,則表示量測資料愈接近規格中心值。其標準化之公式如 201135474 7 ,ΐ-1,2,···,η σ少 (1) y=-(yi+y2+---+yn) η (2) iyi~y)2 +(y2~y)2 +··^^-j)2] (3) 其中 兄為第/組實際量測值資料; ^為在第Z·組資料標準化後的實際量測值資料; J為所有實際量測值資料的平均數; ' S為所有實際量測值資料的標準差; 此處之說明係應用類神經網路(NN)演算法之推估:寅算 法來建立進行虛擬量測的推估模式’並以例如迴歸演算法 之參考預測演算法來建立的驗證此推估模式的參考模式。 然而,本發明亦可使用其他演算法為推估演算法或參考預 測演鼻法’只要參考預測演鼻法係不同於推估演算法即 可’故本發明並不在此限。本發明之推估演算法和參考預 測演算法可分別為例如:倒傳遞類神經網路(Back Propagation Neural Network ; BPNN)、通用迴歸類神經網路 (General Regression Neural Network ; GRNN)、徑向基底類 神經網路(Radial Basis Function Neural Network ; RBFNN)、 簡單回歸性網路(Simple Recurrent Network ; SRN)、支持向 量資料描述(Support Vector Data Description ; SVDD)、支持 201135474 向量機(Support Vector Machine ; SVM)、複迴歸演算法 (Multiple Regression ; MR);部分最小平方法(Partial Least Squares ; PLS)、非線性替代偏最小平方法(Nonlinear Iterative Partial Least Squares ; NIPALS)或廣義線性模式 (Generalized linear models ; GLMs)。In Table 1, y, h, ..., Λ are historical measurements, and Λ +1 is the actual measured value of the first workpiece in the workpiece batch being produced. Usually, a set of actual measured values (ie, ζ·=1, 2, ..., η) is a normal distribution with an average of / /, the standard deviation σ, that is, ~Λ^, σ2). For the sample group (the average of the brothers _, / = 1, 2, ..., 4 and the standard deviation to normalize all the actual measured data, you can get \, Ζ, 2, ..., (also known as Ζ score (z Scores)), where the average of each score is 0, and the standard deviation is 1, ie \~#(〇,1). For actual measurement data, if 4 is closer to 〇, it means measurement The closer the data is to the central value of the specification, the standardized formula such as 201135474 7 , ΐ-1, 2, ···, η σ is less (1) y=-(yi+y2+---+yn) η (2) iyi~ y)2 +(y2~y)2 +··^^-j)2] (3) where the brother is the actual measured value of the group/group; ^ is the actual measured value after the standardization of the Z-group data Data; J is the average of all actual measured data; 'S is the standard deviation of all actual measured data; the description here is the application of neural network (NN) algorithm estimation: 寅 algorithm to establish The estimation mode of the virtual measurement is performed and the reference mode for verifying the estimation mode is established by, for example, a reference prediction algorithm of the regression algorithm. However, the present invention may also use other algorithms as the estimation algorithm or the reference prediction method. As long as the reference prediction algorithm is different from the estimation algorithm, the present invention is not limited thereto. The estimation algorithm and the reference prediction algorithm of the present invention may be, for example, a Back Propagation Neural Network (BPNN), a General Regression Neural Network (GRNN), a radial basis. Radial Basis Function Neural Network (RBFNN), Simple Recurrent Network (SRN), Support Vector Data Description (SVDD), Support 201135474 Vector Machine (Support Vector Machine; SVM) ), Multiple Regression (MR); Partial Least Squares (PLS), Nonlinear Iterative Partial Least Squares (NIPALS) or Generalized Linear Models (Generalized Linear Models; GLMs).

在應用類神經網路演算法和複迴歸演算法時,如其收 敛條件均為誤差平方和(Sum of Square Error ; SSE)最小的 條件下,且《+〇〇時,此兩模式各自標準化後的實際量測值 疋義為與^,則其均應與真正標準化後的實際量測值 &相同。換言之,當時,今=2外=^均代表標準化後 的實際量測值,但為因應不同模式之目的而改變其名稱。 因此、〜,吃)’且a。~斗&,σ!),表示'與〜為相同分配, 由於不同的估§十模式,使得該兩種預測演算法之平均值 與“準差的估計值不同。亦即ΝΝ推估模式標準化後的平 均數估計式(\=〜,)與鮮差估計式(V&)將與複迴歸 模式標準化後的平均數估計式(與標準差估計式 (气^吃)不同。 1心指標值係被設計來簡虛擬量測值的可信賴度, 此U指標值應考㈣虛擬㈣值之統計分配〜與實 際量測值之統計分配z„兩者之_相_度。然而,❹ 1虛擬量測時’並無實際量測值可被使用來評估虛擬量測 的可信賴度(明顯地,若獲得實際量測值則便不需要轉 12 201135474 量測了)。所以本發明採用由參考預測演算法(例如複迴歸 演算法)所估算之統計分配&來取代&之統計分配。本發明 之參考預測演算法亦可為其他相關之預測演算法,故本發 明並不在此限。 睛參照第3圖,其繪示說明本發明之實施例之信心指 標值的示意圖。本發明之信心指標值的定義為計算推估模 式(例如採用類神經網路(NN)演算法)之預測(虚擬量測值) 的分配與參考模式(例如採用複迴歸演算法)之預測(參 考量測值)的分配z允兩者之間的交集面積覆蓋值(重疊面積 A)。因此,信心指標值的公式如下: πσ dx (4) 其中 鲁 當ζ 當則卜2& 則4 = ^. σ係設為1 私山t。心彳示值係隨著重疊面積Α的增加而增加。此現象 :鏵π ^ ί估模式所獲得的結果係較接近於使用參考模式 f, Λ U而相對應之虛擬量測值較可靠。否則相 低^f量測值的可靠度係隨著重疊面積A的減少而降 番矗I 所估計之分配>,與由;所估計之分配&完全 依照統計學的分配理論,其信心指標值等於i; 而當兩分配幾半6人、 以 丁疋全分開時,其信心指標值則趨近於0。 乂下說明推估模式計算虛擬量測值$ 13 201135474 分配的方法。 (SSE)在:二ί: ’:收敛條件為最小化誤差平方和 變異數為咬的分配」,即给〇下,针均數等於 %的ΝΝ估計式為》 ,2二 而 外呤的ΝΝ估計式為。 在進仃NN推估模式的建模制〜 標準化的步驟。 j而先進仃製程資料 NN推估模式製程資料標準化公式如下所示: zv 5«—,厂无/ xj ·ΐ = 1,2,··,n,n^-1,---,m; j = 1,2,-..,When applying the neural network algorithm and the complex regression algorithm, if the convergence condition is the smallest Sum of Square Error (SSE), and the normalization of the two modes after the +〇〇 If the measured values are ambiguous and ^, they should be the same as the actual measured values & In other words, at the time, today = 2 outside = ^ represents the actual measured value after standardization, but the name is changed for the purpose of different modes. Therefore, ~, eat)' and a. ~ bucket &, σ!), indicating that 'and ~ are the same assignment, because the different estimates of the § 10 mode, the average of the two prediction algorithms is different from the estimate of the quasi-difference. The normalized average number estimation formula (\=~,) and the fresh error estimation formula (V&) are different from the standard deviation estimation formula (which is different from the standard deviation estimation formula (qi^^). The value is designed to calculate the trustworthiness of the virtual measured value. The value of this U index should be (4) the statistical distribution of the virtual (four) value ~ the statistical distribution of the actual measured value z _ phase _ degrees. However, ❹ 1 In the virtual measurement, there is no actual measurement value that can be used to evaluate the reliability of the virtual measurement (obviously, if the actual measurement is obtained, it does not need to be measured 12 201135474). Therefore, the present invention adopts The statistical allocations estimated by reference prediction algorithms (such as complex regression algorithms) are substituted for the statistical allocation of & the reference prediction algorithm of the present invention may also be other related prediction algorithms, so the present invention is not limited thereto. Eyes refer to Figure 3, which illustrates the hair Schematic diagram of the confidence indicator value of the embodiment. The confidence indicator value of the present invention is defined as the allocation and reference mode of the prediction (virtual measurement value) of the calculation estimation mode (for example, using a neural network (NN) algorithm) ( For example, the distribution of the prediction (reference measurement value) using the complex regression algorithm allows the intersection area coverage value (overlap area A) between the two. Therefore, the formula of the confidence index value is as follows: πσ dx (4) where Lu When ζ 则 2 & then 4 = ^. σ is set to 1 私山 t. The 彳 彳 值 随着 随着 随着 随着 随着 。 彳 彳 彳 彳 彳 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 。 It is closer to using the reference mode f, Λ U and the corresponding virtual measurement value is more reliable. Otherwise, the reliability of the phase measurement value decreases with the decrease of the overlap area A. ;, and the distribution; the estimated distribution & completely according to the statistical distribution theory, the confidence index value is equal to i; and when the two distributions are divided into six, separated by Ding, the confidence index value approaches 0. His description of the estimation mode to calculate the virtual measurement value $ 13 20113547 4 The method of allocation (SSE) is: 2 ί: ': The convergence condition is to minimize the squared error and the variance is the allocation of the bite, that is, to give the underarm, the mean of the needle is equal to %, and the estimate is "2" The ΝΝ estimate of the outer 呤 is: the modeling system of the NN estimation model ~ the standardization step. j and the advanced 仃 process data NN estimation model process data standardization formula is as follows: zv 5«—, factory no / xj ·ΐ = 1,2,··,n,n^-1,---,m; j = 1,2,-..,

P (5) 3c,. + Ar,,. + …+ ΛΓ ⑹P (5) 3c,. + Ar,,. + ...+ ΛΓ (6)

XJXJ

Xl ⑺ 其中 〜為第/組製程資料中之第y.個製程參數; 為第 數; z經製程資射之第/個標準化後的製程參 巧為第/個製程參數的平均值; %為第y·個製程參數的標準差; 使用此《組標準化後的製程資料 h _ 2.12 輸入所組標準化後的製程資料 應之標 (¾,卜,”·,切,户/上一至NN推估模式中,以 抖 準化後的虛擬量測值〜、: 獲于相對 (8) 201135474Xl (7) where ~ is the y. process parameter in the /group process data; is the number; z is the average of the process parameters of the process / the standard process after the standard process; The standard deviation of the first y· process parameters; use this “standardized process data h _ 2.12 to input the standardized process data should be marked (3⁄4, 卜,”·, cut, household/previous to NN estimation In the mode, the virtual measured value after the jitter is ~, : is obtained in relative (8) 201135474

0ZyN = zyNi,i = J,2r",n,n+J,...,m =i(zyNI + zh2 + - + zhn) 其中為標準化後之虛擬量測值的平均值 ⑼(1〇) 方法以下說明由複迴歸模式計算參考預測值(&,和~ )的 複迴歸演算法的基本假設為「在給定心下,〜的分 配為平均數等於%,變異數為吃的分配」,即給定2、下, 而%的複迴歸估計式為〜4的複迴歸 估計式吃=¾。 • 為求得植標準化後的製程資料以…與 此η組標準化後的實際量測值/=仏,”)間的關係,須定義 利用複迴歸分析中這些Ρ個參數所對應的權重為 。建構與關係如下: fir0+βΓ ,ΖΧιι + fir2ZXi2 + ·. - + prp2^ ^ = β^βΓΙζΧ2+βΓ2ζΧ22+-+βκρΖ^ρ =z' … lp y2 (11)0ZyN = zyNi,i = J,2r",n,n+J,...,m =i(zyNI + zh2 + - + zhn) where is the average of the normalized virtual measurements (9) (1〇) Method The following is a basic hypothesis that the complex regression algorithm for calculating reference prediction values (&, and ~) from the complex regression mode is "under a given heart, the distribution of ~ is equal to %, and the variance is the distribution of food." That is, given 2, and the complex regression estimate of % is a complex regression estimate of ~4 eat = 3⁄4. • In order to obtain the relationship between the process data after standardization and the actual measured value of the η group /=仏,”), the weights corresponding to these parameters in the complex regression analysis must be defined. The construction and relationship are as follows: fir0+βΓ , ΖΧιι + fir2ZXi2 + ·. - + prp2^ ^ = β^βΓΙζΧ2+βΓ2ζΧ22+-+βκρΖ^ρ =z' ... lp y2 (11)

Pr0 + βτ,Ζχη, + β^Ζ\,2 + ' + KZX = Zv n,p y» r L ύ 15 (12) 201135474 假設 k,、 Ζ· 2 yi Ζ y" ζχ 2,1 ζχ 、 ι,ρ ζν Ιρ 2 V η,Ι 夂,··· ζν η.ρ (13) 數析:的最小平方法’可求得參 (14) β^{ζτχΖχγ zrZy 然後,複迴歸模式可得到 【一 7,2,…,η,η+人…,m .+ β^ζ} i-p (15) 艮。因此,在推估階段時,製程資料進來後,依公式(15) 、P可求出其所對應的複迴歸估計值Z&+。標準變異數〜的複 迴歸估計式為、具有: π ' (16) 〜4(W"+z:pJ (17) 田求侍NN推估模式的估計式^,與彡〜及複迴歸模式 201135474 的估计式、與6後’可繪出如第3圖所示之常態分配圖, 計算使用推估模式(例如採用類神經網路(NN)演算法)之預 測(虛擬篁測值)的分配與參考模式(例如採用複迴歸演算法) 之預測(參考量測值)的分配兩者之間的交集面積覆蓋值(重 疊面積A) ’即可求出每一個虛擬量測值的信心指標值。 在獲得信心指標值(RI)後,必須要訂定一個信心指祅 門檻值(RIt)。若RI < RIt,則具有此RI工件的虛擬量^ 的可靠程度低,亦即具有此RI之工件的品質較可能異常, 故需進行實際量測。以下描述決定信心指標門檻值㈣的 方法: 在訂定信心指標門播值(RIt)之前,首先需訂定出最大 可容許誤差上限⑹。虛擬量測值的誤差(办叫為實際量 測值减由NN推估模式所獲得之&的差值,再除以二 實際量測值的平均值後之絕對值的百分率,即Pr0 + βτ, Ζχη, + β^Ζ\, 2 + ' + KZX = Zv n, py» r L ύ 15 (12) 201135474 Suppose k,, Ζ· 2 yi Ζ y" ζχ 2,1 ζχ , ι, ρ ζν Ιρ 2 V η,Ι 夂,··· ζν η.ρ (13) Number analysis: the least square method 'can be obtained as reference (14) β^{ζτχΖχγ zrZy Then, the complex regression mode can be obtained [1-7] , 2,...,η,η+人...,m .+ β^ζ} ip (15) 艮. Therefore, in the estimation stage, after the process data comes in, the complex regression estimate Z&+ corresponding to it can be obtained according to formulas (15) and P. The standard regression number ~ complex regression estimator is: π ' (16) ~ 4 (W"+z:pJ (17) Tian Qiu NN estimation model estimation formula ^, and 彡 ~ and complex regression model 201135474 The estimation formula, and the post-6' can be plotted as the normal distribution map shown in Figure 3, and the calculation uses the estimation (such as the neural network (NN) algorithm) to predict (virtual guess). The intersection area coverage value (overlap area A) between the distribution of the prediction (reference measurement value) of the reference mode (for example, using the complex regression algorithm) can be used to obtain the confidence index value of each virtual measurement value. After obtaining the confidence index value (RI), a confidence index threshold (RIt) must be established. If RI < RIt, the virtual quantity ^ of the RI workpiece has a low degree of reliability, that is, has this RI The quality of the workpiece is more likely to be abnormal, so the actual measurement is required. The following describes the method of determining the confidence threshold (4): Before setting the confidence indicator (RIt), the maximum allowable error upper limit must be set first (6) The error of the virtual measurement value (called the actual measurement value minus NN) The percentage of the absolute difference between the average value, divided by two actual measurement value, i.e.; estimation & amp pattern of the obtained

Εηοη y> y χίοο% (18) 然後’可根據公式(18)所定義之誤差與虛擬量測 確度規格來減最大可料誤差上限⑹。因此,信心指 標門檻值(RIT)係被定義為對應至最大可容許誤差上限曰 之仿心指標值(RI),如第4圖所示。即,Εηοη y> y χίοο% (18) Then 'the maximum allowable error upper limit (6) can be reduced according to the error defined by equation (18) and the virtual quantity measurement specification. Therefore, the confidence index threshold (RIT) is defined as the reference index value (RI) corresponding to the maximum allowable error upper limit ,, as shown in Figure 4. which is,

(19)(19)

17 201135474 //和σ係定義於公式(4)中;及 2center = Zym + [?X ^)]/ay (20) 其中t係定義於公式(3)中。 整艚相似度指標(GSI) 如上所述,當應用虛擬量測時,並未有實際量測值可 獲得來驗證虛擬量測值的精確度。因此,以標準化後的複 迴歸估計值\取代標準化後的實際量測值4來計算信心 指標值(RI)。然而,此種取代可能會造成信心指標值(RI) 的誤差,為了補償這種情形,本發明提出製程的整體相似 度指標(GSI)來幫助判斷虛擬量測的可靠程度。 本發明所提出之GSI的概念是將目前採用來當虛擬量 測系統之輸入的設備製程資料與建模時的所有歷史參數資 料相比較,得到一輸入之製程資料與所有歷史參數資料的 相似程度指標。 本發明可用各種不同的統計距離演算法來量化相似 度’例如:馬氏距離演算法(Mahalanobis Distance)、歐式 距離演算法(Euclidean Distance)和中心法(Centroid Method) 等。馬氏距離係由P.C. Mahalanobis於西元1936年所介紹 之統計距離演算法。此種技術手段係基於變數間的關聯性 以辨識和分析不同樣本組的型態。馬氏距離係用以決定未 知樣本組與已知樣本組間之相似度的方法,此方法考量資 料組間的關聯性並具有尺度不變性(Scale invariant),即不 與$測值的大小相關。若資料具有高相似度,則所計算出 之馬氏距離將會較小。 201135474 本發明係利用所計算出之GSI(馬氏距離)的大小,來分 辨新進之製程資料是否相似於建模的所有製程資料。若計 算出的GSI小,則表示新進之製程資料類似於建模的製程 資料,因此新進之製程資料(高相似度)的虛擬量測值將會 較準確。反之,若計算出之GSI過大,則表示新進之製程 資料與建模的製程資料有些不同。因而具有新進之製程資 料(低相似度)之工件的品質較可能異常,故需進行實際量 測。 推估模式之標準化製程參數z,的計算公式係如式17 201135474 //The σ is defined in equation (4); and 2center = Zym + [?X ^)]/ay (20) where t is defined in equation (3). Twice Similarity Indicator (GSI) As mentioned above, when virtual measurement is applied, no actual measurement is available to verify the accuracy of the virtual measurement. Therefore, the confidence index value (RI) is calculated by substituting the normalized complex regression estimate\ instead of the normalized actual measured value 4. However, such substitution may result in an error in the confidence index value (RI). To compensate for this situation, the present invention proposes a global similarity index (GSI) of the process to help determine the reliability of the virtual measurement. The concept of the GSI proposed by the present invention compares the device process data currently used as the input of the virtual measurement system with all the historical parameter data at the time of modeling, and obtains the similarity between the input process data and all historical parameter data. index. The present invention can quantize similarities using various statistical distance algorithms, such as Mahalanobis Distance, Euclidean Distance, and Centroid Method. The Markov distance is a statistical distance algorithm introduced by P.C. Mahalanobis in 1936. This technique is based on the correlation between variables to identify and analyze the patterns of different sample groups. The Mahalanobis distance is a method for determining the similarity between an unknown sample group and a known sample group. This method considers the correlation between data sets and has a scale invariant, that is, it is not related to the magnitude of the measured value. . If the data has a high similarity, the calculated Mahalanobis distance will be smaller. 201135474 The present invention utilizes the calculated magnitude of GSI (Machine Distance) to distinguish whether the new process data is similar to all process data of the model. If the calculated GSI is small, it means that the new process data is similar to the model process data, so the virtual measurement value of the new process data (high similarity) will be more accurate. Conversely, if the calculated GSI is too large, it means that the new process data is somewhat different from the modeled process data. Therefore, the quality of the workpiece with the new process data (low similarity) is more likely to be abnormal, so actual measurement is required. The calculation formula of the standardized process parameter z of the estimation model is as follows

l,J (5)、(6)和(7)所示。首先,定義樣版參數資料 ,其中等於力,。如此’則標準 化後之建模製程資料之各參數均為〇(亦即標準化後之建模 參數‘為〇)。換言之,4 = 中之所有參數均 為0。接下來計算各個標準化後建模參數之間的相關係數。 假設第s個參數與第t個參數之間的相關係數為rst, 而其中有纽資料,則l, J (5), (6) and (7). First, define the pattern parameter data, which is equal to force. Thus, the parameters of the standardized modeling process data are all 〇 (that is, the standardized modeling parameter ‘is 〇). In other words, all parameters in 4 = are 0. Next, the correlation coefficient between each standardized modeling parameter is calculated. Suppose the correlation coefficient between the sth parameter and the tth parameter is rst, and there is a key information, then

rst k— 1 1-} ^srztl=~,~:{zsrztl+zs2'zt2+- + zsk'ztk) k-1 (21) 在完成計算各參數間的相關係數之後,可得到相關係 數矩陣如下: 1 r12 "· rlp r21 1 "· r2p ^pl ^p2 ... 7 19 (22) (23) 201135474 假設及的反矩陣(及乃係被定義為j,則Rst k— 1 1-} ^srztl=~,~:{zsrztl+zs2'zt2+- + zsk'ztk) k-1 (21) After completing the calculation of the correlation coefficient between the parameters, the correlation coefficient matrix can be obtained as follows: 1 r12 "· rlp r21 1 "· r2p ^pl ^p2 ... 7 19 (22) (23) 201135474 The hypothesis and the inverse matrix (and the system is defined as j, then

an a21 dpi °12 ··. A p a22 … ^p2 ·· aPP A = RJ = 如此,第2筆標準化之製程參數(zj與標準化之樣版參 數資料(zj間的馬氏距離(以)計算公式如下: Ό2λ=(Ζλ-ΖΜγκ-](ζλ-ΖΜ) (24) (25) 可得 = Σ Σ ^ij^azβ j=li=l 而第2筆製程資料之GSI值為z)f/p。 在獲得GSI值後,應用交互驗證(cross Validation)中留 一法(Leave-One-Out ; LOO)原理來定義出GSI門檻值 (GSIT)。GSI門檻值(GSIT)的公式如下: T = a * GSI L〇〇 (26) 所謂「留一法(Leave-One-Out; LOO)原理」係從全部 建模樣本中,抽取一筆作為模擬上線之測試樣本,再使用 其餘的樣本建立GSI模型,然後應用此新建之GSI模型針 對此筆模擬上線之測試樣本計算出其GSI值,此值以 GSILOO表示。接著重覆上述步驟直到建模樣本中所有各筆 201135474 樣本均計算出其相對應之GSIL00。因此,公式(26)中运 代表透過LOO原理由全部建模樣本所計算出之所有 GSILOO的例如90%截尾平均數(Trimme(i Mean)。公式(26) 之a值係介於2至3之間,其可依實際狀況微調之,&之預 設值為3。 以下說明本發明之工件抽樣檢驗方法。 °月參照第5圖,其繪示根據本發明之實施例之工件抽 樣檢驗方法的流程示意圖。在建立推估模式、參考模式和 • 統"十距離模式;及獲得信心指標門檻值(RIT)和GSI門檻值 (GS τ)後,輪入卡匣内之每一個工件的製程參數資料至上 述之推=模式、參考模式和統計距離模式,以計算每一個 工件的传心指標值(RI)和GSI值(步驟1〇〇)。接著,對每一 ^⑧進行步驟110,以判斷其信心指標值(RI)是否小於 彳样門板值(RIt);或其GSI值是否大於asi門檻值 — 若步驟110的判斷結果為是則進行步驟120,以決 符合步驟11G之條件駐件進行量測;否則結束 #是則^之工件抽樣檢驗方法。若步驟120的判斷結果為 疋杏仃步驟120,以由量測機台對此工件進行量 所有I夺t 係對卡®内之 U〇之條件的工件進行量測。在另-實施例 需自奸同一卡Ε内之每一個工件的特性相同,因此,只 lit :之符合步驟110之條件的工件中選出至少-工 士摩/了㈣即可。若步驟12G的判斷結果為否,則結束 本實施例之工件抽樣檢驗方法。 、、° 可理解的是,本發明之工件抽樣檢驗的方法為以上所An a21 dpi °12 ··. A p a22 ... ^p2 ·· aPP A = RJ = Thus, the second standardized process parameter (zj and the standardized pattern data (the Mahalanobis distance between zj) The formula is as follows: Ό2λ=(Ζλ-ΖΜγκ-](ζλ-ΖΜ) (24) (25) Available = Σ Σ ^ij^azβ j=li=l and the GSI value of the second process data is z)f/ p. After obtaining the GSI value, apply the Leave-One-Out (LOO) principle in the cross validation to define the GSI threshold (GSIT). The formula for the GSI threshold (GSIT) is as follows: = a * GSI L〇〇(26) The so-called "Leave-One-Out (LOO) principle" extracts a test sample from the entire modeled sample as a simulated upper line, and then uses the remaining samples to establish a GSI. The model, then apply this newly created GSI model to calculate its GSI value for the test sample on the simulated line, which is represented by GSILOO. Then repeat the above steps until all the 201135474 samples in the modeled sample are calculated. GSIL00. Therefore, Equation (26) represents an example of all GSILOO calculated from all modeled samples by the LOO principle. Such as the 90% truncated average (Trimme (i Mean). The value of the formula (26) a is between 2 and 3, which can be fine-tuned according to the actual situation, the default value of & is 3. Inventive sample sampling inspection method. [Month] Referring to Figure 5, there is shown a flow chart of a sample sampling inspection method according to an embodiment of the present invention, in which a estimation mode, a reference mode, and a "ten distance mode" are established; After obtaining the confidence index threshold (RIT) and the GSI threshold (GS τ), the process parameter data of each workpiece in the cassette is transferred to the above-mentioned push mode, reference mode and statistical distance mode to calculate each workpiece. The centroid index value (RI) and the GSI value (step 1〇〇). Next, step 110 is performed for each ^8 to determine whether the confidence index value (RI) is less than the threshold value (RIt); Whether the GSI value is greater than the asi threshold value - if the determination result in step 110 is yes, proceed to step 120 to determine the condition of the step 11G to perform the measurement; otherwise, the end # is the workpiece sampling inspection method. The judgment result is 疋 仃 仃 step 120, to measure the machine This workpiece is measured by the amount of workpieces that are in the condition of U〇 in the card. In the other embodiment, the characteristics of each workpiece in the same cassette are the same, so only lit: The workpiece that meets the conditions of step 110 may be selected from at least - the mechanics / (four). If the result of the determination in step 12G is negative, the workpiece sampling inspection method of this embodiment is ended. It can be understood that the method for sampling inspection of the workpiece of the present invention is as described above.

21 201135474 述之實施步驟’本發明之内儲用於工件抽樣檢驗之電腦程 式產〇〇’係用以完成如上述之工件抽樣檢驗的方法。 請參照第6A圖至第6C圖,第6A圖為繪示本發明之21 201135474 Description of the Operation Steps The computer program for storing sample inspections of the present invention is used to perform the method of sampling inspection of the workpiece as described above. Please refer to FIG. 6A to FIG. 6C, and FIG. 6A is a diagram illustrating the present invention.

應用例之虛擬量測值和實際量測值的結果示意圖;第6B 圖為繪示本發明之應用例之信心指標值的結果示意圖;第 6 C圖為繪不本發明之應用例之整體相似度指標值的結果 示意圖。 如第6A圖所示,第1至100筆資料為第1至100個 • 工件之歷史量測值,其分別對應至複數組歷史製程參數資 料,用以建立推估模式、參考模式和統計距離模式;及獲 得信心指標門檻值(rIt)和GSI門檻值(GSIT)。第10丨至125 筆資料為卡匣内之複數個工件,其中第101個工件係被習 知之方式所抽測,故具有實際量測值,用以調校建立推估 模式、參考模式和統計距離模式。本發明之工件抽樣檢驗 方法的目的係在於:自第102至125個工件中有效地抽選 出合適的工件以進行量測,而避免發生漏偵測的情形,並 鲁 能及時發現生產機台異常。 、 如第6B圖所示,利用信心指標值(RI)來判斷第幾個工 件(以第幾筆資料組來表示)需被送至量測機台以進行檢 測。接著,如第6C圖所示,利用整體相似度指標值(Gsi) 來判斷工件資料與建模資料的相似程度。其中,第114 資料組’雖然其RI大於RIt(0.567),但其GSI大於 GSIT(5.093),代表需將第114個工件送至量測機台以進行 . 檢測,以預防漏偵测之情況發生。而第107筆與第12〇筆 資料,因其RI小於RIt且因其GSI大於GSIt,故需將第 22 201135474 ι〇7、12〇個工件送至量測機台以進行檢測,以預防涯 之情況發生。除第107筆、114筆與120筆資料外,因^ 其餘工件之RI大於RIt且其GSI小於GSIt,代表無須對 除第107、1H和120個工件外之工件進行檢測,因而節省 人力物力。在另一實施例中,本發明亦可僅自第1〇7、 和120個工件中選出至少一個工件來進行量測。 由上述本發明較佳實施例可知’本發明之工件抽樣檢 驗的方法可有效地抽選出合適的工件以進行量測,而避免 籲發生漏谓測的情形,並能及時發現生產機台異常。。 雖然本發明已以實施方式揭露如上,然其並非用以限 定本發明’任何在此技術領域中具有通常知識者,在不脫 離本發明之精神和範圍内,當可作各種之更動與潤都,因 此本發明之保護範圍當視後附之申請專利範圍所界定者為 準。 【圖式簡單說明】 • 為讓本發明之上述和其他目的、特徵、優點與實施例 月巨更明顯易懂’所附圖式之說明如下: 第1圖為綠示實施本發明之工件抽樣檢驗的方法的系 統架構不意圖 第2圖為繪示根據本發明之實施例之AVM系统的架構 不意圖。 第3圖為纷示說明本發明之實施例之信心指標值的示 意圖。 第4圖為緣示說明本發明之實施例之信心指標門檻僅^ 1 Ci 23 201135474 的示意圖。 第5圖為繪示根據本發明之實施例之工件抽樣檢驗方 法的流程示意圖。 第6A圖為繪示本發明之應用例之虛擬量測值和實際 量測值的結果示意圖。 第6B圖為繪示本發明之應用例之信心指標值的結果 示意圖;第6C圖為繪示本發明之應用例之整體相似度指標 值的結果示意圖。A schematic diagram of the results of the virtual measured value and the actual measured value of the application example; FIG. 6B is a schematic diagram showing the result of the confidence index value of the application example of the present invention; FIG. 6C is a diagram showing the overall similarity of the application example of the present invention Schematic diagram of the results of the indicator values. As shown in Figure 6A, the first to the hundredth data are the historical measurements of the first to the 100th workpieces, which correspond to the complex array history process parameter data, respectively, to establish the estimation mode, the reference mode, and the statistical distance. Mode; and obtain confidence index threshold (rIt) and GSI threshold (GSIT). The 10th to the 125th data is a plurality of workpieces in the cassette, wherein the 101st workpiece is sampled by a conventional method, so the actual measurement value is used to adjust the establishment of the estimation mode, the reference mode and the statistical distance. mode. The purpose of the sample sampling inspection method of the present invention is to effectively select a suitable workpiece from the 102nd to 125th workpieces for measurement, thereby avoiding the occurrence of leak detection, and detecting the abnormality of the production machine in time. As shown in Figure 6B, the confidence index value (RI) is used to determine that the first workpiece (indicated by the first data set) needs to be sent to the measuring machine for testing. Next, as shown in Fig. 6C, the overall similarity index value (Gsi) is used to determine the degree of similarity between the workpiece data and the modeling data. Among them, the 114th data group's although its RI is greater than RIT (0.567), but its GSI is greater than GSIT (5.093), which means that the 114th workpiece needs to be sent to the measuring machine for detection to prevent leakage detection. occur. The 107th and 12th data, because its RI is less than RIt and because its GSI is greater than GSIt, it is necessary to send the 22nd 201135474 ι〇7, 12〇 workpieces to the measuring machine for detection to prevent the RI. The situation happened. In addition to the 107th, 114th and 120th data, since the rest of the workpiece has an RI greater than RIt and its GSI is less than GSIt, it means that it is not necessary to detect the workpieces other than the 107th, 1Hth and 120th workpieces, thus saving manpower and material resources. In another embodiment, the present invention may also select at least one of the first, seventh, and 120 workpieces for measurement. It can be seen from the above preferred embodiment of the present invention that the method of the sample sampling inspection of the present invention can effectively extract a suitable workpiece for measurement, avoiding the occurrence of a leaking test, and can timely detect the abnormality of the production machine. . The present invention has been disclosed in the above embodiments, and it is not intended to limit the invention to any of the ordinary skill in the art, and various changes and modifications may be made without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined by the scope of the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS The above description of the present invention and other objects, features, advantages and embodiments of the present invention are more clearly understood. The description of the drawings is as follows: Figure 1 is a green sampling of the workpiece to which the present invention is implemented. The system architecture of the method of inspection is not intended. FIG. 2 is a schematic diagram showing the architecture of the AVM system according to an embodiment of the present invention. Fig. 3 is a schematic view showing the value of the confidence index of the embodiment of the present invention. Figure 4 is a schematic diagram showing the confidence index threshold of the embodiment of the present invention only ^ 1 Ci 23 201135474. Fig. 5 is a flow chart showing the method of sampling inspection of a workpiece according to an embodiment of the present invention. Fig. 6A is a view showing the results of the virtual measured value and the actual measured value of the application example of the present invention. Fig. 6B is a diagram showing the result of the confidence index value of the application example of the present invention; and Fig. 6C is a diagram showing the result of the overall similarity index value of the application example of the present invention.

【主要元件符號說明】 10:製程參數資料前處理模組12 20 生產機台 22 30 量測機台 40 50 相似度指標模組 60 62 雙階段運算機制 80 82 工件 90 100 :計算工件的RI和GSI 110 :RI是否小於RIt ;或GSI ; 120 :決定是否進行量測 130 :由量測機台進行量測 A : 重疊面積[Main component symbol description] 10: Process parameter data pre-processing module 12 20 Production machine 22 30 Measuring machine 40 50 Similarity index module 60 62 Two-stage operation mechanism 80 82 Workpiece 90 100: Calculate the RI of the workpiece GSI 110: Whether the RI is smaller than the RIT; or GSI; 120: Decide whether to perform the measurement 130: Measurement by the measuring machine A: Overlap area

量測資料前處理模組 製程參數資料 信心指標模組 推估模式 卡匣 AVM 24Measurement Data Pre-Processing Module Process Parameter Data Confidence Index Module Estimation Mode Card 匣 AVM 24

Claims (1)

201135474 七、申請專利範圍: • ^ 什抽樣檢驗的方法,勹人 獲取一生產機台之複數組歷史 從-量測機台取得複數個 m 量測值分別為根據該些組歷 測值,其中該些歷史 (Workpiece)的量測值; 參數所生產之工件201135474 VII. Patent application scope: • ^ The method of sampling inspection, the deaf person obtains the complex array history of a production machine. The multi-measurement value obtained from the measurement machine is respectively based on the historical values of the groups. The measured value of the workpiece; the workpiece produced by the parameter 使用該些組歷史製程參數 立一推估模式,其中該推估模 法; ~ 資料和該些歷史量測值來建 式的建立係根據一推估演算 使用該些組歷史製程參數 建立-參考模式,其中該參考振該些歷史量測資料來 算法’該推估演算法與該參考演。建,係根據一參考演 數個=值史製程參數鳴:而綱複 數個==值史製程參數至該參考模式,而計算出複Using the set of historical process parameters to establish a estimation mode, wherein the estimation model method; ~ data and the historical measurement values are established according to a estimation algorithm using the set of historical process parameters to establish - reference a mode in which the reference oscillates the historical measurement data to the algorithm 'the estimation algorithm and the reference performance. Built, according to a reference number = value history process parameters: and multiple complex == value history process parameters to the reference mode, and calculate the complex 】十算該些歷史虛擬量測值的分配(Distribution)與 〜一^史參考預測值的分配之間的重疊面積而產生複數個 歷史L心指標值(Rdiance Index ;RI),其中當重疊面積愈 大,則信心指標值愈高,代表所對應至該些歷史虛擬量測 值的可信度愈高; 根據該些歷史虛擬量測值、該些歷史參考預測值和該 些歷史量測值來計算出一信心指標門檻值; 收集該生產機台所送出之一卡匣内之複數個工件的製 程參數資料; 25 201135474 輸入該些工件的製程參數資料至該推估模 出該些工件之複數個虛擬量測值; 、式 °十异 輸入該些工件的製程參數資料至該參考模 出該些工件之複數個參考預測值; 、式 °十异】 Calculate the overlapping area between the distribution of the historical virtual measurement values and the allocation of the reference prediction values to generate a plurality of historical L-heart index values (Rdiance Index; RI), wherein when the overlapping area The larger the confidence indicator value is, the higher the reliability corresponding to the historical virtual measurement values is represented; according to the historical virtual measurement values, the historical reference prediction values, and the historical measurement values Calculating a confidence index threshold; collecting process parameter data of a plurality of workpieces in one of the cassettes sent by the production machine; 25 201135474 inputting process parameter data of the workpieces to the plurality of workpieces a virtual measurement value; a formula of a plurality of reference values of the workpieces to which the reference molds are output; 自該些工件中選取其信心 值之至少一第一工件;以及 工件送至該量測機台以進 將該些工件之該至少一第一 行檢測。At least one first workpiece having a confidence value is selected from the workpieces; and the workpiece is sent to the measuring machine to detect the at least one first row of the workpieces. # 演异汝和一頰砰經網路(Neural Network; NN)演算法所組成 之一族群。 3.如申凊專利範圍第1項所述之工件抽樣檢驗的方 法,其中該推估演算法和該參考演算法係分別選自由一倒 傳遞類神經網路(Back Propagation Neural Network ; ΒΡ>ίΝ)、一 通用迴歸類神經網路(General Regression Neural Network ; GRNN)、一徑向基底類神經網路(Radial Basis Function Neural Network ; RBFNN)、一簡單回歸性網路 26 201135474 (Simple Recurrent Network; SRN)、一支持向量資料描述 (Support Vector Data Description ; SVDD)、一支持向量機 (Support Vector Machine ; SVM)、一 複迴歸演算法(Multiple Regression ; MR); —部分最小平方法(Partial Least Squares ; PLS)、一非線性替代偏最小平方法(Nonlinear Iterative Partial Least Squares ; NIPALS)和一廣義線性模式 (Generalized linear models ; GLMs)所組成之一族群。 4.如申請專利範圍第1項所述之工件抽樣檢驗的方 法,更包含: 使用該些組歷史製程參數,並根據一統計距離演算 法,來建立一統計距離模式; 以該些組歷史製程資料及該些歷史量測值,並應用交 互驗證(Cross Validation)中的留一法(Leave-〇ne_0ut; l〇〇) 原理來重建該統計距離模式,並計算出相對應的 GSI(Global Similarity Index ;整體相似度指標)值,以計算 出一 GSI門檻值; 輸入該些工件的製程參數資料至該統計距離模式,而 e十算出S亥些工件之該些虛擬量測值所對應之製程參數資料 的GSI值; 少 自該些工件中選取其GSI值大於該GSI門檻值之至少 一第二工件;以及 將該些工件之該至少一第二工件送至該量測機台以進 行檢驗。 27 201135474 5·如申請專利範圍第4項所述之工件抽樣檢驗的方 法,其中該統計距離演算法係選自由一馬氏距離 (Mahalanobis Distance)演算法、一歐式距離演算法 (Euclidean Distance)和一中心法(Centroid Method)所組成之 一族群。 6. —種工件抽樣檢驗的方法,包含: 獲取一生產機台之複數組歷史製程參數資料; 馨 使用該些組歷史製程參數,並根據一統計距離演算 法,來建立一統計距離模式; 以該些組歷史製程資料及該些歷史量測值,並應用交 互驗證中的留一法原理來重建該統計距離模式,並計算出 相對應的GSI值,以計算出一 GSI門檻值; 輸入該些工件的製程參數資料至該統計距離模式,而 計算出該些工件之該些虛擬量測值所對應之製程參數資料 的GSI值; • 自該些工件中選取其GSI值大於該GSI門檻值之至少 一第一工件;以及 將s亥些工件中之該至少一第一工件送至一量測機台以 進行檢測。 7. 如申請專利範圍第6項所述之工件抽樣檢驗的方 法,其中該統什距離演算法為係選自由一馬氏距離演算 法、一歐式距離演算法和一中心法所組成之一族群。 28 δ.如申讀蓴利範 法,更包含: 弟6項所述之工件抽樣檢驗的方 從誘量蜊機台取得福 量測值分别為根據 數個歷之量踯值,其中該些歷欠 測值;根據‘域歷史製毅參鼓所生產之工件的量 使用該些組歷史製 立一推估模式,其中 數貝科和該些歷史量測值來建 法; X估模式的建立係根據一推估演箕 建立些歷史量測資㈣ *法輸法與該參考二係根據—參考' 數心史虛擬=製程參數至該推估模式’而計算出《 數個以=製程參數至該參考模式’而計㈣ 預測擬量測值的分配與該些歷史參, 町刀配之間的重疊面積而產生複數個歷史信心# ’其中當重疊面積愈大,則信心指標值愈高,代表戶^ 應至該些歷史虛擬量測值的可信度愈高; 1 根據該些歷史虛擬量測值、該些歷史參考預測值和驾 些歷史量測值來計算出一信心指標門檻值; 〜 收集該生產機台所送出之一卡匣内之複數個工件 程參數資料; 、$ 輸入該些工件的製程參數資料至該推估模式,而計算 出該些工件之複數個虛擬量測值; 201135474 些複 1:=料至該參考模式,算 分別計算該此工杜 考預測值的分配:間的分配與該些參 信心指標值,面積Μ生該些I件之複數個 代表所對魅糾積愈大’難心指標值愈高, 一虛擬量洌值的可信度愈高,· 標門檻 值之至件::取其信心指標值小於該信心指 吊—工件;以及 工件送至該量測機台以 、將該些工件中之該至少一第 進行檢驗。 法,其中該申推"^^\圍第8項所述之工件抽樣檢驗的方 算法和自由一迴歸演 法,其中該推:工件抽樣檢驗的方 :神經網路、1用迴 ,、-簡單回歸性網路、一支:-獲向基底類神經 -複迴歸演算法;一=資料插述、-支持 代偏最小平方法和—廣義線性模、平方法、-非線性 π組成之一族群。 6,、载入此腦種式產品’當 所逑之工件抽樣檢驗的方法。 凡成如請求項1或# 汝 汝 汝 一 一 一 一 一 一 一 一 一 一 一 一 一 一 一 Ne Ne Ne Ne Ne Ne Ne Ne Ne 3. The method of sampling inspection of a workpiece according to claim 1, wherein the estimation algorithm and the reference algorithm are respectively selected from a Back Propagation Neural Network (ΒΡ> ΒΡ> ), a General Regression Neural Network (GRNN), a Radial Basis Function Neural Network (RBFNN), a simple regression network 26 201135474 (Simple Recurrent Network; SRN ), a Support Vector Data Description (SVDD), a Support Vector Machine (SVM), a Multiple Regression (MR); Partial Least Squares; PLS), a non-linear alternative partial least-square method (NIPALS) and a generalized linear model (GLMs). 4. The method for sampling inspection of a workpiece according to claim 1 of the patent application, further comprising: using the set of historical process parameters, and establishing a statistical distance mode according to a statistical distance algorithm; Data and these historical measurements, and apply the Leave-〇ne_0ut; l〇〇 principle in Cross Validation to reconstruct the statistical distance pattern and calculate the corresponding GSI (Global Similarity) Index; overall similarity index) value to calculate a GSI threshold value; input process parameter data of the workpieces to the statistical distance mode, and e ten calculate the process corresponding to the virtual measurement values of the workpieces a GSI value of the parameter data; selecting at least one second workpiece whose GSI value is greater than the GSI threshold value from the workpieces; and sending the at least one second workpiece of the workpieces to the measuring machine for inspection . 27 201135474 5. The method for sampling inspection of a workpiece according to claim 4, wherein the statistical distance algorithm is selected from a Mahalanobis Distance algorithm, an Euclidean Distance algorithm, and A group of the Centroid Method. 6. A method for sample sampling inspection, comprising: obtaining a complex array historical process parameter data of a production machine; using the group historical process parameters, and establishing a statistical distance mode according to a statistical distance algorithm; The historical process data of the group and the historical measurement values, and applying the retention-one principle in the interaction verification to reconstruct the statistical distance mode, and calculating a corresponding GSI value to calculate a GSI threshold; Process parameter data of the workpieces to the statistical distance mode, and calculating GSI values of the process parameter data corresponding to the virtual measurement values of the workpieces; • selecting GSI values from the workpieces that are greater than the GSI threshold value At least one first workpiece; and sending the at least one of the workpieces to a measuring machine for detection. 7. The method of sampling inspection of a workpiece according to claim 6, wherein the unified distance algorithm is selected from the group consisting of a Markov distance algorithm, a Euclidean distance algorithm and a central method. . 28 δ. If you apply for the Philip method, it also includes: The sampling test of the workpieces mentioned in the 6th item of the workpiece is obtained from the lure boring machine, and the measured values are based on the number of calendars, respectively. Under-measured value; based on the amount of the workpiece produced by the domain history system, the use of the group history to establish a estimation model, in which several Becco and these historical measurements are used to build the law; Based on a certain estimation and deduction, some historical quantity measurement is established. (4) * Method input method and the reference second system basis - refer to 'number heart history virtual = process parameter to the estimation mode' to calculate "several = process parameters To the reference mode's (4) predicting the distribution of the quasi-measured values and the overlap area between the historical parameters and the chores, resulting in a plurality of historical confidences # 'where the greater the overlap area, the higher the confidence index value , the higher the credibility of the representative virtual measurement values; 1 based on the historical virtual measurement values, the historical reference prediction values and the historical measurement values to calculate a confidence index threshold Value; ~ collect one of the production machines sent a plurality of workpiece process parameter data in the cassette; and $ input the process parameter data of the workpieces to the estimation mode, and calculate a plurality of virtual measurement values of the workpieces; 201135474 Some complex 1:= According to the reference mode, the calculation of the distribution of the predicted value of the work is calculated separately: the distribution between the reference and the value of the confidence indicator, and the area of the plurality of I pieces is the greater the distortion of the charm. The higher the higher the credibility of a virtual quantity depreciation, the higher the value of the threshold value: the value of the confidence indicator is less than the confidence finger-workpiece; and the workpiece is sent to the measuring machine to The at least one of the workpieces is inspected. The method, wherein the method of the sampling test of the workpiece described in Item 8 and the free-regressive method, wherein the push: the sample of the workpiece sampling test: neural network, 1 used back, - Simple regression network, one: - Obtained to the base-like neuro-regressive algorithm; one = data interpolation, - support generation partial least squares method and - generalized linear mode, flat method, - nonlinear π composition a group of people. 6, the method of sampling the inspection of the workpieces that are loaded into this brain type product. Where the requirements are 1 or
TW99110416A 2010-04-02 2010-04-02 Method for sampling workpiece for inspection and computer program product performing the same TWI427487B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW99110416A TWI427487B (en) 2010-04-02 2010-04-02 Method for sampling workpiece for inspection and computer program product performing the same

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW99110416A TWI427487B (en) 2010-04-02 2010-04-02 Method for sampling workpiece for inspection and computer program product performing the same

Publications (2)

Publication Number Publication Date
TW201135474A true TW201135474A (en) 2011-10-16
TWI427487B TWI427487B (en) 2014-02-21

Family

ID=46751902

Family Applications (1)

Application Number Title Priority Date Filing Date
TW99110416A TWI427487B (en) 2010-04-02 2010-04-02 Method for sampling workpiece for inspection and computer program product performing the same

Country Status (1)

Country Link
TW (1) TWI427487B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102262188A (en) * 2010-05-28 2011-11-30 先知科技股份有限公司 Sampling inspection method for workpieces and computer program product
CN103383429A (en) * 2012-05-04 2013-11-06 瑞鼎科技股份有限公司 Method for testing reliability of integrated circuit
CN112563152A (en) * 2019-09-25 2021-03-26 长鑫存储技术有限公司 Manufacturing method and system

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI539298B (en) 2015-05-27 2016-06-21 國立成功大學 Metrology sampling method with sampling rate decision scheme and computer program product thereof

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI283817B (en) * 2003-05-30 2007-07-11 Tokyo Electron Ltd Method of operating a process control system and method of operating an advanced process control system
TWI267012B (en) * 2004-06-03 2006-11-21 Univ Nat Cheng Kung Quality prognostics system and method for manufacturing processes
TWI315054B (en) * 2006-05-10 2009-09-21 Nat Cheng Kung Universit Method for evaluating reliance level of a virtual metrology system in product manufacturing
TWI338916B (en) * 2007-06-08 2011-03-11 Univ Nat Cheng Kung Dual-phase virtual metrology method

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102262188A (en) * 2010-05-28 2011-11-30 先知科技股份有限公司 Sampling inspection method for workpieces and computer program product
CN102262188B (en) * 2010-05-28 2013-06-05 先知科技股份有限公司 Sampling inspection method for workpieces
CN103383429A (en) * 2012-05-04 2013-11-06 瑞鼎科技股份有限公司 Method for testing reliability of integrated circuit
TWI453436B (en) * 2012-05-04 2014-09-21 Raydium Semiconductor Corp Ic reliability testing method
CN103383429B (en) * 2012-05-04 2015-10-28 瑞鼎科技股份有限公司 Method for testing reliability of integrated circuit
CN112563152A (en) * 2019-09-25 2021-03-26 长鑫存储技术有限公司 Manufacturing method and system
CN112563152B (en) * 2019-09-25 2023-01-31 长鑫存储技术有限公司 Manufacturing method and system

Also Published As

Publication number Publication date
TWI427487B (en) 2014-02-21

Similar Documents

Publication Publication Date Title
TWI539298B (en) Metrology sampling method with sampling rate decision scheme and computer program product thereof
TWI521360B (en) Metrology sampling method and computer program product thereof
Coble et al. Identifying optimal prognostic parameters from data: a genetic algorithms approach
US20220163320A1 (en) Monitoring system and method for verifying measurements in pattened structures
TWI315054B (en) Method for evaluating reliance level of a virtual metrology system in product manufacturing
TW201945923A (en) Integrated circuit profiling and anomaly detection
US8650137B2 (en) Method and apparatus for creating state estimation models in machine condition monitoring
TW200949596A (en) Server and system and method for automatic virtual metrology
US11836617B2 (en) Techniques for analytical instrument performance diagnostics
White et al. Methodological tools
JP2020061575A (en) System for determining critical parameters using high-dimensional variable selection model
CN113484813B (en) Intelligent ammeter fault rate prediction method and system under multi-environment stress
KR101953558B1 (en) Apparatus and Method for Fault Management of Smart Devices
CN101118422A (en) Virtual measurement prediction generated by semi-conductor, method for establishing prediction model and system
TW201135474A (en) Method for sampling workpiece for inspection and computer program product performing the same
CN117556331B (en) AI-enhancement-based air compressor maintenance decision method and system
Song et al. Fiducial inference-based failure mechanism consistency analysis for accelerated life and degradation tests
CN113742248A (en) Method and system for predicting organization process based on project measurement data
CN107850607A (en) Detect the transient error in body fluid sample
CN109213119B (en) Complex industry key component fault prediction method and system based on online learning
KR20170018794A (en) Selection system and method of software reliability evaluation model using partial failure data
US11847447B2 (en) Anomaly identification within software project under development
CN113887990A (en) Electrical equipment maintenance decision optimization method
Saxena et al. Realiability Assessment Model to Estimate Quality of the Effective E-Procurement Process in Adoption
Pan et al. Survey on Reliability Engineering for AI Software Systems: An Extension Based on the IEEE 1633 Standard