JP2011129580A - Defective device detection system, and defective device detection program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically adjust statistical parameters used for detecting a defect of a device. <P>SOLUTION: A latest P-value correction factor is calculated based upon browsing state data, including the number of times of browsing and the degree of process attention of an analysis result list picture showing analysis results associated with product IDs, specified by a user, by the process IDs, and a P-value correction factor. Then a P value is calculated for each combination of a process ID and a device ID to be contrasted and compared by applying merge data of inspection data on a product and device history data of a manufacturing device to a predetermined significant difference examination method, and a new P value is calculated based upon the P value and P-value correction factor. Then the new P value is compared with a predetermined significant level to perform device difference analysis, and a result thereof is displayed as an analysis result list picture. Statistics of the number of times of browsing and the degree of process attention of the analysis result list picture are taken by product IDs to generate and update the browsing state data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an abnormal device detection system and an abnormal device detection program for detecting an abnormal device from among a large number of devices used in a manufacturing factory for semiconductors and the like.

  In general, in a manufacturing apparatus variation analysis system in a semiconductor manufacturing factory, measurement data and apparatus processing history are acquired, a significant difference test is performed on these data, and a “risk rate (hereinafter referred to as“ P value ”)” is obtained. The engineer is alerted that the process / equipment is abnormal if the level is below a certain level of significance, and the engineer will follow the alert to determine whether the target process / equipment is actually abnormal. Investigation (see, for example, Patent Document 1).

JP 2003-289025 A

Hochberg, Y. and Benjamini, Y. (1995) “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing” Journal of the Royal Statistical Society. Series B (Methodological), Vol.57, No.1, 289 -300

  However, in general, there are few cases where the result of a significant difference test using a statistical method leads to the discovery of an actual abnormal process / device, and the detailed survey of engineers often ends up in trouble. This means that the cause of the abnormality in the manufacturing process is not always caused by a specific device. In other words, since it is not desirable to overlook an abnormality in the device, alerting many abnormalities with a low (high) significance level would be a waste of labor for engineers. On the other hand, if the significance level is too strict (low), an important device abnormality will be overlooked.

  In this way, the distribution of data and the number of manufacturing devices used within a certain period differ depending on the input amount of the product to be manufactured and the type of manufacturing process. In most cases, the significance level was set uniformly. Therefore, since the accuracy of the test varies depending on the target product / process, if the significance level is set to a constant value, an alarm is often issued for a certain product / process, but the alarm is completely different for another product / process. The phenomenon of not coming out occurs. In order to solve this problem, a method of fine-tuning the significance level for each product / process has been proposed (see Non-Patent Document 1). Setting the significance level manually for a process requires a lot of effort.

  In addition, production line conditions change with time, and there is no guarantee that the significance level once set is still optimal, and it is necessary to continue to adjust the significance level to improve accuracy. In addition, since the above-mentioned adjustment must be performed for different products and different processes, the load on operation is extremely large, and in fact, such a fine adjustment is hardly performed.

  SUMMARY OF THE INVENTION In view of the above-described problems of the prior art, an object of the present invention is to provide an abnormal device detection system and an abnormal device detection program capable of automatically adjusting statistical parameters used for detecting an abnormality of a device. .

  An abnormal device detection system according to the present invention includes an inspection data recording unit that records inspection data relating product IDs, lot IDs, and inspection results of products manufactured by a plurality of devices and processes, the product ID, and the lot. A device history data recording unit that records device history data that associates an ID, a process ID, a device ID, and a processing time, and obtains the inspection data and the device history data, and merges the product ID and the lot ID as a key. A merge data creation unit that creates data, and the number of times the analysis result list screen represents the analysis result related to the product ID specified by the user by a trend diagram and a box mustache diagram for each process ID, and the analysis result list screen The browsing status data that records the browsing status data including the process attention degree based on the number of selections of the process ID. A data recording unit, a P value correction factor recording unit that records a P value correction factor for each process ID, and calculating the latest P value correction factor based on the browsing status data and the P value correction factor. A P-value correction factor calculating unit for updating the P-value correction factor recording unit based on a P-value correction factor, and the process ID and the device to be compared with each other by applying the merge data to a predetermined significant difference test method A P value is calculated for each combination of IDs, and a new P value is calculated for each combination of the process ID and the device ID to be compared based on the P value and the P value correction factor. A device difference analysis unit that performs device difference analysis by comparing the P value with a predetermined significance level, displays the result of the device difference analysis as the analysis result list screen, and displays the analysis result of the user. An analysis result display unit that creates the browsing status data by statistically calculating the browsing count and the process attention degree related to the list screen for each product ID, and updates the browsing status data recording unit. To do.

  An abnormal device detection program according to the present invention includes an inspection data recording step for recording inspection data relating product IDs, lot IDs, and inspection results of products manufactured by a plurality of devices and processes in a storage device, and the product ID A device history data recording step for recording device history data relating the lot ID, process ID, device ID, and processing time in the storage device, obtaining the inspection data and the device history data, and obtaining the product ID and Merge data creation step for creating merge data using the lot ID as a key, and the number of times the analysis result list screen is displayed for each process ID showing the analysis result related to the product ID as a trend diagram and a box mustache diagram And process attention based on the number of selections of the process ID on the analysis result list screen A browsing status data recording step for recording browsing status data including: a P value correction factor recording step for recording a P value correction factor for each of the process IDs; and the latest based on the browsing status data and the P value correction factor A P-value correction factor calculating step of updating the P-value correction factor recording unit based on the latest P-value correction factor, and applying the merged data to a predetermined significance test A P value calculating step for calculating a P value for each combination of the process ID and the device ID to be compared, and the process ID and the device ID to be compared with each other based on the P value and the P value correction factor. A device difference analysis step of calculating a new P value for each combination and comparing the new P value with a predetermined significance level to perform device difference analysis; A screen display step for displaying the analysis result as the analysis result list screen, and creating the browsing status data by statistically calculating the number of times of browsing and the process attention degree related to the analysis result list screen of the user for each product ID And a browsing status data creating step for updating the browsing status data recording unit.

  According to the present invention, an abnormal device detection system and an abnormal device detection program capable of automatically adjusting statistical parameters used for detecting an abnormality of the device are provided.

1 is a block diagram showing an example of the overall configuration of an abnormal device detection system according to an embodiment of the present invention. The figure which shows the specific example of the test | inspection data recorded on the test | inspection data recording part of FIG. The figure which shows the specific example of the test | inspection data recorded on the apparatus historical data recording part of FIG. The figure which shows the specific example of the data merge in the merge data preparation part of FIG. The figure which shows the specific example of the browsing condition data recorded on the browsing condition data recording part of FIG. The figure which shows the specific example of the P value correction factor recorded on the P value correction factor recording part of FIG. The figure which shows the specific example of the analysis result recorded on the analysis result recording part of FIG. The figure which shows the hardware structural example applied to the abnormal device detection system which concerns on one Embodiment of this invention. The flowchart which shows the specific example of the process of the abnormal device detection system which concerns on one Embodiment of this invention. The flowchart which shows the specific example of the process in the P value correction factor calculation part of FIG. The flowchart which shows the specific example of the process in the apparatus difference analysis part of FIG. The flowchart which shows the specific example of the process performed after the process of above-mentioned FIG. 11 among the processes in the apparatus difference analysis part of FIG. The flowchart which shows the specific example of the process in the analysis result display part of FIG. The figure which shows the specific example of the analysis result list screen output by the analysis result display part of FIG. The figure which shows the specific example of the detailed result screen output by the analysis result display part of FIG. The figure which shows the specific example of the weight set for every user regarding attention degree calculation.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing an example of the overall configuration of an abnormal device detection system 1 according to an embodiment of the present invention. The abnormal device detection system 1 is a computer system that automatically detects an abnormality of a manufacturing device in a semiconductor manufacturing process, and includes an inspection data recording unit 101, an apparatus history data recording unit 102, a merge data creation unit 103, and browsing status data. A recording unit 104, a P value correction factor recording unit 105, a P value correction factor calculation unit 106, an apparatus difference analysis unit 107, an analysis result recording unit 108, and an analysis result display unit 109 are provided.

  The inspection data recording unit 101 is a storage device that records inspection data relating product IDs, lot IDs, and inspection results of products manufactured by a plurality of devices and processes. FIG. 2 is a diagram showing a specific example of inspection data recorded in the inspection data recording unit 101 of FIG.

  The device history data recording unit 102 is a storage device that records device history data that associates a product ID, a lot ID, a process ID, a device ID, and a processing time. FIG. 3 is a diagram showing a specific example of the inspection data recorded in the apparatus history data recording unit 102 of FIG.

  The merge data creation unit 103 is a program that acquires inspection data and apparatus history data, and creates merge data of inspection data and apparatus history data using the product ID and lot ID as keys. Here, products, processes, and periods for which data is to be acquired are set in advance, and inspection data (process QC data, device data, etc.) and device history data (device ID, processing time, etc.) are set according to the settings. It is automatically acquired, and both data are linked using the product ID and lot ID as keys to generate one data table. FIG. 4 is a diagram showing a specific example of a data table created by the merge data creation unit 103 of FIG. By referring to this data, it is possible to grasp which device is used in the manufacturing process of the lot of the target product and whether the quality of the lot is appropriate.

  The browsing status data recording unit 104 is a storage device that records user browsing status data for a certain period. This browsing status data includes the number N of browsing of an analysis result list screen, which will be described later, the number of process ID selections on the analysis result list screen, and the process attention degree S based on the questionnaire result on the detailed result screen. FIG. 5 is a diagram showing a specific example of browsing status data recorded in the browsing status data recording unit 104 of FIG. FIG. 5 (a) shows the relationship between the product ID and the number of browsing times N on the analysis result list screen. FIG. 5B shows the relationship between the product ID, the process ID, and the process attention degree S on the analysis result list screen.

  The P value correction factor recording unit 105 is a storage device that records a P value correction factor γ for correcting the P value (risk rate) for each process ID of the target product. In this system, the P value correction factor γ is sequentially corrected from a preset initial value based on the viewing status of the user. FIG. 6 is a diagram showing a specific example of the P value correction factor recorded in the P value correction factor recording unit 105 of FIG. Here, the relationship between the product ID, the process ID, and the P value correction factor γ is shown.

  The P value correction factor calculation unit 106 updates the latest P based on the merge data created from the inspection data and the device history data in the merge data creation unit 103, the browsing status data for a certain period, and the recorded P value correction factor γ. This program calculates the value correction factor γ ′ and updates the P value correction factor γ recorded in the P value correction factor recording unit 105 based on the latest P value correction factor γ ′.

  The apparatus difference analysis unit 107 applies merge data obtained by connecting inspection data and apparatus history data to a predetermined analysis of variance method and multiple test method, and adds P for each combination of the process ID of the target product and the apparatus ID for pairwise comparison. In addition to calculating the value, a new P value (hereinafter referred to as “P ′ value”) is set for each combination of the process ID and the device ID for pair comparison based on the P value and the latest P value correction factor γ ′. Is calculated, and a device level analysis is performed by comparing a new significance level α ′ obtained by correcting a predetermined significance level α (for example, α = 0.05) by a predetermined correction method with the calculated P ′ value. It is.

  Here, a device difference analysis is performed using a statistical method from the data table (merge data) generated by the merge data creation unit 103, and the P value obtained as a result is calculated by the P value correction factor calculation unit 106. The P ′ value is calculated by multiplying the latest P value correction factor γ ′, and if this P ′ value falls below the significance level α ′, the process ID, the combination of the device IDs for which the comparison is performed, and P ′ The value is recorded in the analysis result recording unit 108 as the analysis result of the target product (abnormal device candidate). Further, using the method described later, the only abnormal device in the process is specified, and the process ID, the device ID of the specified abnormal device, and the process-specific P ′ value (average value) are analyzed (the abnormal device for each process). As a list) in the analysis result recording unit 108. In addition, when device difference analysis is performed using multiple testing, the probability (Familywise Error Rate) of erroneously determining that there is a device difference increases as the number of target devices increases. As a model for correcting this, Benjamini & Hochberg (1995) has proposed a method of correcting the significance level based on the Fail Discovery Rate (FDR) (hereinafter referred to as “BH method”). In the present embodiment, based on the already-known BH method, a method for incorporating the inherent variation of each process apparatus and the knowledge of engineers into a model is proposed.

  The analysis result recording unit 108 is a storage device that records the analysis result in the device difference analysis unit 107. FIG. 7 is a diagram showing a specific example of analysis results (abnormal device candidate list and abnormal device list for each process) recorded in the analysis result recording unit 108 of FIG. In the abnormal device candidate list, the relationship between the process ID, the combination of the device ID, and the new P value (P ′ value) is shown, and the relationship between the device ID-1 and the device ID-2 is unique within each process. Combination. For example, when the significance level is 0.05, the analysis result of step A indicates that EQP1, EQP2, EQP3, EQP4, and EQP5 are candidates for abnormal devices. From this abnormal device candidate list, a method that will be described later is used to obtain the only statistically most likely abnormal device. That is, the appearance frequency of the device ID appearing in the column of device ID-1 and device ID-2 is obtained from the abnormal device candidate list, and the device having the highest appearance frequency is specified as the abnormal device in the target process. If there are a plurality of devices having the same appearance frequency, the average value of the P ′ values is obtained for each device ID, and the device having the smallest average value is specified as the only abnormal device. By this process, one abnormal device is specified for each process, and a P ′ value (average) unique to the process is obtained. The obtained process ID, apparatus ID, and P ′ value (average) are stored in the abnormal apparatus list for each process.

  Specifically, taking FIG. 7 as an example, in step A, EQP2 appears twice, and EQP2 has the highest appearance frequency. That is, EQP2 recognizes that it is the only abnormal device in process A. Here, as indicated by the arrow in FIG. 7, the average of the P ′ values of the records including EQP2 is (0.001 + 0.002) /2=0.015. Therefore, the abnormal device in step A is EQP2, and its P ′ value (average) is 0.0015.

  In this embodiment, a method for obtaining one abnormal device for each process is used. However, there may be a plurality of abnormal devices in practice. However, in the present embodiment, the purpose is to provide information that is narrowed down to the analysis engineer as much as possible, so that only one abnormal device is shown on the system side. In an analysis result display unit 109 to be described later, other abnormal candidate devices may be displayed in different colors as additional information to the analysis engineer.

  The analysis result display unit 109 reads the analysis result from the analysis result recording unit 108, displays an analysis result list screen including a trend diagram and a box mustache diagram in each process, and the number of times the analysis result list screen is viewed, the analysis result list screen The number of process ID selections and the answer to the questionnaire on the detailed result screen are statistically created for each product ID to create browsing status data, and the browsing status data recording unit 104 is updated.

  The analysis result display unit 109 displays a detailed result screen with the process ID as a key when the user selects a process to be noticed interactively based on the analysis result list screen, and based on the number of process selections and the questionnaire result. The browsing status data recording unit 104 is updated with the browsing status data of the calculated process attention degree (detailed result).

  FIG. 8 is a diagram illustrating a configuration example of a computer applied to the abnormal device detection system 1 according to an embodiment of the present invention. As shown in the figure, a computer applied to the abnormal device detection system 1 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an input / output interface 204, and a system. A bus 205, an input device 206, a display device 207, an auxiliary storage device 208, and a communication device 209 are included.

  The CPU 201 is a processing device that executes arithmetic processing using various programs and data related to abnormal device detection recorded in the ROM 202 and RAM 203. The ROM 202 is a read-only storage device that stores basic programs and environment files for causing the computer to function. A RAM 203 is a storage device that stores a program executed by the CPU 201 and data necessary for execution of each program, and can be read and written at high speed. The input / output interface 204 is a device and a program that mediate connections between various hardware and the system bus 205. A system bus 205 is an information transmission path shared by the CPU 201, ROM 202, RAM 203, and input / output interface 204.

  The input / output interface 204 is connected to hardware such as an input device 206, a display device 207, an auxiliary storage device 208, and a communication device 209. The input device 206 is a device that processes input from the user, and is, for example, a keyboard or a mouse. The display device 207 is a device that displays a calculation result, a creation screen, and the like to the user, and is, for example, a CRT, a liquid crystal display, a plasma display, or the like. The auxiliary storage device 208 is a large-capacity storage device that stores programs and data, and is, for example, a hard disk device.

  Hereinafter, the operation of the abnormal device detection system 1 configured as described above will be described with reference to the drawings. FIG. 9 is a flowchart showing a specific example of processing of the abnormal device detection system 1 according to an embodiment of the present invention.

In step S901, the merge data creation unit 103 acquires target inspection data, process ID, device ID, and processing time used for each lot processing from the corresponding database, and merges both data using the lot ID as a key.
In step S <b> 902, the P value correction factor calculation unit 106 determines whether there is an unprocessed process among the manufacturing processes of the target product. If it is determined that there is an unprocessed process, the process proceeds to S903. On the other hand, if it is determined that there is no unprocessed process, the process proceeds to S905.

In step S903, the P value correction factor calculation unit 106 acquires browsing status data related to the target process ID from the browsing status data recording unit 104, and calculates a P value correction factor γ ′ using the data.
In step S904, the apparatus difference analysis unit 107 performs a significant difference test using the merged data, corrects the P value obtained as a result of the test using the latest P value correction factor γ ′, and creates a new P value (P ′ Device difference analysis is performed by comparing the new P value (P ′ value) with a new significance level α ′ obtained by correcting the predetermined significance level α by the BH method, and the analysis result is an analysis result recording unit. After recording in 108, the process returns to S902, and S902 to S904 are repeated until the processing is completed for all steps.
In step S <b> 905, the analysis result display unit 109 acquires the analysis result from the analysis result recording unit 108, displays an analysis result list screen in which time-series trend diagrams and box-whisker diagrams are arranged horizontally for each process, and ends the process. .

FIG. 10 is a flowchart illustrating a specific example of processing in the P-value correction factor calculation unit 106 in FIG. 1, and corresponds to S903 in FIG.
In step S1001, the P value correction factor γ _i of the target process _i is acquired from the P value correction factor recording unit 105.
In S1002, it is determined whether or not there is a P value correction factor γ _i for the target process i. If it is determined that there is a P value correction factor γ _i , the process proceeds to S1003. On the other hand, if it is determined that there is no P value correction factor γ _i , the process proceeds to S1007.

In S1003, browsing state data including the number N of browsing of the analysis result list screen, the number of times the user has selected the target process i from the list within a certain period, and the process attention degree S _i (S _i ≧ 0) based on the questionnaire result. Obtained from the browsing status data recording unit 104.
In S1004, the presence / absence of a list display is determined from the obtained browsing status data. If it is determined that there is a list display (N> 0), the process proceeds to S1005. On the other hand, if it is determined that there is no list display (N = 0), it is unnecessary to update the P value correction factor γ _i , and the process ends.

In S1005, the latest P value correction factor γ _i ′ is calculated from the number N of browsing times obtained in S1003, the degree of process attention S _i, and a preset reference value δ (0 <δ <1.0). Here, the latest P value correction factor γ _i ′ is calculated from the following equation (1).
γ _i ′ = γ _i × (1 + δ × S _i / N) Expression (1)
However, P ′ = P _max (when P ′> P _max )
P ′ = P _min (when P ′ <P _min )
In S1006, the latest P value calculated by the correction factor gamma _i 'to update the P value correction factor gamma _i of P value correction factor recording unit 105, and ends the process.
In step S1007, the P value correction factor γ _i is newly set to 1.0 in the P value correction factor recording unit 105, and the process ends.

FIG. 11 is a flowchart showing a specific example of processing in the apparatus difference analysis unit 107 in FIG. 1, and corresponds to S904 in FIG.
In S1101, outlier removal of the inspection data of the target process i is performed.
In S1102, in order to confirm the normality of the inspection data of the target process i, the “distortion” and “kurtosis” of the data are calculated.

  In S1103, the calculated skewness and kurtosis are respectively compared with preset threshold values to determine whether or not there is normality. If it is determined that the skewness and kurtosis are within the threshold value range and there is normality, the process advances to step S1104. On the other hand, if the skewness and kurtosis exceed the threshold and it is determined that there is no normality, the process proceeds to S1105.

In step S1104, an average value of inspection data among all apparatuses related to the target process i is obtained using parametric analysis.
In step S1105, an average value of inspection data among all apparatuses related to the target process i is obtained using non-parametric analysis.

In S1106, it is verified whether or not there is a difference in the average value of the inspection data among all apparatuses related to the target process i. If it is determined that there is a difference between the average values, the process proceeds to S1107. On the other hand, if it is determined that there is no difference in the average value, the process proceeds to S1109.
In S1107, multiple tests between individual devices are performed, and a P value is obtained for each device combination related to the target process i.

In S1108, a P ′ value is calculated for each combination of devices related to the target process i by multiplying the P value obtained by the multiple test by the P value correction factor γ _i related to the target process i.
In S1109, the P ′ value is set to 1.0.
In S1110, a predetermined significance level α is corrected based on the BH method to obtain a new significance level α ′.

In S1111, the presence / absence of device difference is determined by comparing the P ′ value obtained in S1108 with the significance level α ′ obtained by the correction in S1110. Here, if the P ′ value is below the significance level α ′ and it is determined that there is a device difference, the process proceeds to S1112. On the other hand, if the P ′ value is greater than or equal to the significance level α ′ and it is determined that there is no device difference, the process is terminated.
In S1112, by rejecting the null hypothesis that “there is no difference between devices”, the combination of the device ID determined to have a device difference and the P ′ value are recorded in the analysis result recording unit 108, and the process ends.

FIG. 12 is a flowchart showing a specific example of processing executed after the processing of FIG. 11 described above in the processing in the apparatus difference analysis unit 107 of FIG. 1, and corresponds to S904 of FIG. 9 as described above.
In S1201, the abnormal device candidate list recorded in the process shown in FIG. 11 is read.
In S1202, the presence / absence of an unprocessed process in the list read in S1201 is determined. If it is determined that there is an unprocessed process, the process proceeds to S1203. On the other hand, if it is determined that there is no unprocessed process, the process proceeds to S1209.

In step S1203, the abnormality candidate list read in step S1201 is narrowed down to the process to be processed this time.
In S1204, all the columns of device ID-1 and device ID-2 shown in the example of FIG. 7 are searched from the list narrowed down in S1203, and the appearance frequency for each device ID in this list is obtained.

  In S1205, the device having the highest appearance frequency is obtained from the appearance frequency for each device ID obtained above, and the list narrowed down in S1203 is further narrowed down using the condition that only the device is concerned. When there are a plurality of devices having the maximum appearance frequency, all devices having the maximum appearance frequency are narrowed down to the target.

In S1206, an average value of P ′ values is obtained for the list narrowed down in S1205. When there are a plurality of narrowed down devices, an average value is obtained for each device ID.
In S1207, the device ID that minimizes the average value obtained in S1206 is extracted.
In step S1208, the target process ID, the extracted device ID, and the P ′ value (average) are stored as one record in a table (result array) on the memory. S1202 to S1208 are repeated until the processing is completed for all steps. By repeating this process, the table on the memory has the number of records corresponding to the number of steps.

In step S1209, the combination list of the process ID, device ID, and P ′ value (average) stored in the table (result array) on the memory in step S1208 is rearranged in order of increasing P ′ value (average).
In S1210, the list (result array) of the process ID, device ID, and P ′ value (average) rearranged in S1209 is recorded in the analysis result recording unit.

FIG. 13 is a flowchart showing a specific example of processing in the analysis result display unit 109 of FIG. 1, and corresponds to S905 of FIG.
In step S1301, it is determined whether there is an analysis result display request from the user. If it is determined that there is a display request, the process proceeds to S1302. On the other hand, if it is determined that there is no display request, the system waits until a display request is made.
In S1302, a processing request is made to the apparatus difference analysis unit 107, and the latest analysis result is acquired from the analysis result recording unit.

  In S1303, an analysis result list screen is created and displayed based on the acquired analysis results. FIG. 14 is a diagram showing a specific example of an analysis result list screen output by the analysis result display unit 109 of FIG. In this screen, process IDs are arranged vertically in order of increasing P ′ value (average) using the analysis results shown in the list of abnormal devices for each process in FIG. In addition, a time-series trend diagram and a box mustache diagram divided on the basis of the device ID are displayed side by side as minimum information for the user to roughly grasp whether the device is abnormal or not. . In the trend diagram and the box mustache diagram, it is preferable to display abnormal devices in different colors (such as hatching) because it is easy to grasp the abnormal devices. Furthermore, event information such as the maintenance time of the apparatus may be described as additional information in the trend diagram according to the user's needs.

  In S1304, the browsing status data acquired from the browsing status data recording unit 104 is incremented by 1 to create new browsing status data, and the browsing status data recording unit 104 is updated with this data. That is, when one user browses the analysis result list screen, the analysis result display unit 109 increments N, which represents the number of times the user browsed the list of abnormal devices within a certain period. However, if the same person displays a list many times in a short time, it is handled as one time (for example, managing a WEB session). The “short time” in the present embodiment assumes several hours, but the setting can be arbitrarily changed.

  In step S1305, it is determined whether the user has selected a process while the analysis result list screen is displayed. In the case of the analysis result list screen shown in FIG. 14, it is determined whether or not the display portion of each process has been clicked. If it is determined that a process is selected, the process proceeds to S1306. On the other hand, if it is determined that there is no process selection during the display of the analysis result list screen, the process ends.

  In S1306, a detailed result is acquired using the process ID of the selected process as a key, and a detailed result screen is created and displayed. FIG. 15 is a diagram showing a specific example of a detailed result screen output by the analysis result display unit 109 of FIG. Although a rough result is displayed in a reduced scale on the analysis result list screen, a lot of information necessary for the user to grasp the abnormal device is added to the detailed result screen. In addition, in the figure, as additional information, a maintenance chart for each apparatus and a recipe change time are shown in the trend diagram portion, and one point indicates one lot of inspection data. When the mouse cursor is brought close to a point on the graph, lot attribute information (lot ID, event generated by lot processing, etc.) is displayed in a pop-up window. In addition, an average value and a standard deviation are added to the box mustache diagram. When the mouse cursor is brought close to the box portion of the box mustache, detailed information (device ID, device type, manufacturer, recipe, etc.) for the device is displayed in a pop-up window. In addition, the upper limit value (USL) and the lower limit value (LSL) of the standard are displayed in the trend diagram and the box mustache diagram, respectively. In both figures, information useful for grasping the state of the apparatus can be further added according to the needs of the user. For example, the maximum value / minimum value, the number of processing lots from the previous maintenance, and the mask used in the case of a lithography apparatus. Also, at the end of the detailed result screen, a questionnaire “is this result useful?” Is added.

In step S1307, the user's answer to the questionnaire included in the detailed result screen is acquired.
In S1308, the process attention degree S _i representing the degree of attention to the user's process i within a certain period is determined based on the number of user process selections on the analysis result list screen and the weights x and y defined in advance for each user. The browsing status data of the browsing status data recording unit 104 is calculated and updated. Here, when displaying the detailed results screen of Fig. 15, first, the process saliency S _i +1. Further, the answer to this questionnaire if "yes" in the x added to S _i, in the case of "No", calculates the process attention S _i by subtracting y from S _i. x and y are fixed values set in advance such that x ≧ 1 and y ≧ 2.

  However, when a user with high skill level clicks the “Yes” or “No” button and when a user with low skill level clicks in the same way, the weight of the result may be different. In other words, a response from a low-skilled user becomes “noise” on the contrary, which may reduce the detection accuracy of the apparatus. Therefore, in this embodiment, x and y are set for each user in order to give more weight to responses from highly skilled users. In this case, it is necessary to create a weighting definition table in which x and y are described for each user ID in advance.

FIG. 16 is a diagram illustrating a specific example of the weighting definition table. For users with a high level of skill, x and y are set large, and for users with a low level of skill, x and y are set small. The user logs into the system using the user ID given individually. The system keeps track of the user ID used at the time of login. When the “Yes” or “No” button is clicked, the system refers to the above-described weighting definition table and uses x and y corresponding to the user ID. The attention degree S _i is calculated. As a result of this operation, the process that the user pays attention to increases the S _i because the frequency of clicks is high, so that P ′ decreases. Therefore, the detection sensitivity of the apparatus difference is increased, and the order of the list display is shifted to the top. Conversely, in the process that the user does not look at, P _i increases because Si decreases. Accordingly, the detection sensitivity of the apparatus difference is lowered, and the order of list display is also shifted downward. It should be noted that the process attention degree can be calculated from only the number of process selections without using the weighting defined for each user.

  In S1309, the apparatus difference analysis unit 107 is requested to perform analysis processing based on the fed-back browsing status data, and an analysis result list screen is created and displayed from the analysis result, and the process returns to S1305. That is, when either “Yes” or “No” is clicked, the screen returns to the analysis result list screen and the analysis result is referred to. Note that the processing of S1305 to S1309 is repeated until the analysis result list screen for the product is closed.

  As described above, in the conventional example, the presence / absence of significant difference is determined by directly comparing the significance level α determined in advance for each device type with the P value obtained as a result of the statistical test. In the abnormal device detection system 1 according to the present embodiment, the accuracy is automatically improved by correcting the statistically calculated P value in accordance with the user's browsing situation obtained from the analysis result display unit 109. Can be improved. In other words, it is the significance level α used to determine the presence / absence of a significant difference that greatly affects the accuracy of the significant difference test, but the preset significance level α as in the prior art is considered by engineers. If the number of processes reaches several hundreds, it is practically difficult for an engineer to set an optimal α for each process. In contrast, in the abnormal device detection system 1 according to the present embodiment, “correction of the significance level α” can be realized in the form of “correction of the P value”. Therefore, even if an appropriate significance level α is initially set, the P value is “automatically” gradually adjusted to the optimum value by the use of the user. It is possible to reduce the manpower required for adjusting the statistical parameter (P value) and to shorten the time required for the adjustment. Moreover, the abnormality detection accuracy of the manufacturing apparatus can be greatly improved by adjusting the verification parameter for each product / process.

  Furthermore, in the conventional method of detecting an abnormal device using a statistical method, as described above, the knowledge of an engineer (for example, information on differences in sensitivity for each process or type of device) is detected. Incorporation into the system could only be realized by the engineer fine-tuning the significance level α based on intuition and experience, but in the abnormal device detection system 1 according to the present embodiment, “engineer knowledge (intuition and experience) ) ”Can be incorporated into the test model by replacing“ Web usage frequency ”.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment.

DESCRIPTION OF SYMBOLS 1 ... Abnormal apparatus detection system 101 ... Inspection data recording part 102 ... Apparatus history data recording part 103 ... Merge data creation part 104 ... Browsing situation data recording part 105 ... P value correction factor recording part 106 ... P value correction factor calculation part 107 ... Device difference analysis unit 108 ... analysis result recording unit 109 ... analysis result display unit 201 ... CPU
202 ... ROM
203 ... RAM
204 ... Input / output interface 205 ... System bus 206 ... Input device 207 ... Display device 208 ... Auxiliary storage device 209 ... Communication device

Claims (5)

An inspection data recording unit that records inspection data relating product IDs, lot IDs, and inspection results of products manufactured by a plurality of apparatuses and processes;
A device history data recording unit for recording device history data relating the product ID, the lot ID, the process ID, the device ID, and the processing time;
A merge data creation unit that obtains the inspection data and the device history data, and creates merge data using the product ID and the lot ID as a key;
Process attention based on the number of browsing of the analysis result list screen that represents the analysis result related to the product ID specified by the user by a trend diagram and a box mustache diagram for each process ID, and the number of selections of the process ID on the analysis result list screen A browsing status data recording unit that records browsing status data including the degree,
A P value correction factor recording unit for recording a P value correction factor for each process ID;
A P value correction factor calculating unit that calculates the latest P value correction factor based on the browsing status data and the P value correction factor, and updates the P value correction factor recording unit based on the latest P value correction factor; ,
The merge data is applied to a predetermined significance test to calculate a P value for each combination of the process ID and the device ID to be compared, and the process is performed based on the P value and the P value correction factor. A device difference analysis unit that calculates a new P value for each combination of ID and the device ID to be compared, and compares the new P value with a predetermined significance level to perform device difference analysis;
The result of the apparatus difference analysis is displayed as the analysis result list screen, and the browsing status data is created by statistically analyzing the number of times of browsing and the process attention degree of the user for each product ID. And an analysis result display unit for updating the browsing status data recording unit,
An abnormal device detection system comprising:   The analysis result display unit further displays a detailed result screen including a detailed analysis result of each device related to the process ID selected by the user on the analysis result list screen and a questionnaire input component for the analysis result, and The abnormal device detection system according to claim 1, wherein the attention degree of the process is calculated based on the number of selections and the result of the questionnaire, and is recorded in the browsing status data recording unit as a part of the browsing status data.   The abnormal device detection system according to claim 2, wherein the analysis result display unit uses a weight set in advance for each user for calculation of the process attention degree.   The apparatus difference analysis unit performs apparatus difference analysis by comparing a new significance level obtained by correcting a predetermined significance level by a predetermined significance level correction method and the new P value. The abnormal device detection system according to claim 3. An inspection data recording step for recording inspection data relating product IDs, lot IDs and inspection results of products manufactured by a plurality of devices and processes in a storage device;
A device history data recording step of recording device history data in which the product ID, the lot ID, the process ID, the device ID, and the processing time are related to the storage device;
A merge data creation step of acquiring the inspection data and the device history data, and creating merge data using the product ID and the lot ID as a key;
Process attention based on the number of browsing of the analysis result list screen that represents the analysis result related to the product ID specified by the user by a trend diagram and a box mustache diagram for each process ID, and the number of selections of the process ID on the analysis result list screen Browsing status data recording step for recording browsing status data including the degree,
A P value correction factor recording step for recording a P value correction factor for each process ID;
A P value correction factor calculating step of calculating a latest P value correction factor based on the browsing status data and the P value correction factor, and updating the P value correction factor recording unit based on the latest P value correction factor; ,
A P value calculating step of calculating a P value for each combination of the process ID and the device ID to be compared by applying the merge data to a predetermined significant difference test method;
Based on the P value and the P value correction factor, a new P value is calculated for each combination of the process ID and the device ID to be compared, and the new P value is compared with a predetermined significance level. A device difference analysis step for performing a difference analysis;
A screen display step for displaying the result of the apparatus difference analysis as the analysis result list screen;
A browsing status data creation step of statistically generating the browsing status data for each product ID, and updating the browsing status data recording unit; ,
An abnormal device detection program for causing a computer to execute.

Images