KR102670325B1 - Method and system for root cause analysis of deviation associated with equipment - Google Patents

Description

Method and system for root cause analysis of equipment-related deviations {METHOD AND SYSTEM FOR ROOT CAUSE ANALYSIS OF DEVIATION ASSOCIATED WITH EQUIPMENT}

The present invention relates to root cause analysis, and more specifically to mechanisms for root cause analysis of equipment-related deviations.

Within a fish facility/manufacturing facility, many operations are controlled by various inputs (e.g. data) generated from sensors installed in process equipment and building facilities. This data includes both structured and unstructured formats. These tasks can perform a number of functions, including process optimization and identification of abnormal events and disruptions in the operations of a production/manufacturing facility.

Within a production/manufacturing facility, events and disruptions of various scales continually impact process operations. Events and disturbances are addressed by root cause analysis of process-related deviations. Currently, these events and disturbances are handled by a variety of systems and methods. However, while conventional systems and methods are somewhat effective in root cause analysis of deviations related to equipment, memory power, loss of contextual information due to disconnected data storage, mining of structured data, and unstructured data. It includes both advantages and disadvantages in terms of mining raw data, prioritization of resolution actions, consideration of equipment state changes, cost, accuracy and rapidity.

Therefore, there remains a need for robust systems and methods for root cause analysis of equipment-related deviations.

The technical problem that the present invention seeks to solve is to provide a method for analyzing the root cause of equipment-related deviations.

Another technical problem that the present invention aims to solve is to provide a mechanism for detecting equipment-related deviations.

Another technical problem to be solved by the present invention is to provide a mechanism for determining a Deviation Resolution sub-Graph (DRsG) for resolving deviations based on a Deviation Resolution Graph (DRG).

Another technical problem that the present invention aims to solve is to provide a mechanism for recommending DRsG to resolve equipment-related deviations.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

A system for analyzing the root cause of deviations related to equipment according to an embodiment of the present invention for achieving the above technical problem includes a Sensor Resolution Action (SeRA) entity that detects deviations related to at least one equipment, A DRG entity that resolves deviations based on at least one Deviation Resolution Graph (DRG) by determining at least one Deviation Resolution sub-Graph (DRsG), wherein the at least one DRsG is at least one A DRG entity comprising a resolution action and at least one observation associated with the at least one piece of equipment, and recommending the at least one DRsG to resolve the deviation associated with the at least one piece of equipment. Contains entities.

In some embodiments of the invention, the observation is an original observation.

In some embodiments of the invention, the observation is a secondary observation.

In some embodiments of the invention, the remedial action includes a repair action.

In some embodiments of the invention, the solution task includes a secondary task.

In some embodiments of the invention, the solving task includes a solving task.

In some embodiments of the invention, the solution operation includes a reset operation.

In some embodiments of the invention, the DRG is formed based on data received from multiple sources. In an embodiment of the invention, the data is structured data. In an embodiment of the invention, the data is unstructured data.

In some embodiments of the invention, the deviation is identified based on a SeRA graph. The SeRA graph shows the relationship between equipment groups and solving tasks.

In some embodiments of the invention, the SeRA graph is formed based on a before-and-after analysis of a set of DRsGs of the device. The before-and-after analysis identifies equipment containing signatures of repair operations, creating causal relationships that are stored for automated recommendations by the DRsG.

In some embodiments of the invention, the back-and-forth analysis engine calculates a deviation score based on whether the time series signal of the at least one device includes a change point. The confidence engine determines a confidence score based on the deviation score. The fuzzy score globalizer determines a deviation vector based on the scores. The sensor selector establishes a causal relationship between the deviation and the equipment based on the SeRA graph.

In some embodiments of the invention, if the change point is not available in the time series signal, Window-let Dynamic Time Warping (WDTW) is performed to change the deviation score of the time series signal of the equipment. Calculate .

In some embodiments of the invention, if the change point is available in the time series signal, Window-let Gamma (WGamma) is performed to calculate the deviation score of the time series signal of the equipment.

In some embodiments of the invention, if the change point is available in the time series signal, Fisher discriminant index (FDI) and Dynamic Time Warping (DWT) are performed to determine the time series signal of the equipment. Calculate the deviation score of .

In some embodiments of the invention, it is calculated based on the long term trend and short term aging trend.

In some embodiments of the present invention, the DRsG is triggered by the sensor selector when the deviation vector satisfies a threshold.

In some embodiments of the invention, the time series signal is obtained from video data of the at least one device.

In some embodiments of the invention, the SeRA entity provides at least one recommendation for a deviation of the at least one equipment based on the video data.

A method for analyzing the root cause of deviations related to equipment according to an embodiment of the present invention for achieving the above technical problem includes detecting deviations related to at least one equipment by a Sensor Resolution Action (SeRA) entity. Detecting and, by a Deviation Resolution Graph (DRG) entity, determining at least one Deviation Resolution sub-Graph (DRsG) to resolve the deviation based on at least one DRG, At least one DRsG includes at least one solution task and at least one observation associated with the at least one piece of equipment, and a recommendation entity recommends the at least one DRsG to resolve the deviation associated with the at least one piece of equipment. It includes doing.

A computer program product comprising computer-executable program code recorded on a computer-readable non-transitory storage medium for analyzing the root cause of deviations related to equipment according to an embodiment of the present invention for achieving the above technical problem, By means of a Sensor Resolution Action (SeRA) entity, a deviation associated with at least one piece of equipment is detected, and by a Deviation Resolution Graph (DRG) entity, at least one Deviation Resolution subgraph (Deviation Resolution subgraph) is detected. -Graph; DRsG) to resolve the deviation based on at least one DRG, wherein the at least one DRsG includes at least one solution task and at least one observation related to the at least one device, and a recommendation entity. and recommending the at least one DRsG to resolve the deviation associated with the at least one equipment.

Embodiments of the present invention will be better appreciated and understood when considered in conjunction with the following detailed description and accompanying drawings. However, the following detailed description, which refers to preferred embodiments and their numerous specific details, is intended to illustrate the invention and is not intended to limit it. Many changes and modifications can be made to the embodiments of the invention without departing from the spirit of the invention, and the embodiments described herein include all such changes.

1 is a schematic diagram of a system for root cause analysis of equipment-related deviations according to an embodiment of the present invention.
Figure 2 is a diagram showing different objects according to properties according to an embodiment of the present invention.
Figure 3 shows a deviation resolution graph according to an embodiment of the present invention, where "P" represents PASS and "F" represents FAIL.
Figure 4 shows a deviation resolution subgraph according to an embodiment of the present invention.
Figure 5 shows traversing of a deviation resolution subgraph according to an embodiment of the present invention.
Figure 6 is a flowchart illustrating a method for analyzing the root cause of equipment-related deviations according to an embodiment of the present invention.
Figure 7 shows a time series with change points according to an embodiment of the present invention.
Figure 8 shows a time series divided into sub-series based on change points, according to an embodiment of the present invention.
Figure 9 shows a time series with narrow and wide subseries according to an embodiment of the present invention.
Figure 10 is a flowchart of a method for obtaining a final deviation score vector according to an embodiment of the present invention.
Figure 11 shows a sample SeRA graph, a sample DRSG, and a final DRG (DRSG) with information augmented from the SeRA graph according to an embodiment of the present invention.
Figure 12 illustrates a proposed pipeline for triggering a DRSG assisted by monitoring structured Manufacturing Execution System (MES) data according to an embodiment of the present invention.
Figure 13 is a diagram showing sensor and recipe objects according to their properties according to an embodiment of the present invention.
Figure 14a shows the DRSG evolution/alternative DRSG due to state change for the same original observation - secondary observation (OO-SO) according to an embodiment of the invention, and Figure 14b shows the alternative DRSG according to an embodiment of the invention. This is a flow chart to explain recommendations based on equipment status.
Figure 14c is a graph for explaining recommendations based on equipment status according to an embodiment of the present invention.
Figure 15 shows a time series with Change Point Present (CPP) and Change Point Absent (CPA) characteristics corresponding to the first and second differences according to an embodiment of the present invention.
Figure 16 is a heatmap diagram of the proposed WDTW matrix for sensor time series data classified under CPA captured before and after repair operations, according to an embodiment of the present invention.
17 is exemplary time series data for each category according to an embodiment of the present invention.
Figure 18 shows a proposed fully fuzzy score generalization (FSG) based score generalization unit according to an embodiment of the present invention.
Figure 19A shows a FSG surface according to an embodiment of the present invention.
Figure 19B is a schematic diagram of a system that provides recommendations regarding variations within equipment based on video data.
Figure 20 illustrates a computing environment implementing a mechanism for root cause analysis of deviations associated with equipment in accordance with an embodiment of the present invention.

Embodiments of the present invention and their various features and advantages are described in more detail in connection with non-limiting embodiments illustrated by the accompanying drawings and described by the following description. Descriptions of well-known components and processing techniques are omitted so as not to obscure the embodiments of the present invention. Additionally, the various embodiments of the invention are not necessarily mutually exclusive, and some embodiments of the invention may be combined with one or more other embodiments to form new embodiments. As used herein, “or” means non-exclusion unless otherwise indicated. The examples of the present invention are merely for understanding how the embodiments of the present invention are practiced, and to enable those skilled in the art to practice the embodiments of the present invention. Accordingly, the examples should not be understood as limiting the scope of the invention.

Entity, module, and unit are used interchangeably throughout this specification.

Embodiments of the present invention disclose a system for fundamental analysis of equipment-related deviations. The system includes a SerRA entity that detects equipment-related deviations. The DRG entity determines the DRsG and resolves deviations based on the DRG. DRsG includes remedial actions and observations related to equipment. The recommender entity recommends DRsGs to resolve equipment-related deviations.

Unlike conventional methods and systems, the proposed method can perform root cause analysis of equipment-related deviations in an easy, reliable, accurate, fast, efficient and flexible manner. The proposed method can be used to rank DRSGs using different attributes such as time, execution cost, quality of results, and execution frequency, thereby improving the accuracy of root cause analysis of equipment-related deviations. The proposed method can obtain a confidence level for the deviation score of sensor data using a fuzzy inference system. The proposed method provides better analysis based on gradual changes in equipment condition between preventive maintenance (PM) cycles. The proposed method provides better analysis during rapid changes in equipment status. The proposed method can effectively record DRSG and sensor analysis data in the Comprehensive Elucidation Table (ComET) scheme.

The proposed method can use modeling techniques different from the Bayesian network method. The proposed system can be used as a recommendation system by both users or machines. The proposed method can achieve reliable fault detection based on statistical models without the need for extensive manual intervention. The proposed method uses a heuristic-less analysis method for out-of-spec (OOS) detection that does not require recipe information. The proposed method provides for solving two equally feasible recommendations by extracting the necessary support data, instead of querying the user/analysis system with a second (successive) question/query. The proposed method can provide accurate understanding of deviations and resolution actions (RA). The proposed method can provide fast, accurate and reliable recommendations of solution tasks to resolve deviations. The proposed method can provide reliable and robust real-time identification of deviations/defects and timely triggering/recommendation of appropriate RA. The proposed method can improve expert system power.

The proposed method can form an expert system with a deviation resolution/resolution action recommendation system for production process/manufacturing process in industry (such as semiconductor industry, industrial plants, petroleum refineries, etc.).

Referring to the drawings, and more particularly to Figures 1 to 20, where like reference numbers consistently refer to corresponding components, preferred embodiments are shown.

1 is a schematic diagram of a system 100 for root cause analysis of deviations related to equipment 200 according to an embodiment of the present invention. The system 100 includes a SeRa entity), a DRG entity 104), a recommendation entity 106, and a comprehensive description table 108. SeRA entity 102 detects deviations associated with equipment 200. After detection of a deviation, the DRG entity 104 resolves the deviation by determining a Deviation Resolution sub-Graph (DRsG).

In an embodiment of the present invention, DRsG includes Resolution Action (RA) and observations associated with equipment 200. In some embodiments of the invention, the observation is an original observation. The original observation identifies the beginning of the deviation. Original observations trigger resolution actions for deviations. Typically, raw observations are performed using metrology equipment (e.g., out-of-spec (OOS) critical dimension equipment, etc.).

In some embodiments of the invention, the observation is a secondary observation. Secondary observations are performed within the process of deviation response. Secondary observation is also referred to as in-process observation. Typically, secondary observations are performed on process equipment (eg, high-reflection power devices, etc.).

In some embodiments of the invention, a remedial action (RA) refers to any modifications made to the equipment 200 to resolve the deviation. These properties include the equipment ID on which modifications were made, the modified part or module, and the name and value of the modified parameter.

Based on the DRsG, the recommender entity 106 recommends DRsGs to address deviations associated with the equipment 200. Additionally, ComET 108 stores the SeRA graph and DRG.

In some embodiments of the invention, the DRG is formed based on data received from multiple sources. In some embodiments of the invention, the data is structured data. In some embodiments of the invention, the data is unconstructed data.

The unstructured data comes from various sources such as deviation reports, vendor manuals, standard operating procedures, emails, user logs, etc. Unstructured data contains descriptions of regularly generated deviations. According to several analyses, unstructured data will account for approximately 79% of all data in the near future. Unstructured data typically contains information about the original observation recorded at the beginning of the deviation (e.g., whether the critical dimensions of the device were out of specification, actions taken to correct the matter (e.g., the radio frequency used in the recipe), (reduction of RF power), intermediate/secondary observations (e.g. high reflex power) and final examination results.

Structured data includes information related to equipment 200. Structured data includes, for example, sensor reading information, equipment availability, recipe per wafer, measurements, sensor time series data for a specific wafer run, information on replaced parts on equipment, etc. do. Unlike unstructured data, structured data is recorded in real time according to the wafer processing schedule.

In some embodiments of the invention, structured and unstructured data are used to recommend correct resolution of deviations. Additionally, the proposed method performs real-time defect detection and identification of resolution tasks using a combination of the structured and unstructured data.

In some embodiments of the invention, the deviation is identified based on a SeRA graph. The SeRA graph represents the relationship between equipment groups and solved tasks.

In some embodiments of the invention, a SeRA graph is formed based on a before-and-after analysis across a set of DRsGs of the device. Pre- and post-analysis identifies equipment with signatures of repair work, creating causal relationships stored for DRsG's automated recommendations.

A back-and-forth analysis engine (not shown) calculates a deviation score based on whether the device's time series signal includes a change point. A confidence engine (not shown) determines a confidence score based on the deviation score. A fuzzy score globalizer (not shown) determines a deviation vector based on the confidence score. A sensor selector (not shown) establishes a causal relationship between the deviation and the equipment based on the SeRA graph. The sensor selector triggers the DRsG based on the deviation vector when a deviation is detected.

In some embodiments of the invention, if the change point is not available in the time series signal, Window-let Dynamic Time Warping (WDTW) is performed to change the deviation score of the time series signal of the equipment. Calculate .

In some embodiments of the invention, if the change point is available in the time series signal, Window-let Gamma (WGamma) is performed to calculate the deviation score of the time series signal of the equipment.

In some embodiments of the invention, if the change point is available in the time series signal, Fisher discriminant index (FDI) and Dynamic Time Warping (DWT) are performed to determine the time series signal of the equipment. Calculate the above deviation score.

In some embodiments of the invention, it is calculated based on the long term trend and short term aging trend.

In some embodiments of the present invention, the DRsG is triggered by the sensor selector when the deviation vector satisfies a threshold.

In some embodiments of the invention, the time series signal is obtained from video data of the at least one device.

In some embodiments of the invention, DRSG provides an action flow for any negative observations. DRSG provides the flexibility to analyze tasks even when the situation at hand is unique. Additionally, DRSG recommends modifying the workflow even if the deviation has not occurred in the past. DRSG is assisted by data analysis of structured data. Data analytics assists real-time monitoring of processes, flag deviations and related DRSGs.

In some embodiments of the invention, the proposed method is used to convert unstructured data, such as deviation reports, into objects. This can be done using Natural Language Processing (NLP) or statistical models (e.g., Term-Frequency-Inverse-Document-Frequency (TF-IDF), entropy, etc.) assisted by tools such as Stanford CoreNLP, Natural Language Toolkit (NLTK), etc. based method, etc.) can be obtained using a text mining algorithm. To increase the visibility of all related tasks that need to be performed to understand and resolve deviations, the proposed method classifies deviations into the following main object types:

Different objects with their own properties are shown in Figure 2 to perform data analysis. Objects have various properties such as date, time, equipment ID to which the object is linked, observed wafer/lot ID, previous and subsequent object IDs, etc. Figure 3 shows a DRG according to an embodiment of the present invention, where P refers to PASS and F refers to FAIL.

Additionally, once the deviation responses are converted into the above-described objects, the recommendation process links them. These connections are mainly established based on the date and time characteristics of the objects. In a well-written deviation report, a time series of objects can be tracked by the positions of different groups of tasks within the document. For structured data, objects typically have a system-generated timestamp associated with them. Once a connection is established, it forms a chain of objects leading from the beginning of the deviation to its end. Figure 3 shows this object chain. This chain is referred to as a deviation resolution graph (DRG). As shown in Figure 4, deviation resolution includes one original observation (OO ₁ ), four secondary observations (SO ₁ - SO ₄ ), and four solution operations (RA _A - RA _D ). Between secondary observations, a number of solution tasks are arranged. To further analyze the deviation response, the method divides the DRG into smaller subsets based on the analysis rules of the deviation resolution graph.

Analysis rules for deviation resolution graphs: In a typical DRG, multiple repairs and secondary observations are interspersed between each. In order to interpret the solution tasks that have a significant impact in resolving the issue, the steps are listed below.

From the deviation resolution graph, the proposed method can be used to generate sections starting from a negative observation and ending with the next positive observation of the same type. Additionally, the method terminates the graph section by adding a final node (E) after the last positive observation node. This graph section is referred to as the deviation resolution sub-graph (DRSG). Figure 4 shows four DRSGs obtained from Figure 3.

In one embodiment of the invention, in case there is no positive observation for the corresponding negative observation, the proposed method includes the entire section starting from the positive observation node to the end of the DRG. Therefore, for observations of SO4, the quadratic deviation resolution graph is shown in Figure 4c. Based on the above process, DRG becomes DRSG after addition of the terminal E node.

In some embodiments of the invention, the proposed method treats all other in-process observations of different types as branching points. These branch points will be used as entry/exit points from other deviations.

In some embodiments of the invention, each DRSG is treated as a tree branch with observation nodes acting as entry/exit points for other DRSGs according to the following rules:

At any observation node (SO ₂ ), if the result is opposite to the expected value (e.g. SO ₂ (F) instead of SO ₂ (P)), the flow exits the DRSG and returns to a node that satisfies the following criteria: We arrive at a graph: the target DRSG must have at least one instance of the same node, i.e. at least one node must be SO ₁ (P).

In selecting multiple target DRSGs that satisfy the criteria for observation nodes and their values, the same ranking method is listed below.

Before exiting, the user is given the option to continue to DRSG or exit, and the user's decision overrides the procedure.

If there is no DRSG that satisfies the criteria, the flow moves to the next node after obtaining user approval. If the user approves, ComET 108 is updated with another DRSG with the opposite node. For example, in the above case, ComET 108 gets another entry with the SO ₂ (F) node replacing SO ₂ (P). This is because, even with the error value of SO2 and that information, the case of the solution flow that proceeds can be used to resolve subsequent deviations.

The procedure is shown in Figure 5. This includes DRSG for SO ₁ . At the second node, SO ₂ (P), instead of returning the expected value for each original DRSG, an error value is obtained. Then, based on user confirmation, the flow branches to another DRSG in SO1 with SO ₂ (F) node, or continues to the same DRSG. If the decision is to continue with the same DRSG, ComET 108 is updated with the new DRSG. If the decision is branching, subsequent steps are directed by the new DRSG. Using the above rules, we can create DRSGs and traverse them for the desired execution of the solution flow. This multi-tree structure can be stored in the form of a table or database with appropriate indexes. This collection of databases is referred to as the Comprehensive Elucidation Table (ComET) scheme.

For example, a typical piece of equipment within a manufacturing environment includes hundreds of sensors measuring a variety of properties such as temperature, pressure, gas flow, incident and reflected RF power, optical emission spectra, current and voltage, etc. Additionally, there are actuators such as pressure valves, gas flow valves, solenoids, heaters, RF power controllers, etc. that are controlled by recipes executed within the equipment. The MES of a semiconductor fab includes not only actuator data but also sensor data. If the equipment includes a complete set of sensors, the time series deviation of at least one sensor/actuator can be tracked. However, knowing which sensor to monitor for a given deviation and identifying the deviation at the sensor level can require specialized knowledge. In the following sections, the proposed method automates model learning and detection of deviation and fault events at the sensor level and uses this information to improve the applicability of DRG and DRSG.

Before/After Resolution Action (B/A RA) Analysis: B/A RA analysis includes learning and monitoring phases. In the learning phase, for each DRSG, the proposed method explores all changes in sensor behavior immediately after the first node of the DRSG (i.e., before the solving operation) and immediately after the end of the DRSG (i.e., after the solving operation). If any significant change in behavior is observed and the monitored sensors show similar deviations in behavior, this means that the corresponding node in the DRSG can be raised with a certain confidence.

FIG. 6 is a flowchart 600 illustrating a method for analyzing the causes of deviations related to equipment according to an embodiment of the present invention. At step 602, the method includes detecting a deviation associated with the equipment. In one embodiment of the invention, the method allows the SeRA entity 102 to detect deviations related to equipment. At step 604, the method includes determining the DRsG based on the DRG to resolve the deviation. In one embodiment of the invention, the method may allow the DRG entity 104 to determine the DRsG based on the DRG and resolve any deviations. At step 606, the method recommends DRsGs to address deviations associated with the equipment. In one embodiment of the invention, the method allows the recommending entity 106 to recommend DRsGs to resolve equipment-related deviations.

The various tasks, operations, blocks, steps, etc. of the proposed method may be performed in the described order or in a different order, or may be performed simultaneously. Additionally, in some embodiments of the invention, some operations, operations, blocks, steps, etc. may be omitted, added, changed, or skipped without departing from the spirit of the invention.

Figure 7 shows a time series with change points according to an embodiment of the present invention.

Numerical time series time - series ) data : One of the first steps is when sensors detect jumps (e.g. gas flow, RF power sensors), peaks (e.g. optical emission spectra), peaks (e.g. temperature), or oscillations (e.g. The goal is to identify whether it contains any change points (CPs), such as the optical emission spectrum. Two categories are defined: Change Point Absent (CPA) and Change Point Present (CPP). This information helps perform fine-tuned analysis. Figure 7 shows an example of a curve with apex and jump change points.

In one embodiment of the invention, when change points are not available in the time series signal, Window-let Dynamic Time Warping (WDTW) is performed to calculate a deviation score of the device's time series signal.

In one embodiment of the invention, if a change point is available in the time series signal, Window-let Gamma (WGamma) is performed to calculate the deviation score of the device's time series signal.

In one embodiment of the invention, when change points are available in the time series signal, Fisher discriminant index (FDI) and Dynamic Time Warping (DWT) are performed to calculate the deviation score of the instrument's time series signal. Calculate.

CPA : In the absence of change points (CP), the WDTW process is proposed, which involves bisecting the B/A time series data in succession and calculating the dynamic time warping (DTW) score between the two sensor time series at each level. do. The WDTW matrix can be monitored for any out-of-spec (OOS) value over time in any time window of interest. The advantage of using a window-let approach using a single DTW score (computed over the entire time series) is that the time window of deviation can be narrowed as desired without knowledge of the downstream information of the recipe in the time series. This allows the proposed invention to be accessed without recipe information. Additionally, due to windowing, averaging of results when the entire time series or a fixed window size is considered is avoided. Based on the regions within the WDTW matrix where cause-effect manifestations are observed, these are identified as regions of interest (ROI). These ROIs are monitored during each successive wafer process to check for deviations to detect defects and thus trigger corresponding remediation actions. The value of the WDTW score integrated from the ROI of the WDTW matrix is analyzed by a confidence engine and a confidence score is given based on the dynamics of the process and process equipment.

CPP - Temporal deviation monitoring : When a change point (CP) exists, there are two possibilities: temporal deviation and amplitude bias/slope deviation. Temporary deviations are described first. Following detection of temporal deviations between before and after the time series, we successively bisect the B/A time series data and This leads to creating matrices and calculating their singular value decomposition (SVD). Here, n represents the maximum number of samples within the bisected time window at each level, and i represents the sensor number. Once the SVD is obtained, the ratio of the largest and smallest singular values is calculated, which is , where A and B refer to before and after the deviation and (p, q) refer to the time window over which the calculation is performed. A single score (when considering the entire time series as a single window) can cause problems similar to those described above (a single DTW score) and may not pinpoint the exact location of the deviation. Therefore, from such entries, a matrix is formed, which is referred to as the Window-let Gamma matrix, and can be monitored, similar to WDTW in the case of CPP. The integrated values from the ROI of the WGamma matrix are passed to the confidence engine for further analysis.

CPP : Amplitude deviation monitoring : To monitor amplitude deviation (both in bias change and slope change), the time series is divided into sub-series using CP, i.e. the start and end point of the window, as shown in Figure 8. And they are classified into “narrow range” and “wide range” based on the kurtosis of the subseries values as shown in Figure 9. While the wide subseries can provide amplitude bias and slope bias deviation, the narrow subseries can only provide amplitude bias deviation. Depending on this, the DTW or FDI based method or the DTW method plus the FDI based method is used to obtain the deviation score. The corresponding scores: S _FDI and S _{FDI - DTW} are passed to the Confidence Engine for further analysis.

Categorical /nominal time series data : To calculate deviation scores for numerical time series, the proposed method follows similar lines as before, using the Window-let Min-Hash matrix ; It provides a categorical time series based on the min-hash distance of 2 to 3 grams leading to the creation of WMH). The obtained deviation scores are passed to the sensor selector module for further analysis.

Even after obtaining a deviation score, there is an inherent problem with the level of confidence in the ability to accurately detect the deviation. To deal with this, the method utilizes a confidence engine that provides a confidence score for each deviation score calculated above. The deviation scores calculated from the numerical time series are passed to the confidence engine. Due to the nature of deviations occurring in categorical time series data, the corresponding deviation scores through the WMH matrix are directly transmitted to the sensor selector module. Therefore, the input to the confidence engine is a set of four deviation scores derived from numerical time series: S _WDTW , S _WGamma , S _FDI and S _{FDI-DTW arranged as vectors.}

Confidence Engine (CE): The mere occurrence of a high deviation score for a time series may not be a good criterion for triggering the corresponding DRSG. One of the main reasons for the above observation is the change in the state of the equipment, which can occur slowly or rapidly. The proposed method deals with these equipment state changes. The results from the confidence engine constitute one of the key inputs of this method. The confidence engine includes a historical data store (HDS) module and a fuzzy score globalization (FSG) submodule.

The HDS module analyzes historical data to obtain a set of local confidence scores. The local confidence score is used to obtain the global confidence score. The overall confidence score is multiplied by the corresponding deviation score to obtain the final deviation score. The following local confidence scores are calculated by the sensor bunching unit, the long-term dynamics/aging unit, and the short-term dynamics/aging unit.

The long-term dynamics/aging unit operates by: considering the long-term dynamics of the deviation scores through a regression model and providing insight into how the current values fit the learned long-term characteristics. Since there are four deviation scores, this submodule computes the corresponding four local confidence scores β _i , (i = 1, 2, 3, 4) .

The short-term dynamics/aging unit operates by considering the short-term dynamics of the deviation scores as quantified by the variance/standard deviation of the deviation scores over a range of previous processes and a certain forgetting factor. Additionally, the short-term dynamics/aging unit provides insight into how current values fit to short-term characteristics. Since there are four deviation scores, this submodule computes the corresponding four local confidence scores γ _i , (i = 1, 2, 3, 4) .

The output of these sub-modules is consolidated in the form:

This is passed to the FSG module to obtain an overall confidence score.

The Fuzzy Score-Globalization (FSG) sub-module operates based on the Fuzzy Inference System (FIS) that receives the integrated vector. The FSG submodule has four corresponding overall confidence values: Calculate . These values are the four deviation scores and multiplied by The final deviation score per sensor is obtained as . This final deviation vector is used by the sensor selector unit to establish a causal relationship between the sensor and the deviation, and further establishes the final deviation vector ( ) Triggers DRSG RA based on

Sensor Selector (SS) : Sensor 'groups' and sensor resolution-action (SeRA) graphs representing relationships between sensors (groups) and resolution tasks are inferred through the B/A RA analysis pipeline and sensor aggregation unit. do. In a SeRA graph, sensors and solving tasks form nodes. If a causal relationship between a given solution task and sensor (group) data is observed, an association (edge edge) is added between the given solution task and the sensor (group). Additionally, each edge is annotated with an association weight. When considering edges, if causality is high (e.g., if the relationship is more frequent or dominant when inferred based on past observations and/or by experts in a particular field), the higher the relative association. Weights are assigned to those edges. Association weight is a necessary and sufficient condition for an association graph. This is because the mapping between the set of solving tasks (i.e. S _RA ) and the set of sensors (i.e. S _S ) is not necessarily bijective. Therefore, weights quantify an indicator of the confidence with which an association can be modeled.

An example SeRA graph for the following scenario is shown in Figure 11 along with the event-graph for the overall view. The RF power sensor measurement is out of specification (OOS) in the negative direction (current value is usually less than required). This may be due to damage to the power line to the RF coil or damage to the RF coil. The final (sample) DRSG derived from a combination of historical data querying of the power of the DRSG and real-time RA triggering of the power of the SeRA is also shown in Figure 11. The final deviation score (vector) along with the final DRG (DRSG) is used to track and trigger resolution actions in case anomalies are observed. The same information is stored within ComET 108 in tabular form. While a given group of sensors is being monitored (focused monitoring), if If is greater than the threshold; Based on, the corresponding deviation DRSG is triggered. Here arg() refers to the 'declination' of the direction in which the deviation was observed, which can be mapped directly to a deviation score or a combination thereof ({ F _WDTW , F _WGamma , F _FDI and F _{FDI - DTW} } and/or S _WMH ). There is, and it is out of specification (OOS) or abnormal.

Figure 12 shows a proposed pipeline for triggering DRSG assisted by monitoring structured MES data according to an embodiment of the present invention. A complete data flow diagram summarizing the proposed pipeline for triggering DRSG assisted by monitoring structured MES data is shown in Figure 12. Previous sequential wafer process results are stored for OOS inspection and learning of regression and variance models for long-term and short-term trends. Assurance Engine's HDS is supplied in-house or externally based on a number of actual infrastructure factors. The output of the sensor selector is combined with the DRSG and thus tabulated into ComET 108 to form an ES with real-time (wafer process level) automatic defect detection and DRSG (RA) triggering capabilities.

ComET configuration : Output from DRSG, confidence engine including B/A RA analysis and SeRA graph from sensor selector are stored in ComET 108. ComET 108 also includes deviation responses and additional relevant data extracted from knowledge in the field. ComET 108 forms the basis for discovering and learning relationships between various processes within a manufacturing process. From a relational database perspective, a schematic with the following tables (i.e. main table, sensor selector table, recipe table, cost table, timing table, data collection frequency table, equipment-independent process variation table and similar equipment tables) It can be explained as a (schema).

The main table contains DRSG information forming the ComET schema. The main table will store the nodes along with all their attributes. The sensor selector table includes information on sensor objects as shown in FIG. 13. The Deviation ID(s) attribute links sensors from the main table to the DRSG. Information from the SeRA graph obtained from the B/A RA analysis as well as the confidence engine and sensor selector is stored as attributes in this table.

The recipe table stores information about recipes used in equipment. For example, a recipe table stores properties such as the number of substeps, duration, pressure, temperature, power, gas flow, alarm or interlock level of each parameter. Recipe tables can be pre-formed from equipment or process specification documents. An example of a recipe object is shown in Figure 13.

The OO-SO table represents secondary observations that are necessary and sufficient conditions for the original observation to pass. This table can actually be created by merging the OO (original observation) and SO (secondary observation) tables of deviation IDs. Typically within a manufacturing environment, original observations are performed on product wafers. This may be done on a statistical sampling basis, for example. If this OO is OOS, the SO associated with the equipment is used to confirm the pass/fail status of the original observation.

Cost Table: OO requires product wafers to be run and therefore can prove expensive. Additionally, OO may require the wafer to undergo some or all of the steps of the process prior to the point at which OO is performed. On the other hand, SO can be low-cost because it is performed in a short loop that requires a process that goes through limited steps.

Time Table : The time taken to prepare a wafer for SO is generally much less compared to OO, as OO wafers require the entire process up to that point, while SO goes through a short loop.

Data collection frequency table : In general, SO has a higher data collection frequency than OO. Therefore, changes in equipment state can be obtained with much better granularity with respect to time.

Process changes unrelated to equipment: Some causes of poor OO can be processes performed upstream, wafer-related issues, or poor operating techniques such as incorrect recipes. These causes are unrelated to the condition of the process equipment in question. SO is largely immune to these issues as it involves only a small number of process steps and can be performed on multiple wafers at low cost and in a short time.

The purpose of using this OO-SO table is to know candidates for subgraphs for solution. For example, if SO ₂ and SO ₄ are necessary and sufficient conditions for OO ₁ , then all solving subgraphs starting with SO ₂ and SO ₄ will be candidate subgraphs for the solving task for any deviation associated with OO ₁ . Among these candidates, SO ₂ and SO ₄ are sorted and recommended to the user based on frequency, time, cost, etc., as described in more detail below.

Similar Equipment Table : Within a manufacturing environment, multiple pieces of equipment may be used interchangeably to perform the same process task. In other words, there is a group of equipment that is considered equivalent for a particular process. This information exists in the form of tables and is available within the MES database. This information will be used to perform data analysis in the proposed method. For example, if equipment A and B are considered similar, then for any deviation of OO ₁ within equipment A, the solution graph of equipment B corresponding to OO ₁ can be used for analysis. This is especially useful when new equipment is introduced into the manufacturing environment and the historical variation associated with it is not sufficient.

Advantages of the ComET schematic: A typical problem-solving procedure in semiconductor equipment or manufacturing processes involves performing multiple observations interspersed among the solving tasks. In many cases, it is difficult to know the exact impact of an operation on observations. Therefore, all or part of the work that precedes the observation may affect the results. There is no need to know the exact nature of this relationship because the proposed format stores all operations that cause a change in the observed state (from fail to pass).

Solving subgraphs for the same type of deviation can be sorted according to different properties of the graph, such as number of branches, time and cost of solution, availability of resources, frequency of occurrence, etc. Depending on the desired alignment method, different machine learning algorithms, such as Bayesian networks, neural networks, and decision trees, for example, can be adjusted to obtain the desired recommended resolution flow.

Because they can be sorted by frequency, it becomes possible to account for equipment state changes that occur gradually between PMs or suddenly after part replacement. This mechanism is explained in detail in the following section.

According to the above-mentioned rules, each negative observation node also becomes the starting node of DRSG. As a result, it is possible to identify DRSGs whose deviations did not occur independently in the past. That is, SO ₄ (F) in FIG. 3 occurs as part of the main solution flow. However, after generating the DRSG according to the prescribed rules as shown in Figure 4, we find a list of actions to perform if SO ₄ is the only deviation observed. That is, SO ₄ (F)->RA->SO ₁ (F)->RA _C ->SO ₁ (P)->RA _D -> SO ₃ (F)->RA _A -> SO ₃ (P) -> E (from Figure 4c).

Each branch point depends on the observation result corresponding to the node. Based on this, the recommended solution flow may vary. Therefore, a dynamic solution is established that can adapt according to the results of the ongoing procedure. The proposed method provides dynamic reconfiguration of recommended DRSGs.

Due to the connection of sensors (groups) to the DRSG and real-time (wafer process level) monitoring through B/A RA analysis as well as SeRA graphs obtained from the assurance engine and sensor selector, ComET 108 identifies deviations and provides solutions in real time. to provide. It provides real-time identification of defects and recommended actions.

Pruning the DRSG: When initially created, there may be many variations in the length and content of the DRSG for the same observation. For example, compare the first two DRSGs in Figure 4. To eliminate these changes, the following method for conciseness of subgraphs (i.e., opinion of experts in the field and pass/fail possibility of in-process observations) is proposed.

Opinion from subject matter experts : It is important in DRSG to draw on the knowledge of subject matter experts in determining specific tasks/observations and their sequencing. This knowledge is much more reliable than asking experts about probabilities and possibilities. This is because knowledge of the necessary and sufficient conditions for operation/observation can be confirmed by the equipment supplier's instructions. Therefore, it is less likely to go wrong. Examples of these queries are below.

Regarding the first subgraph in Figure 4, one can ask “Are in-process observations such as SO ₂ , SO ₃ , etc. necessary?” According to the answer, the proposed method can be used to create a subgraph by removing unnecessary operations/observations. can be simplified.

Regarding the second subgraph in Figure 4, “Is RC a necessary and sufficient condition for solving SO ₁ (F)? Or do we need both RA _A and RA _B ?". Depending on the answer, the proposed method can promote/demote the second subgraph with respect to the first subgraph.

Based on the expert's answers, recommendations for deviation responses can be strengthened.

Possibility of pass/fail in-process observations: if in-process observations in a solution subgraph (e.g. SO ₂ , SO ₃ , SO ₄ in the first subgraph of Figure 4 ) meet a certain threshold (e.g. θ) If it has more than a pass/fail probability, it can be removed from the subgraph. Therefore, in the same example, if SO ₂ passes with probability > θ, SO ₃ fails with probability > θ, and SO4 fails with 0 < probability < θ- Δ (θ > Δ > 0), then SO ₂ and SO ₃ can be removed and SO ₄ can remain in the DRSG.

Recommendation Entity 106 : Once ComET data is generated, it can be used to create a recommendation entity 106. The recommendation entity 106 selects the correct DRGS with the appropriate alignment when the user identifies a failing observation or when the B/A RA analysis, confidence engine, and sensor selector observe deviations in sensor measurements and trigger a flag. can be provided. Since all data related to the deviation and DRSG is stored in ComET 108, recommending entities 108 within the DRSG's list is a matter of querying for the correct OO/SO/Sensor ID.

Additionally, unlike traditional/classic RS, which asks the user for details if more than one recommendation of an RA is retrieved for a query, the recommender entity 106 collects the minimum possible number of most accurate RA recommendations from the analysis pipeline (ideally unique). It is possible to automatically collect and reinforce the information needed to clearly show the information. For example, consider a scenario where query Q1 is queried. If two actionable recommendations are explored: R1 and R2 are explored using ComET (108) (and DRSG) tracking using OO, SO or sensor ID - where R1 is true only in sub-condition SC1 On the other hand, R2 is true only in the mutually exclusive subcondition SC2. At this time, classic RS asks the user a second query. “Is R1 true or R2 true?” When the user enters an answer to the second query, a corresponding recommendation is provided. However, the ComET-based recommender entity 106 determines that such endorsement relationships are Automatically references MES data (or inventory management system (IMS) or yield management system (YMS)) if it can be established between /YMS data and the time/place where the problem/event/observation occurred. You can query only the wafer ID or timestamp for and collect data when R1 or R2 is true. Entering the wafer ID or timestamp for the time/place where the problem/event/observation occurred determines whether R1 is true. Alternatively, more detailed questions, such as determining whether R2 is true, can be entered much more easily by the user. Additionally, in situations where it is feasible, automatically collecting and enriching the necessary data can be done rather than relying on the user each time. Faster and sometimes more accurate. Therefore, thanks to real-time support from background systems such as MES, the proposed method can help implement a fast and highly automated solution task recommendation system.

The main remaining challenge is to align the DRSG and account for the most unique characteristic of semiconductor devices, namely state changes.

Ranking (fine-tuning) of DRSGs: In a typical semiconductor manufacturing environment, the same defect observation may have multiple DRSGs, such as SO ₁ (F) of shown in FIG. 4 . When this situation occurs, the following methods for alignment (frequency of occurrence, time required for DRSG execution, cost of DRSG execution, quality of results obtained by DRSG termination) are proposed at the point of recommendation to the user.

Frequency of occurrence : Solving flows corresponding to the same observation can be sorted according to the number of times they have been executed. For example, from Figure 4, if the second subgraph has been executed more times than the first subgraph to solve SO ₁ (F), a higher rank is set to the second subgraph and the It will be provided as a first recommendation. A potential drawback of this approach is that changes in the state of the equipment may be ignored.

Time taken for DRSG execution : Another alternative to using frequency is to sort subgraphs according to the time it takes for execution to complete. It should be noted that ComET 108 includes information on the time required to complete execution of each object. From this, the total time required can be obtained and the recommended DRSGs can be sorted accordingly.

Execution Cost of DRSG : Frequency and Execution Time Another alternative is to rank DRSGs by the cost associated with their execution. These costs consist of operational costs such as required resources, replacement parts, and number of people required to execute the work.

Quality of results obtained at the end of a DRSG : DRSGs can be sorted according to the quality of the final results. This can be defined as a function called Q as follows.

where V _i , TV _i , USL _i , LSL _i , and W _i are observation value, target value, upper spec limit, lower spec limit, and i, respectively. This refers to the weight of the observation within the second process. Q can also be replaced with an equivalent value that signifies the quality of the final result. Therefore, if the first subgraph in Figure 4 has a better Q value compared to the second subgraph, the first subgraph should be recommended if Q has priority for frequency, time, and cost. Due to the scalable nature of ComET 108, the Q value is stored in the main table in conjunction with the deviation ID for reference.

Description of equipment status changes : The frequency of occurrence of DRSGs can be used to determine the recommended route for work. Other machine learning algorithms, such as Bayesian networks, neural networks, decision trees, deep and reinforcement learning algorithms, may also be used to determine recommendations. However, this approach assumes static behavior of the system. Manufacturing equipment is essentially non-stationary. The state of equipment changes depending on use and certain external events. The state change rate may be gradual, such as due to build-up of residual processing material between preventive maintenance (PM) cycles within the process chamber, or wear of components, or defective components. It can be drastic due to things like the replacement of . As a result, the frequency of repairs may vary over time, and it is important to take this phenomenon into account in order to predict the correct deviation resolution subgraph. Below, an approach is proposed that considers not only equipment state changes between PM cycles, but also rapid state changes (i.e., equipment state changes between PMs and equipment state changes after component replacement).

Typically within a manufacturing environment, PM schedules are based on number of runs, wafers processed, or turnaround time. As a result, repair frequency can vary within a PM cycle. Figure 14a shows two subgraphs with the same variance (high average critical dimension). The solution task for the first subgraph RA _A is tuning of the matching circuit of the etch equipment, while for the second subgraph RA _B it is chamber cleaning. Now, if OO ₁ is observed at the end of the PM cycle when many wafers are passing through the chamber, then RA _B is more likely to be a valid solution than RA _A. The reverse would also be possible if OO ₁ was observed at the beginning of the PM cycle. If the analysis of the recommended solution subgraph were simply based on frequency or machine learning algorithms, it would not yield a predicted answer like this. The proposed method takes into account the change in repair frequency within a PM cycle when multiple solved subgraphs exist for the same observation, using ComET 108 as shown below.

First, a method to obtain a set of reference PM-solving subgraph vectors is described.

1. Assume that data is available for n PM cycles and that these cycles are determined by the number of wafer runs. From this data, we find the median number of wafer runs between two instances of the same subgraph. In subgraph SG1, η is the median of the number of wafer runs.

2. Divide the total PM cycle execution by η and obtain the sum of occurrences of SG1 in each interval. In this way, one vector per PM cycle is obtained. Note that these vectors can have different orders in cases where some PMs have more or fewer runs than a given number of runs.

Here, SG1 _PM1 is the vector for the first PM cycle (PM1) and the dimension of the vector is given by nSG1PM1 .

1. Once the vectors of different subgraphs are obtained, they are stored in memory to be used as a reference.

The reference pattern samples different equipment state spaces. If new deviations occur within the PM cycle from the same observation (e.g. high average critical dimension), the following steps follow:

1. In the subgraph, vectors are formed in the current PM cycle in a similar way as described above. For example, in m runs, RA _A will have the following vector:

2. Next, the distance between the current PM and the reference PM is obtained by truncating the reference PM dimension by nSG1PMcurrent . For example, the distance value between PM1 and the current PM is given by:

here, is a very small positive real number to prevent division by zero. Additionally, p= 1 is the Manhattan distance (city block/Manhattan distance) and p=2 is the Euclidean distance. The choice of p may be based on the accuracy required and may vary depending on the type of task.

1. To obtain the predicted value for the next index of the current PM of SG1, the weighted sum of the reference PM is obtained.

Finally, the index values for different SGs are compared and provided to the user in descending order. It is predicted that the SG with the highest value is the one most recommended for execution.

Figure 14b is a flow chart 1400b for explaining a recommendation method based on equipment status according to an embodiment of the present invention. At step 1042a, the method includes obtaining the median of the executions among the two instances of DRsG. At step 1404a, the method includes converting the PM cycle to a vector using the median as the bin size. At step 1406a, the method includes finding a similarity metric between the current PM vector and the vector of the previous PM cycle. At step 1408a, the method includes obtaining an estimate for the next bin of the current PM cycle for each DRsG. At step 1410a, the method compares expectations across similar DRsGs and selects the maximum.

The proposed method utilizes previous information of DRsG implementations to predict the occurrence of the next DRsG.

The various tasks, operations, blocks, steps, etc. of the proposed method may be performed in the described order or in a different order, or may be performed simultaneously. Additionally, in some embodiments of the invention, some operations, operations, blocks, steps, etc. may be omitted, added, changed, or skipped without departing from the spirit of the invention.

Figure 14c is a graph for explaining recommendations based on equipment status according to an embodiment of the present invention.

Equipment status change after parts replacement : The method described above is for gradual changes in the PM cycle. Additionally, rapid changes, such as replacement of components or improvements in equipment configuration, can change the deviation profile. In Figure 14a, for example, if the matching circuit of the etching equipment has been upgraded, the first subgraph is less likely to be problematic. In these cases, rapid changes in the frequency of the associated subgraphs are expected. As a result, the reference PM vector obtained above may not be valid. In this case, the following is suggested:

1. After repair, ask the expert whether this is a routine repair where recurrence of the deviation is expected or an unusual repair where the deviation may be corrected in the near future.

1. If it is a regular repair, no change in frequency trend is expected when compared to historical data.

2. If this is an unusual one-time repair, follow these steps:

a. Ask the expert whether a rough wafer run can be obtained before defect recurrence. This information may be obtained from previous experience or supplier manuals.

b. Next, we ask the expert which DRSG is linked to the unusual repair. Then, the connected DRSG is flagged as least recommended for the number of runs obtained above. In the current example, if OO ₁ (high average critical dimension) occurs, the first subgraph (SG1) will be recommended as the last option.

c. If the number of runs between two consecutive SG1 runs is less than or equal to the median of SG1's runs (in this case η), remove the aforementioned flag set and implement the mechanism described in the section "Equipment State Changes Between PMs". Note that if none of the previous SGs solve OO1(F), the user will have to run SG1.

Additionally, unstructured data can be mined by event chains, observations/measurements, and resolution tasks to form a DRG or multiple DRSGs. Additionally, ComET based on DRG can store and retrieve information for further analysis. The analytics platform can be assisted by a statistical analysis chain or a more sophisticated machine learning-based analysis chain. Additionally, analysis accuracy and robustness (i.e., indicators of goodness) can be improved by limiting the solution space or fine-tuning the model/structure of the analysis module. This can generally be aided by including knowledge in the relevant domain. Once the information base needed to form an ES is established, that information can be interrogated in the future to quickly retrieve the necessary information.

Additionally, the proposed method provides a pipeline for automatically triggering appropriate DRSGs. For example, MES data contains equipment health/status information and is interpreted by monitoring time series data collected in real time by on-board sensor measurements.

In one embodiment of the present invention, the B/A RA analysis pipeline learns sensor time series data before and after performing a solution task to focus on monitoring structured MES data. Discovery and characterization of causal relationships between resolution tasks and MES data are leveraged in a robust multi-resolution and multi-step inference pipeline capable of real-time defect and RA triggering at the wafer process level. In the process of wafers W _B and W _A set to recipe R _e , the equipment E _e is 'before' and 'after' the resolution operation (group/chain of) to respectively resolve the detected issue due to the occurrence of the _key event M e. Let's define it to have states 'B' and 'A', meaning '. S ^B ={S _i ^B }, S ^A ={S _i ^A }, i=1, … , M, which could be a set of M sensors before and after the solving task. Data recorded by sensors may be numeric or categorical/nominal in nature.

Sensor Measurements - Numeric Time Series: Due to the heterogeneous nature of sensors and process recipes, a single deviation numerical indicator does not fit the purpose. For example, because DTW-based techniques are not affected by biases and delays (time distortions) in the time series, DTW-based techniques allow the time series to detect changes (jumps, peaks) when phase bias and delay are important deviation indicators. , continuous vibration, etc.), it will not be successful. This means that the time series is analyzed based on CP so that the time series can be narrowed down to the applicability of characterization similarity indices/algorithms based on the presence/absence of change points (CP). Based on CP, the proposed method defines two categories: Change Point Present (CPP) and Change Point Absent (CPA).

The time series is analyzed based on the following process.

I. Calculate the Median Absolute Deviation (MAD) of the wavelet coefficients at the most precise level. Based on the shape of the noise distribution (e.g. Gaussian, Laplace, Poisson, uniform), the noise standard deviation ( ) can be estimated. For example, if the noise distribution is Gaussian, the noise standard deviation estimate is given by MAD/0.6745 . Calculate the first difference ( ts' ) of the sensor time series ( ts ).

II. The histogram of ts ' is given by: , where I is the set of class-intervals (histogram bins) on the amplitude scale (time-series-value axis) am. The class spacing/bin size is calculated by:

III. In a smooth time series with no jump discontinuities, the histogram is in the zero region (i.e. ) is focused on. If a non-zero value in the histogram is found outside this region, a jump (change point) is detected. The time series is denoted as CPP.

IV. Compute ts '' (the second difference of ts ) and repeat the third and fourth steps. If a non-zero value is found outside the zero region, a change point is detected. The time series is denoted as CPP. Additionally, the method can be used to detect higher order cusps (i.e. acceleration cusps), and higher order differences can be considered (i.e. third difference ts ''' for acceleration cusps). ).

V. If the above condition is false, the time series is displayed as CPA.

The time series with CPP and CPA characteristics along with the first and second differences are shown in Figure 15. If a time series is classified as CPP or CPA, additional processing is performed. Additionally, the method can be used to detect and quantify any significant deviations before and after repair of the time series. Because a significant number of mixed/heterogeneous operations occur in a given recipe-run/wafer process, a single deviation score across the information contained in the time series of the entire recipe-run/wafer process is, in most cases, It does not contain information that can be used to distinguish between causes.

For all practical purposes, it is useful to narrow down the information contained in the deviation scores to a certain resolution in homogeneous time series. This partitioning can be assisted by including recipe sub-step information. Deviation scores can be calculated on the time series division of each recipe substep. Although in most cases this level of resolution is sufficient, certain deviations may only be revealed in certain areas of the recipe downstream. Information about the areas containing these deviations and information may disappear. To solve this problem, the proposed method utilizes a window-let based deviation detection method. Let it be a time series data vector of N samples/values for this ith sensor. To indicate the time series data of the ith sensor (S _i ^B and _Si ^A ) before and after the solution operation, the appropriate superscripts ts _i ^B and ts _i ^A are added to the notation, respectively. Split between pth and qth samples Define.

WDTW Method: Dynamic Time Warp Score is class between It is calculated as, Form a window-let DTW (WDTW) score matrix.

Index quantity used in partitioning: N/x or (N/x)+1 in a given row (i.e. depth-level) if it turns out to be a non-integer entity. The adjustment is performed by applying a ceiling/flooring operation to ensure unambiguous coverage of the entire time series. The proposed window-let DTW matrix has K columns and W rows, where K is the depth number ( It is a configurable value with a default value of . Additionally, W is calculated as lcm(s _a ), where lcm means the least common multiple, and sa is {K, K-1, K-2, … , l} is a set. K is scale-resolution and W is temporal resolution. The kth column is referred to as the column at depth-k. The depth-1 column is a constant row with the same entry as the DTW score of the entire time series ts _i ^A and ts _i ^B. The depth-2 column has two unique entries.

The first W/ 2 row is the first half time series split between ts _i ^A and ts _i ^B , i.e. and It is specified as the same value as the DTW score calculated. Similarly, the next W/ 2 row is the next half time series split between ts _i ^A and ts _i ^B , i.e. and It is specified as the same value as the DTW score calculated. In general, the (k, w) th component of a matrix is given by:

A heat-map diagram of the proposed WDTW matrix for CPA-sorted sensor time series data obtained before and after hydraulic analysis is shown in Figure 16, illustrating the inference process during the monitoring phase for fault detection and RA triggering. Among the elements of the WDTW matrix, the highest values are displayed as predetermined gray bright areas of the heatmap diagram, and relatively low values are displayed as relatively dark gray areas.

As shown in Figure 16, the horizontal axis indicates that each basic interval is a Temporal Resolution Index (TRI) ( ) refers to the time resolution expressed as ). The vertical axis represents depth resolution, and each basic unit is Depth Index (DI) ( ) means.

From Figure 16, the maximum DI is 6 and TRI ranges from 1 to lcm (6!) =60. At a given depth index, the relationship between TRI and the value shown in the heatmap can be calculated as a DTW score using the above-described equation. For example, at DI =1 , all values are essentially interpreted as a single DTW score on a single window (i.e., of the entire time series). At DI = 2 , a TRI of 1 to 30 means the DTW score calculated in the first half of the time series and a TRI of 31 to 60 means the DTW score calculated in the next half of the time series. Looking at the last DI shown in FIG. 16, i.e. 6, the maximum deviation between time series quantified by DTW scores is observed in a window spanning resolution indices 20 to 30. This is the area(s)/region(s)/duration of interest - this is referred to as Window-Of-Interest (WOI) in the proposed framework. A WOI can be a single or multiple adjacent windows or separate windows/zones.

Under the conditions of the two corresponding time series before and after the solution task, if the value of WOI is OOS (out of specification), the WOI and the ith sensor are "effected", and the solution task "occurred" by the physical parameters measured by the sensor. is displayed as the corresponding time series data. Therefore, a causal relationship between the solution action (cause) and the time series deviation (effect) can be established. Details of this causal relationship search method are described below. If the inverse causal relationship is also established as true (with some certainty) (i.e., if the time series deviation of the ith sensor (cause) of a given WOI is mapped to the 'required solution action' (effect), then the proposed method can monitor the time series data of the ith sensor in real time during continuous wafer processing and trigger an 'event' when a deviation occurs, indicating the need for related remedial action. Given these criteria, the inference process for fault detection (and triggering of resolution actions) follows:

1. B/A analysis monitors the sensors that generate causal relationships with solving actions as causes and B/A ‘deviation’ effects (events that can be modified by solving actions).

2. In one wafer process, monitoring is performed for continuous wafer processes with a sliding window, and a WDTW matrix is constructed for each continuous wafer process time series (related parameters such as recipe, chamber, etc. are fixed/ known).

3. Start from the lowest resolution (DI = 1 ) in the current WDTW metrics.

4. If the observed value is OOS (when compared to the value at DI = 1 in the WDTW matrix corresponding to the previous continuous wafer process), then the matrix from DI = 1 to DI = DI _max (6 in the figure) to obtain the WOI at the highest resolution level (DI).

5. If the value of the WOI is OOS compared to the corresponding value in the WDTW matrix corresponding to the previous successive wafer process, 'mark' the WOI. OOS can also be observed when there is a significant 'peak' at a given DI when compared to adjacent values (i.e. the values for TRI of 20-30 in Figure 16 for 10-20 and 30-40). Compare with ).

6. The WOI's scores (deviation scores) are summed and passed to the deviation score combiner and confidence engine for calculation of 'deviation detection' confidence. If no changes/OOS are detected, 0 is passed. If the sensor time series has the nature of CPP, the WDTW operation/pipeline cannot be applied and a value equal to 0 is passed. This score is referred to as S _WDTW .

Note that due to the nature of the DTW score, the OOS boundary and threshold are single values, as opposed to two values (Upper-Spec-Limit (USL) and Lower-Spec-Limit (LSL)). The OOS threshold should be based on false alarms and reality, such as the 3 sigma rule, hypothesis testing framework (with maximum a posteriori (MAP) or maximum likelihood (ML)), etc. The robustness of false alarms in OOS due to noise/outliers can be established using all known standard methods to reduce the likelihood/probability of missing events or critical events. More specifically, improvements can be made by running a consensus time series instead of a single time series before and after, based on a WDTW matrix calculated between a single (i.e. corresponding to one wafer process) time series before and after the resolution operation. Unlike decisions, the WDTW matrix can be computed between the consensus time series data before the solving operation and the consensus time series data after the solving operation. The consensus time series can be based on simple average, weighted average, synchronized weighted averaging, or DTW. Considering the consensus time series can be computed by any of a variety of methods described in the literature, such as merging, cluster centroids, etc., thereby reducing the influence of noise/outliers and thus increasing causal confidence. If appropriate, reliability can be further improved by examining similarly interdependent time series data.

To find the CPP, the proposed method uses various techniques, i.e., eigenvalue decomposition method to detect significant temporal changes (i.e. deviations in the That is, a method of detecting a deviation indicating a bias in values or a change in slope/rate of change in the values of the Y-axis, value axis, and amplitude axis is utilized.

The eigenvalue decomposition method is used to detect significant temporal changes (i.e. deviations in the X-axis, time-axis, or index-axis) using the SVD-based WGamma matrix.

SVD-based WGamma matrix: The block matrix S _i is defined as follows.

Here, ts _i ^A and ts _i ^B represent time series data before and after the solution of the ith sensor in vector form. Through the synchronization process, the number of elements in the time series vector must be the same so that the matrix S _i can be formed. In order for the solution operation to have a significant impact on the variation characteristics of the time series in terms of time delay, the columns of Si must be linearly independent (within tolerance limits), which determines the false alarm rate/probability and miss rate/probability. This minimum can be determined based on a hypothesis testing based threshold calculation method that works across or interactively with historical data.

1. Compute the singular value decomposition (SVD) of S _i as [ U _si D _si V _si ] = SVD( _Si ) , where U _si (and its orthonormal counterpart V _si ) are the rows included ( column) is a matrix whose singular values are singular-vectors corresponding to elements of the diagonal matrix D _si .

2. gamma-score (ratio of S _i 's largest and smallest singular values) Calculate as

3. If Gamma Score If is greater than the threshold, it is confirmed that a deviation has been detected.

The presence of multiple change points and heterogeneous temporary changes may lead to incorrect setting of the threshold. The proposed method can be used to define methods similar to the WDTW matrix-based method. Defines the WGamma matrix.

Here, each element is a gamma score calculated as follows:

here, is the singular value matrix calculated by SVD:

Inference: Based on the WGamma matrix, inference, fault detection and RA triggering can be identified using a heat map diagram. Elements with high WGamma are shown in light gray and elements with low WGamma are shown in dark gray. The proposed technique tracks light gray cells from the lowest resolution depth ( DI=1 ₎ to the highest ( DI = DImax ). The WGamma matrix technique can be used to establish a causal relationship between the solution task and the deviations in the time series. Under the assumption that the reverse causality will also be true (with some confidence) (i.e., if time series deviations from a given WOI (cause) are mapped to the 'required remedial action (effect)', the proposed technique can be used for continuous wafer processing. WOI can monitor time series data in real time and trigger ‘events’ when deviations occur, comparing them to previous heatmaps corresponding to the last batch of a continuous wafer process. Additionally, the values (deviation scores) of the indicated WOI(s) are merged and passed to the deviation score combiner and confidence engine. If no changes/OOS are observed, the sensor time series has a CPA property, approximately equal to 0. The value is passed. This score is defined as SW _Gamma .

The process below can detect changes on the time axis and divide time series data into multiple sub-steps.

1. Calculate the first difference ( ts' ) and second difference ( ts'' ) of the time series ( ts ).

2. Using the MAD estimate of the wavelet coefficients of the time series at the highest level of resolution, calculate a noise floor of three times the MAD estimate.

3. Detect the positions of the impulse peaks at ts ' and ts '' with values greater than the noise. The spread of the impulse can cause groups of adjacent values to appear. This can be solved by using the median of adjacent values as the location of the peak.

4. Separate the time series at the location of the detected peak as follows: where α, β, δ are the positions in the time axis of the impulse peaks at ts' and ts'' that mark the step/jump and velocity cusp at ts. Additionally, ts(p, q) represents the division of time series of indices p to q (including p, q).

Based on the statistical properties of the values of the time series at each sub-step, two processes can be applied to detect significant value changes before and after the solution operation. If the range of values is 'narrow', the proposed method can detect significant value changes using pseudo-clustering and FDI. If the range is 'wide', the proposed method utilizes the DTW+FDI based method to detect significant value changes. Time series substeps are classified into 'narrow' or 'wide' ranges based on the kurtosis of the values of a given substep (shown in Figure 9). If the kurtosis is greater than -1, the time series substep is classified as 'wide' range, otherwise it is classified as narrow range.

Similar Clustering and FDI Score Method: In the case of 'narrow' range features, the algorithm is described as below.

1. The values within the partitions ts _i ^B ( p,q ) and ts _i ^A ( p,q ) , the time series before and after the solution operation, are two separate one-dimensional clusters because each comes from the corresponding partition. ) - can be treated as a similar cluster - because grouping is inherent. If the 'placement' of similar clusters in one-dimensional space is significantly different, there is a deviation between the time series segmentation before and after repair.

2. The deviation score can be calculated by FDI as follows:

3. The score can also be obtained by arbitrary inter-cluster - intra-cluster distance measurement.

4. The higher the FDI score, the better the separation between similar clusters. if If so, the time series segments can be marked as significantly different (i.e., as influenced by the solution task). Terms and is the noise floor (standard deviation) estimated using, for example, the MAD-based estimation described above.

DTW +FDI based method: In case of 'wide' range characteristics, the DTW score can be used to detect whether two time series have significant deviations in slope. FDI score-based methods can still be applied for detection of bias induced deviation. Based on these points, the algorithm is described as follows.

One. We represent ts _i ^B ( p,q ) and ts _i ^A (p,q) as separate time series before and after the solution operation and calculate the DTW before them.

2. If the DTW score is higher than the threshold, there is a significant difference between the time series and a causal relationship can be established between the solution task performed and the time-series behavior during the appropriate substep. The thresholds described above can be learned and fixed from past data, or can be adjusted interactively. This method can be applied to detect tilt bias.

3. Calculate the FDI score similar to the steps described previously. if If so, this means the deviation derived from the bias.

Additionally, the DTW method cannot be applied to sub-steps of time series with a 'narrow' range, and the method cannot be used to detect bias/shifts in values - the main cause of deviation in time series with a 'narrow' range (e.g. RF power sensors). time series) - does not detect. 'Broad' range time series lower stages are also sensitive to slope changes/biases. The DTW method is best suited to differentiate these changes. In addition, FDI is used to detect deviations derived from bias.

Inference: If the reverse causal relationship between salient deviations and solution tasks is also true (with some certainty), then the time series deviations at a given substep (similar to WOI) can be mapped to the 'required solution tasks'. Therefore, the proposed method can be used for monitoring real-time time series data of continuous wafers/processes and triggering relevant remedial action events required when deviations occur. Detection of an occurrence of deviation (OOS) can be related to values obtained from analysis of previous successive wafer processes.

Course-I - Deviation score as follows: is passed to the deviation score combiner and confidence engine.

Process II - Integrated deviation score as follows: is passed to the deviation score combiner and confidence engine.

Sensor Measurements - Categorical Time Series : Figure 17 shows example categorical time series data. The proposed method describes the problem of narrowing the deviation to a specific resolution using a windowlet-based technique. Since all or part of the time series/sequence by category can be treated as an array of characters or a string as shown in FIG. 17, the deviation value is a string similarity metric - min-hash Technique - can be detected based on Consider that the proposed method can be used to define an extended time series formed by the summation/concatenation of 2 and 3 grams as follows:

here, Is and 2 grams of phosphorus, silver , and It is 3 grams. n-grams ( n=2, 3 ) preserve order information in calculating deviation values. For example, from Figure 17, if ramen, am. In the extended time series data of the ith sensor, the partition between the pth and qth samples (including p and q) is defined, respectively. and It is expressed as Similarity value calculated by min-hash and thus the deviation value is It is defined as Additionally, the window-let MH (Min-Hash) matrix is defined as follows:

here am.

Based on the WMH matrix, inference and fault detection/RA triggering can be obtained using a heatmap diagram.

Deviation Score Combinator: The numerical sensor time series data, namely S _WDTW , S _WGamma , S _FDI and S _{FDI - DTW} are combined by the deviation score combinator to form a deviation score vector of length 4: SD = [S _WDTW , S _WGamma , S _FDI and S _{FDI - DTW} ] This vector is passed to the confidence engine module for determining confidence-of-computed-score and calculating the final deviation score. Depending on how the category-specific time series measurements were obtained, the noise characteristics of the series/sequence are different. If categorical values are obtained and measured directly (e.g. switch on-off state), the measured noise can be treated as absent for all practical purposes. In contrast, if a numerical measurement is made and quantized to obtain a time series for each category, the quantization noise is virtually uniform. Because the set of values affected by this noise is uniform over the entire interval, the raw deviation score can be treated as the final score without needing to be confidently calculated. Therefore, S _WMH is provided directly to the sensor selection module for inference.

Confidence Engine: Confidence Engine is also referred to as fuzzy-logic based assurance engine. The fuzzy-logic based confidence engine takes into account the dependencies and dynamics of these deviation scores and calculates an overall, robust, reliable and usable confidence score for triggering resolution actions. The assurance engine includes an HDS unit and a FSG unit.

The HDS unit stores all historical data for analysis of sensor dynamics such as aging and long-term trends, outlier detection and exclusion, and local variation emmd. More precisely, the (local) confidence score is calculated from historical data and updated regularly (periodically or by manual triggering) by the HDS. These are the inputs to the FSG module for calculation of the final confidence score vector. For the calculation of local scores, it mainly consists of three units: sensor aggregation unit (α), long-term dynamics/aging model (β) and short-term dynamics model (γ).

The aggregation unit sorts ‘similar sensor’ data (if any) for robust analysis and functions as an outlier exclusion unit. 'Similarity' rules are determined based on equipment architecture/layout and sensor time series characteristics (e.g. correlation). This can be illustrated by the following example (taking the correlation as the 'similarity value'): a single thermostat placed at different positions in the chamber (front left, front right, rear left and rear right) for temperature control; Consider four temperature sensors with devices (actuators). For a given recipe under normal operating conditions, the time series data from all four sensors are 'highly' correlated. If a particular temperature sensor (e.g., rear left) is OOS (out of specification) based on its current deviation score, and the other temperature sensor's readings are not OOS, it is likely that the sensor (front left) is defective. . The weight may be determined in proportion to the correlation value between sensors. Correlation value comparison (high/low) is not based on absolute values, but is based on comparison to current values under normal circumstances. Additionally, the same results can be obtained using a thermostat input power monitor, if available. Based on the above-described model, the data flow of the aggregation module is as follows:

1. An initial level of aggregation is obtained (if feasible) by pair-wise string similarity (e.g. minhash, LSH, Levenstein edit distance) based measurements. Each sensor name and recipe name are concatenated to form a recipe-sensor name (string). Based on the similarity between the composite strings, the sensors (taking the recipe into account) are bundled. This ensures that certain temperature sensors are grouped together to a first approximation, but pressure sensors and temperature sensors are not. For example, recipe-sensor names: Recipe1_ChamberRearLeftTemperatureMonitorValue , Recipe1_ChamberRearRightTemperatureMonitorValue , Recipe1_ChamberFrontLeftTemperatureMonitorValue and Recipe1_ChamberFrontRightTemperatureMonitor are grouped and Recipe1_ChamberPressureMonitor, Recipe1_RFForwardPower are included in the formed group. It does n't work.

The final cluster/group of sensors in the feature space represents 'similar' sensors under typical operating conditions. This representation is quantified by calculating the centroid and ISID (ratio of the minimum of the intra-cluster distribution and the intra-cluster distance) for each cluster. This can be used as a basis for ensuring that the current sensor time series characteristics do not show any outlier-like measurements. For the current sensor time series features, based on their placement in feature space, the weights are calculated as follows:

Here, d(C, S) is the distance (e.g., Euclidean distance, absolute difference) between the current sensor time series and the center of the nearest cluster/group.

Alternatively, rule-based aggregation models derived from domain information (if available) can also be applied. In this scenario, scores are defined by an expert.

Aging Model/Long-Term Dynamics Model_: The aging model tracks long-term trends in the deviation scores of each sensor. In contrast to aggregate units whose operating domain is inter-sensor comparisons, aging models use intra-sensor comparisons. Unlike the aggregate unit score α derived for each sensor, the aging model score is derived according to the type of deviation (combined by the deviation score combiner) β _i where i = { WDTW , WGamma , FDI, FDI- DTW } (combined by the combiner) There are four types of deviation scores generated.) The aging model for each deviation score is calculated through regression analysis (e.g., non-linear) using historical data (previous values of the deviation score).

Consider the previous value of the deviation score as time series data with the sequence number of the wafer process as the time series index. To model long-term trends, the deviation scores of successive/adjacent wafer processes including short-term dynamics are not considered, but rather the 'decimated value' (the value of the selected P wafer process in the interval) is considered. can be, where the value of P can be determined based on an equipment (part) aging model.

In one embodiment of the invention, the average of the deviation scores for the P wafer process can be considered a single value in the time series for estimation of regression model parameters. Based on the regression model extrapolation, the current value of the deviation score is predicted. The predicted values are compared with the calculated values of the deviation scores for the currently considered successive wafer process pairs. Their differences (if any) are quantified as follows:

Here, i = { WDTW , WGamma , FDI, FDI- DTW } , and d( P _i , S _i ) is the distance between the predicted and calculated values of the ith deviation score (e.g. Euclidean distance, absolute distance etc.).

Short- term dynamics model: The short-term dynamics model quantifies short-term sensor deviation score variation. Quantification of dynamics is performed using the change in the ith deviation score and is referred to as γ _i ( i = {WDTW, WGamma, FDI, FDI-DTW} ). It is clear that γ _i has the range [ 0 ,∞). Although the probability of change is close to ∞, it could theoretically occur, and the probability of occurrence is close to 0 when considering well-controlled equipment operation. γ _i is determined (conditioned) by mapping high values to 1,0 so that the range is within [0,1]. This can be obtained by techniques such as histogram transformation techniques.

Fuzzy score generalization ( Fuzzy score - globalization ; FSG ) unit: The FSG unit is a fuzzy inference system (Fuzzy Inference) that receives α, β _i and γ _i as input and outputs global confidence C _i (( i = {WDTW, WGamma, FDI, FDI- DTW } ) Given the open-ended range and the continuous, not necessarily uniformly distributed nature of the scores, a crisp logic threshold is required to measure and represent/store/store the values. Additionally, most of the conversion rules from the local confidence score to the global score are unknown, with the full range of scores being low (L), medium (M), and high. Dividing by H), all low (L) scores correspond to low (L) overall confidence and all high (H) scores correspond to high overall confidence, but with β and γ in the range of H, which are in the range of M. Also, even if it is known, periodic adjustments may be necessary all of these factors make a crisp-logic decision unit infeasible. Given that ", a FIS system can interpolate over the entire range of a decision surface given a set of edge/boundary/center rules. This suggests a fuzzy logic based score generalization unit as opposed to a clear logic based unit. Motivation: The known basic rules are shown in Table 1. The entries in the table are for illustrative purposes (examples of rule sets) and may be changed based on knowledge in the field.

α β _i γ _i C _i L L L L L L H L L H L L L H H M H L L L H L H M H H L M H H L H

The proposed secure FSG based on score generalization unit is shown in Fig. 18 along with the proposed rule-based technique. The relationship of the FSG surface ( C _i variation) for α and β _i at γ _i = 0, 0.25, 0.5, 0.75 and 1.0, respectively, on a given rule basis is shown in Figure 19a. C i obtained from FSG is the overall confidence score. This is multiplied by each deviation score to give the final deviation score vector for a given sensor. provides.

Figure 19B is a schematic diagram of a system that provides recommendations for deviations within equipment based on video data. In one embodiment of the invention, video is converted into time series signal and text information. Time series signals are transmitted for B/A analysis and text information is transmitted for DRsG formation. The proposed system provides additional capabilities to index and can be used to search the video based on tagged text.

Figure 20 depicts a computing environment 2002 that implements mechanisms for root cause analysis of equipment-related deviations in accordance with an embodiment of the present invention. The computing environment (2002) includes a control unit (2004), at least one processing unit (2008) consisting of an Arithmetic Logic Unit (ALU, 2006), a memory (2010), a storage unit (2012), and a plurality of networking. It includes a device 2016 and a plurality of input/output devices 2014. The processing unit 2008 receives instructions from the control unit 2004 and performs processing. Additionally, any logical and arithmetic operations related to the execution of the above instructions are computed with the help of ALU (2006).

The entire computing environment 2002 may include a plurality of homogeneous or heterogeneous cores, a plurality of CPUs of different types, special media, and other accelerators. Processing unit 2008 processes instructions of the above techniques. Additionally, the plurality of processing units 2008 may be arranged on a single chip or across multiple chips.

The technology, including instructions and code required for implementation, is stored in the memory unit 2010 or the storage unit 2012 or both. Upon execution, instructions may be fetched from corresponding memory 2010 or storage 2012 and executed by processing unit 2008.

In the case of hardware implementation, various networking equipment 2016 or external input/output device 2014 may be connected to the computing environment 2002 to support implementation through the networking unit and the input/output device unit.

Therefore, the proposed mechanism wipes out the memory of the associated solution tasks when an unusual repair is performed, and then builds up the data when the state of the equipment returns to its original state in a gradual manner.

Those skilled in the art will understand that the above-described description of the present invention in the form of claims is exemplary and that many modifications, changes, changes, substitutions and equivalents may be made without departing from the spirit of the present invention. You will understand that you can. Additionally, those skilled in the art will understand that the use of terms is domain-specific and that the invention will vary depending on implementation in a particular industry.

Embodiments of the present invention may be implemented through at least one software program that runs on at least one hardware device and performs a network management function to control components. The components shown in FIGS. 1 to 20 include blocks that may be composed of at least one hardware device or a combination of a hardware device and a software module.

The description of the foregoing embodiments will reveal the general nature of the embodiments of the present invention, and may be easily modified and/or applied by others for various purposes without significantly departing from the general concept by applying current knowledge, and thus such applications and modifications may be made. It should be understood within the scope of meaning and equivalent to the embodiments of the present invention. It is to be understood that the phraseology and terminology employed herein is for the purpose of description and not of limitation. Therefore, although the preferred embodiments of the present invention have been described, those skilled in the art will understand that the embodiments of the present invention may be practiced with modification within the spirit and scope described.

100: System 102: SeRA entity
104: DRG entity 106: Recommended entity
108: ComET 200: Equipment

Claims (32)

a Sensor Resolution Action (SeRA) entity that detects deviations associated with at least one piece of equipment;
A DRG entity that resolves deviations based on at least one Deviation Resolution Graph (DRG) by determining at least one Deviation Resolution sub-Graph (DRsG), wherein the at least one DRsG is at least one A DRG entity comprising a resolution action of and at least one observation related to the at least one piece of equipment; and
A system for root cause analysis of a deviation associated with at least one piece of equipment, comprising a recommender entity that recommends the at least one DRsG to resolve the deviation associated with the at least one piece of equipment. According to clause 1,
A system for root cause analysis of deviations related to at least one piece of equipment, wherein the observation is one of an original observation and a secondary observation. According to clause 1,
A system for root cause analysis of a deviation associated with at least one piece of equipment, wherein the corrective action includes one of a repair action, a secondary action, a corrective action, and a reset action. According to clause 1,
The DRG is formed based on data received from a plurality of sources, wherein the data is at least one of structured data and unstructured data. A system for cause analysis. According to clause 1,
A system for root cause analysis of a deviation associated with at least one device, wherein the deviation is identified based on a SeRA graph, wherein the SeRA graph represents a relationship between a group of devices and the at least one DRG. According to clause 5,
A system for root cause analysis of deviations associated with at least one device, wherein the SeRA graph is formed based on a before-after analysis of a DRsG set of the at least one device. According to clause 6,
The pre- and post-analysis identifies the at least one piece of equipment that contains a signature of a repair operation, forming a causal relationship that is stored for automated recommendation of the at least one DRsG. A system for root cause analysis of deviations related to . According to clause 7,
The causal relationship that recommends the at least one DRsG is:
Calculate, by a before/after analysis engine, a deviation score based on whether a time series signal of the at least one device includes a change point,
By a confidence engine, determine a confidence score based on the deviation score,
Determine a deviation vector based on the confidence score by a fuzzy score globalizer,
Root cause analysis of a deviation associated with at least one piece of equipment, formed by a sensor selector, establishing a causal relationship between the deviation and the at least one piece of equipment based on the SeRA graph. system for. delete delete delete delete delete delete delete According to clause 1,
A system for root cause analysis of deviations associated with at least one equipment, further comprising a Comprehensive Elucidation Table (ComET) storing SeRA graphs and DRGs. detecting, by a Sensor Resolution Action (SeRA) entity, a deviation associated with at least one piece of equipment;
By a Deviation Resolution Graph (DRG) entity, determine at least one Deviation Resolution sub-Graph (DRsG) to resolve the deviation based on the at least one DRG, wherein the at least one DRsG includes at least one solution task and at least one observation associated with said at least one device,
A method for root cause analysis of a deviation associated with at least one equipment, comprising recommending, by a recommending entity, the at least one DRsG to resolve the deviation associated with the at least one equipment. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete