US8412356B2 - Methods and apparatus for automated predictive design space estimation - Google Patents
Methods and apparatus for automated predictive design space estimation Download PDFInfo
- Publication number
- US8412356B2 US8412356B2 US13/336,623 US201113336623A US8412356B2 US 8412356 B2 US8412356 B2 US 8412356B2 US 201113336623 A US201113336623 A US 201113336623A US 8412356 B2 US8412356 B2 US 8412356B2
- Authority
- US
- United States
- Prior art keywords
- input
- range
- values
- input values
- performance metric
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013461 design Methods 0.000 title claims abstract description 155
- 238000000034 method Methods 0.000 title claims abstract description 145
- 230000004044 response Effects 0.000 claims abstract description 320
- 230000008569 process Effects 0.000 claims abstract description 56
- 238000004590 computer program Methods 0.000 claims abstract description 19
- 238000009826 distribution Methods 0.000 claims description 37
- 238000004519 manufacturing process Methods 0.000 claims description 31
- 238000000342 Monte Carlo simulation Methods 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 9
- 239000004065 semiconductor Substances 0.000 claims description 6
- 238000003860 storage Methods 0.000 claims description 4
- 239000000446 fuel Substances 0.000 description 28
- 239000004071 soot Substances 0.000 description 18
- NBIIXXVUZAFLBC-UHFFFAOYSA-K phosphate Chemical compound [O-]P([O-])([O-])=O NBIIXXVUZAFLBC-UHFFFAOYSA-K 0.000 description 13
- 238000004364 calculation method Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 11
- 238000002474 experimental method Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 239000000203 mixture Substances 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000005259 measurement Methods 0.000 description 5
- 238000013400 design of experiment Methods 0.000 description 4
- 239000007789 gas Substances 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000010925 quality by design Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- MWUXSHHQAYIFBG-UHFFFAOYSA-N Nitric oxide Chemical compound O=[N] MWUXSHHQAYIFBG-UHFFFAOYSA-N 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- GUBGYTABKSRVRQ-QKKXKWKRSA-N Lactose Natural products OC[C@H]1O[C@@H](O[C@H]2[C@H](O)[C@@H](O)C(O)O[C@@H]2CO)[C@H](O)[C@@H](O)[C@H]1O GUBGYTABKSRVRQ-QKKXKWKRSA-N 0.000 description 2
- 229910019142 PO4 Inorganic materials 0.000 description 2
- 239000013543 active substance Substances 0.000 description 2
- 239000001913 cellulose Substances 0.000 description 2
- 229920002678 cellulose Polymers 0.000 description 2
- 239000008101 lactose Substances 0.000 description 2
- 238000012417 linear regression Methods 0.000 description 2
- 238000007620 mathematical function Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000010452 phosphate Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- VCUVETGKTILCLC-UHFFFAOYSA-N 5,5-dimethyl-1-pyrroline N-oxide Chemical compound CC1(C)CCC=[N+]1[O-] VCUVETGKTILCLC-UHFFFAOYSA-N 0.000 description 1
- GUBGYTABKSRVRQ-XLOQQCSPSA-N Alpha-Lactose Chemical compound O[C@@H]1[C@@H](O)[C@@H](O)[C@@H](CO)O[C@H]1O[C@@H]1[C@@H](CO)O[C@H](O)[C@H](O)[C@H]1O GUBGYTABKSRVRQ-XLOQQCSPSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013401 experimental design Methods 0.000 description 1
- 238000010348 incorporation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003134 recirculating effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000010206 sensitivity analysis Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
Definitions
- the present invention relates generally to computer-based methods and apparatuses, including computer program products, for automated predictive design space estimation.
- QbD quality by design
- QbD refers to a systematic process to build quality into a product from the inception to the final output.
- QbD refers to the level of effectiveness of a design function in determining a product's operational requirements (and their incorporation into design requirements) that can be converted into a finished product in a production process.
- This is often referred to as a design space, which is the multidimensional combination and interaction of input factors (e.g., material attributes) that have been demonstrated to provide assurance of quality.
- input factors e.g., material attributes
- a design space is the set of possible designs and design parameters (input factors) that meet a specific product requirement (one or more output responses). Exploring a design space requires evaluating the various design options possible with a given technology and optimizing the input factors and output responses with respect to specific constraints (e.g., power, cost, mixture design).
- a typical process tool used in current semiconductor manufacturing can be described by a set of several thousand process variables.
- the variables are generally related to physical parameters of the manufacturing process and/or tools used in the manufacturing process.
- several hundred variables are dynamic (e.g., changing in time during the manufacturing process or between manufacturing processes).
- the dynamic variables e.g., gas flow, gas pressure, delivered power, current, voltage, and temperature
- change based on various operating factors e.g., a specific processing recipe, the particular step or series of steps in the overall sequence of processing steps, errors and faults that occur during the manufacturing process or changes (e.g., referred to as “drift”) in parameter values based on use of a particular tool or chamber).
- Operators can control the manufacturing process by, for example, changing input factors, which are variables that influence the production process.
- output response values e.g., temperature, yield, quality attributes
- Experiments can be performed for the particular manufacturing process to determine what combinations of input factor values result in acceptable output response values.
- a Design of Experiment (DOE) method is a structured, organized method for determining the relationship between input factors for a process (e.g., a manufacturing process, mixture design) and the output responses of that process.
- the DOE method can quantify indeterminate measurements of input factors and interactions between the input factors statistically through observing the results of methodical changes of the input factors. If there are operational criteria associated with the process, output response values are measured for different combinations of input factors to determine if the operational criteria are satisfied for each combination.
- Exploring a design space requires manually evaluating the various design options possible based on any operational criteria. There is no way to automatically provide for acceptable regions of variability of each input factor, or to predict result regions. The analysis process is further complicated when several demands on the output responses have to be met at the same time with different types of constraints. Further, it would be desirable for graphical methods to display design spaces in a way that can be easily analyzed and interpreted by an operator.
- the invention features a computerized method for estimating a design space of input factors and output responses for a physical process.
- a physical process can be an industrial process, a manufacturing process, a semiconductor process, an analytical system or procedure, or a design project.
- the method includes receiving, via a processor, data for one or more input factors for a physical process, one or more output responses for the process, and criteria.
- the criteria includes a target response criterion for each of the one or more output responses, an estimated optimal value for each of the one or more input factors to achieve the target response criteria, and an experimented range of input values for each of the one or more input factors, where each input value in the experimented range of input values is determinative of an output response value for each of the one or more output responses.
- the method includes calculating, using the processor, for each of the one or more input factors, a calculated range of input values within the corresponding experimented range of input values.
- Calculating comprises selecting a first range of input values for each of the one or more input factors, predicting values of the one or more output responses based on the first range of input values to determine a first performance metric for each of the one or more output responses, and creating the calculated range of input values for each of the one or more input factors by adjusting the first range of input values based on a comparison of the first performance metric and a predetermined performance metric.
- the method includes calculating, using the processor, a modified range of input values for each of the one or more input factors.
- the method For each input factor of the one or more input factors, the method includes selecting a second range of input values by expanding the calculated range of input values by a predetermined percentage, predicting values of the one or more output responses based on the second range of input values to determine a second performance metric for each of the one or more output responses, and creating the modified range of input values for the input factor by adjusting the second range of input values based on a comparison of the second performance metric and the predetermined performance metric.
- the method includes predicting, using the processor, a design space estimate based at least on the modified ranges of input values, wherein the modified ranges of input values each comprise a largest region of variability where the criteria are fulfilled for one or more of the input factors.
- the design space estimate further comprises a distribution of output response values for each of the one or more output responses, wherein the modified ranges of input values are determinative of the distribution of output response values.
- a prespecified number of output response values within the distribution of output response values can be within predetermined limit values for the output response if the predetermined performance metric is satisfied.
- predicting the design space estimate includes predicting an individual largest region of variability for each input factor while any remaining input factors are set to their associated estimated optimal value. Predicting the design space estimate can include predicting a combined largest region of variability for each input factor, wherein values of each of the one or more input factors can be anywhere within the corresponding combined largest region of variability and satisfy the predetermined performance metric for each output response.
- creating the calculated range of input values includes selecting a third range of input values by reducing the first range of input values by a predetermined percentage if the first performance metric is below the predetermined performance metric.
- the method can include predicting values of the one or more output responses based on the third range of input values to determine a third performance metric for each of the one or more output responses, and creating the calculated range of input values for each of the one or more input factors by adjusting the third range of input values based on a comparison of the third performance metric and the predetermined performance metric.
- creating the calculated range of input values includes the following steps if the first performance metric is above the predetermined performance metric: setting a step size to a predetermined percentage of a size of the second range of input values, selecting a third range of input values by expanding the first range of input values based on the step size, predicting values of the one or more output responses based on the third range of input values to determine a third performance metric for each of the one or more output responses, and creating the calculated range of input values for each of the one or more input factors by adjusting the third range of input values based on a comparison of the third performance metric and the predetermined performance metric.
- creating the calculated range of input values includes the following steps if the third performance metric is below the predetermined performance metric: selecting a fourth range of input values by expanding the third range of input values based on the step size, predicting values of the one or more output responses based on the fourth range of input values to determine a fourth performance metric for each of the one or more output responses, and creating the calculated range of input values for each of the one or more input factors by adjusting the fourth range of input values based on a comparison of the fourth performance metric and the predetermined performance metric.
- Creating the calculated range of input values can include the following steps if the third performance metric is above the predetermined performance metric: setting the third range of input values equal to the first range of input values, reducing the step size by a predetermined percentage, selecting a fourth range of input values by expanding the third range of input values based on the step size, predicting values of the one or more output responses based on the fourth range of input values to determine a fourth performance metric for each of the one or more output responses, and creating the calculated range of input values for each of the one or more input factors by adjusting the fourth range of input values based on a comparison of the fourth performance metric and the predetermined performance metric.
- creating the modified range of input values includes the following steps if the second performance metric is above the predetermined performance metric: setting the modified range of input values equal to the second range of input values, and associating a completion flag with the input factor.
- Creating the modified range of input values can include the following steps if the second performance metric is below the predetermined performance metric: selecting a third range of input values by expanding the second range of input values by a step size, wherein the step size was used to select the second range of input values, predicting values of the one or more output responses based on the third range of input values to determine a third performance metric for each of the one or more output responses, and creating the modified range of input values for the input factor by adjusting the third range of input values based on a comparison of the third performance metric and the predetermined performance metric.
- calculating the modified range of input values for each of the one or more input factors further includes the following steps: determining each of the one or more input factors are associated with a completion flag, reducing a step size used to select the second range of input values by a predetermined percentage, and for each of the one or more input factors of the one or more input factors: selecting a third range of input values by expanding the second range of input values by the step size, predicting values of the one or more output responses based on the third range of input values to determine a third performance metric for each of the one or more output responses, and creating the modified range of input values for the input factor by adjusting the third range of input values based on a comparison of the third performance metric and the predetermined performance metric.
- predicting the values of the one or more output responses includes performing a Monte Carlo simulation, and the first performance metric, the second performance metric, and the predetermined performance metric are a measure of the number of values of the one or more output responses within predetermined limit values.
- Selecting the first range of input values for each of the one or more input factors can include setting the first range of input values to a predetermined percentage of a lower outer bound value of the corresponding input factor.
- the method includes the following steps, receiving a constraint for the one or more input factors, and predicting values of the one or more output responses based on the first range of input values further comprises adjusting the prediction to compensate for the constraint.
- the constraint can include a user defined limit on one or more input factors, a factor distribution, or any combination thereof.
- the target response criterion for each of the one or more output responses can include a target response value, a range of acceptable target response values, or a critical target response value, where the output response can either be above or below the critical target response value.
- the physical process comprises a manufacturing process, an industrial process, a design project, a semiconductor project, or any combination thereof.
- the invention in another aspect, features an apparatus for estimating a design space of input factors and output responses for a physical process.
- the apparatus includes a processor configured to receive data for one or more input factors for a physical process, one or more output responses for the process, and criteria.
- the criteria includes a target response criterion for each of the one or more output responses, an estimated optimal value for each of the one or more input factors to achieve the target response criteria, and an experimented range of input values for each of the one or more input factors, where each input value in the experimented range of input values is determinative of an output response value for each of the one or more output responses.
- the processor is further configured to calculate, for each of the one or more input factors, a calculated range of input values within the corresponding experimented range of input values.
- Calculating includes selecting a first range of input values for each of the one or more input factors, predicting values of the one or more output responses based on the first range of input values to determine a first performance metric for each of the one or more output responses, and creating the calculated range of input values for each of the one or more input factors by adjusting the first range of input values based on a comparison of the first performance metric and a predetermined performance metric.
- the processor is further configured to calculate a modified range of input values for each of the one or more input factors.
- Calculating includes, for each input factor of the one or more input factors, selecting a second range of input values by expanding the calculated range of input values by a predetermined percentage, predicting values of the one or more output responses based on the second range of input values to determine a second performance metric for each of the one or more output responses, and creating the modified range of input values for the input factor by adjusting the second range of input values based on a comparison of the second performance metric and the predetermined performance metric.
- the processor is further configured to predict a design space estimate based at least on the modified ranges of input values, wherein the modified ranges of input values each comprise a largest region of variability where the criteria are fulfilled for one or more of the input factors.
- the invention in another aspect, features a computer program product, tangibly embodied in an information carrier.
- the computer program product includes instructions being operable to cause a data processing apparatus to receive data for one or more input factors for a physical process, one or more output responses for the process, and criteria.
- the criteria includes a target response criterion for each of the one or more output responses, an estimated optimal value for each of the one or more input factors to achieve the target response criteria, and an experimented range of input values for each of the one or more input factors, where each input value in the experimented range of input values is determinative of an output response value for each of the one or more output responses.
- the computer program product further includes instructions being operable to cause a data processing apparatus to calculate, for each of the one or more input factors, a calculated range of input values within the corresponding experimented range of input values. Calculating includes selecting a first range of input values for each of the one or more input factors, predicting values of the one or more output responses based on the first range of input values to determine a first performance metric for each of the one or more output responses, and creating the calculated range of input values for each of the one or more input factors by adjusting the first range of input values based on a comparison of the first performance metric and a predetermined performance metric.
- the computer program product further includes instructions being operable to cause a data processing apparatus to calculate a modified range of input values for each of the one or more input factors.
- Calculating includes, for each input factor of the one or more input factors, selecting a second range of input values by expanding the calculated range of input values by a predetermined percentage, predicting values of the one or more output responses based on the second range of input values to determine a second performance metric for each of the one or more output responses, and creating the modified range of input values for the input factor by adjusting the second range of input values based on a comparison of the second performance metric and the predetermined performance metric.
- the computer program product further includes instructions being operable to cause a data processing apparatus to predict a design space estimate based at least on the modified ranges of input values, wherein the modified ranges of input values each comprise a largest region of variability where the criteria are fulfilled for one or more of the input factors.
- the invention in another aspect, features a system for estimating a design space of input factors and output responses for a physical process.
- the system includes means for receiving data for one or more input factors for a physical process, one or more output responses for the process, and criteria.
- the criteria include a target response criterion for each of the one or more output responses, an estimated optimal value for each of the one or more input factors to achieve the target response criteria, and an experimented range of input values for each of the one or more input factors, where each input value in the experimented range of input values is determinative of an output response value for each of the one or more output responses.
- the system further includes means for calculating, for each of the one or more input factors, a calculated range of input values within the corresponding experimented range of input values.
- Calculating includes selecting a first range of input values for each of the one or more input factors, predicting values of the one or more output responses based on the first range of input values to determine a first performance metric for each of the one or more output responses, and creating the calculated range of input values for each of the one or more input factors by adjusting the first range of input values based on a comparison of the first performance metric and a predetermined performance metric.
- the system further includes means for calculating a modified range of input values for each of the one or more input factors.
- Calculating includes, for each input factor of the one or more input factors, selecting a second range of input values by expanding the calculated range of input values by a predetermined percentage, predicting values of the one or more output responses based on the second range of input values to determine a second performance metric for each of the one or more output responses, and creating the modified range of input values for the input factor by adjusting the second range of input values based on a comparison of the second performance metric and the predetermined performance metric.
- the system further includes means for predicting a design space estimate based at least on the modified ranges of input values, wherein the modified ranges of input values each comprise a largest region of variability where the criteria are fulfilled for one or more of the input factors.
- the techniques which include both methods and apparatuses, described herein can provide the advantages of automating a process for defining a largest region of variability for each input factor, and predicting a distribution of output response values for each output response with a given level of probability.
- the largest regions of variability and the distributions of output response values can be presented in a way to incorporate all the different criteria under consideration in an efficient and correct way.
- the principles described herein can handle many output responses simultaneously with different prediction models for design space estimates, which can be handled in combination with expansion criteria. Additionally, constraints for the input factors and/or the output responses can be incorporated into the prediction models.
- FIG. 1 is a schematic illustration of a system for predicting an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- FIG. 2 is a flow chart illustrating a method for predicting an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- FIG. 3A is a flow chart illustrating a method for calculating a calculated range of input values, according to an illustrative embodiment of the invention.
- FIG. 3B is a flow chart illustrating a method for a stepwise region expansion for calculating a calculated range of input values, according to an illustrative embodiment of the invention.
- FIG. 3C is a flow chart illustrating a method for a stepwise region expansion for calculating a calculated range of input values, according to an illustrative embodiment of the invention.
- FIG. 4A is a flow chart illustrating a method for calculating a modified range of input values, according to an illustrative embodiment of the invention.
- FIG. 4B is a flow chart illustrating a method for a stepwise region expansion for calculating a modified range of input values, according to an illustrative embodiment of the invention.
- FIG. 5A is a diagram illustrating a graphical display for displaying an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- FIG. 5B is a diagram illustrating a graphical display for configuring parameters used to calculate an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- FIG. 5C is a diagram illustrating a graphical display for displaying an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- FIG. 5D is a diagram illustrating a graphical display for displaying an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- FIG. 5E is a diagram illustrating a graphical display for displaying an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- a predictive design space estimate is determined for a physical process (e.g., a manufacturing process, industrial process, or a design project) based on input factors, output responses, and various criteria associated with the input factors and/or the output responses.
- the PDSE includes a distribution of output response values for each of the one or more output responses.
- the PDSE also includes one or more largest regions of variability for each input factor.
- the PDSE includes an individual largest region of variability and a combined largest region of variability for each input factor, both of which are calculated based on the distribution of output response values to satisfy the criteria associated with the input factors and/or the output responses.
- FIG. 1 is a schematic illustration of a system 100 for performing automated predictive design space estimate, according to an illustrative embodiment of the invention.
- the system includes a data input unit 102 .
- the data input unit 102 transmits data to the design space estimate module 104 .
- the design space estimate module 104 includes a receiver 106 .
- the receiver 106 is in communication with a processor 108 .
- the receiver 106 and the processor 108 are in communication with a database 110 .
- the processor 108 is in communication with a control unit 112 .
- the processor 108 is also in communication with a display 114 .
- a user inputs data (e.g., input factor data and output response data) for a process through the data input unit 102 (e.g., a keyboard or a computer), which the design space estimate module 104 uses to calculate the PDSE.
- data input unit 102 e.g., a keyboard or a computer
- starting criteria are configured through the control unit 112 , which are used by the design space estimate module 104 while calculating the PDSE.
- the starting criteria include, for example, a target response criterion for each of the one or more output responses.
- the target response criterion specifies the desired outcome for each of the one or more output responses.
- the design space estimate module 104 designs the PDSE so that the largest region of variability for each of the input factors best satisfies the target response criterion of 220 mg/stroke.
- the target response criteria is a target response value (e.g., fuel consumption equals 220 mg/stroke), a range of acceptable target response values (e.g., fuel consumption between 200 mg/stroke and 240 mg/stroke), or a critical response value, where the output response can either be above or below the critical target response value (e.g., either above or below 220).
- the design space estimate module 104 retrieves data from the database 110 instead of directly from a user.
- data acquired from a user input (e.g., the data input unit 102 ) or from a computer system is stored in the database and subsequently provided to the receiver to allow a user to store data for later use by the design space estimate module 104 .
- a number of experiments are run for the physical process (e.g., industrial process, manufacturing process, or design project) before generating the PDSE.
- the experiments can be designed by application specialists who can determine what input factors and output responses should be included in the design. These experiments are used to test the relationship between the input factors and the output responses for the system. For example, the experiments can run the process or problem for various combinations of input factor values to determine what output response values are generated by the combinations. This data provides known outcomes for the output responses based on known combinations of the input factors.
- a user inputs the experimental data through the data input unit.
- the design space estimate module 104 preprocesses the data received by the receiver 106 .
- the design space estimate module 104 calculates an estimated optimal value for each of the one or more input factors that achieves the target response criteria. For example, the calculation can be carried out via a simplex search. Or the design space estimate module 104 can use a penalty function.
- the design space estimate module 104 sets constraints for the output responses (e.g., a target and a high/low limit). For each set of results for a particular combination of input factor values, the penalty term grows when the constraints are violated, or achieves 0 in the region where the constraints are not violated.
- the design space estimate module 104 is configured to calculate a PDSE with multiple output responses. Further, the design space estimate module 104 can calculate one PDSE model across the output responses, or calculate multiple PDSE models across the output responses.
- the preprocessed data only provides one estimated optimal value for each of the one or more input factors that achieve the target response criterion.
- the design space estimate module 104 using the estimated optimal values in combination with the other data for the process (e.g., the input factor values, output response values, and constraints), calculates a largest region of variability for the one or more input factors where the criteria are fulfilled (the PDSE). For example, if a user would like to run a process at a known temperature, the largest region of variability for the PDSE provides the tolerance ranges for the input factors (e.g., pressure, gas flow).
- the input factors e.g., pressure, gas flow
- the data input unit 102 , the design space estimate module 104 , and the control unit 112 can be implemented in digital electronic circuitry, in computer hardware, firmware, and/or software implemented with a processor.
- the implementation can, for example, be a programmable processor, a computer, and/or multiple computers. While FIG. 1 illustrates the data input unit 102 , the design space estimate module 104 , and the control unit 112 as separate components, one skilled in the art can appreciate that the functionality of one or more of these components can be combined.
- FIG. 2 is a flow chart illustrating a computerized method 200 for performing automated predictive design space estimate, according to an illustrative embodiment of the invention.
- Step 202 involves receiving data (e.g., via the processor 108 ).
- the data includes data for one or more input factors for a manufacturing or industrial process, one or more output responses for the process, and criteria.
- the criteria includes a target response criterion for each of the one or more output responses, an estimated optimal value for each of the one or more input factors to achieve the target response criteria, and an experimented range of input values for each of the one or more input factors, where each input value in the experimented range of input values is determinative of an output response value for each of the one or more output responses.
- the design space estimate module 104 calculates a largest region of variability for each of the one or more input factors.
- the design space estimate module 104 calculates (e.g., using the processor 108 ), for each of the one or more input factors, a calculated range of input values within the corresponding experimented range of input values.
- the design space estimate module 104 calculates (e.g., using the processor 108 ) a modified range of input values for each of the one or more input factors.
- the design space estimate module 104 predicts a design space estimate based at least on the modified ranges of input values, wherein the modified ranges of input values each comprise a largest region of variability for one or more of the input factors where the criteria are fulfilled.
- the largest region of variability is calculated with a stepwise expansion of a percentage (e.g., 2.5%) of the input factor range.
- the input factor range is the region of values of each input factor for which an experiment was conducted as described above.
- the design space estimate module 104 expands the input factor range by a percentage and performs Monte Carlo simulations to predict output response values.
- the design space estimate module 104 stops the stepwise expansion of the factor range when the predicted output response values are met for one or more of the output responses.
- the stepwise expansion is done for each of the one or more input factors on a one-by-one basis.
- the calculated largest region of variability for each input factor is used as the starting criteria for the search (steps 206 and 208 ) of the combined largest region of variability of the input factors where all input factors can vary and still fulfill the output response specifications.
- Steps 206 and 208 calculate the combined largest region of variability of the input factors where all input factors can vary and still fulfill the output response specifications.
- the design space estimate module 104 calculates the calculated range of input values for all the input factors together (simultaneously). Step 206 is described in further detail with reference to FIG. 3A .
- the design space estimate module 104 calculates the modified range of input values for each input factor on a one-by-one basis. Step 208 is described in further detail with reference to FIG. 3B .
- the design space estimate module 104 can use different mathematical regression models for each output response to estimate the largest range of variability (e.g., for all input factors individually or for all of the input factors in combination, the PDSE) with Monte Carlo simulations.
- the design space estimate module 104 automatically calculates the PDSE using prespecified criteria.
- the criteria include, for example, specific distributions (e.g., a normal distribution or a triangular distribution), user set criteria on input factors and/or output responses (e.g., user defined limits on specific input factors, user defined input factor distributions), technical constraints (e.g., for mixture designs, the sum of all input factors should always equal one).
- constraints such as qualitative input factor settings (e.g., “good” and “bad”) or discrete input factor settings (e.g., “low”, “medium”, or “high”), can not be processed (e.g., the qualitative input factors can not be randomized as required for the repeated computations of Monte Carlo simulations) in the same way as a continuous input factor.
- Functions can be included to perform calculations (e.g., Monte carlo simulations) with random settings of qualitative input factors.
- An exemplary function is a random setting for the qualitative input factor, where the distribution can be set to specific probabilities (e.g., 60% of A, 30% of B, and 10% of C) as a constraint (e.g., by assuming that the random population of discrete settings for the input factor has the final distribution set by the constraint).
- the design space estimate module 104 normalizes the input factors each time after the random variation is added for the Monte Carlo simulation so the sum of all the input factors equals a constant (e.g., one).
- FIG. 3A is a flow chart illustrating a method 300 for calculating a calculated range of input values, according to an illustrative embodiment of the invention.
- the design space estimate module 104 selects a first range of input values for each of the one or more input factors.
- the design space estimate module 104 predicts values of the one or more output responses based on the first range of input values to determine a first performance metric for each of the one or more output responses.
- the design space estimate module 104 creates the calculated range of input values for each of the one or more input factors by adjusting the first range of input values based on a comparison of the first performance metric and a predetermined performance metric.
- the design space estimate module 104 sets the first range to a percentage (e.g., 75%) of the largest region of variability for each input factor.
- the design space estimate module 104 performs Monte Carlo simulations with a number of samples specified by the user (e.g., a setting stored in the database 110 ). Monte Carlo simulations, or methods, are computational methods that rely on repeated random sampling to compute their results to simulate physical (and mathematical) systems. Monte Carlo simulations are often used when it is unfeasible or impossible to compute an exact result with a deterministic algorithm. Generally, a Monte Carlo simulation involves first defining a domain of possible inputs (e.g., input factors for a physical process).
- Inputs are generated randomly from the domain (the physical process).
- a deterministic computation is performed using the inputs (e.g., comparing the output response values to the desired criteria for the output response values).
- the results of the individual deterministic computations are aggregated into the final result (e.g., the PDSE).
- the first performance metric is a criterion called a dots per million operations (DPMO).
- the DPMO is the number of calculated output response values that are outside the specification limit range for a particular output response.
- the specification limit range (which can be defined by a user) begins at a lower output response value and ends at an upper output response value, defining a range of acceptable output response values for an output response.
- a random error is added to each input factor based on the selected distribution for the Monte Carlo simulation (e.g., a normal distribution, a uniform distribution, or a triangular distribution) and ranges (e.g., the first range) to improve the robustness of the Monte Carlo simulation.
- a random error based on the Monte Carlo model error is added to each predicted output response value (e.g., a Student's t-distribution random error).
- FIG. 3B is a flow chart illustrating a portion of the method 300 for a stepwise region expansion for calculating the calculated range of input values, according to an illustrative embodiment of the invention.
- the first performance metric is compared to the predetermined performance metric.
- the method 300 proceeds to step 324 .
- the design space estimate module 104 Select a second range of input values by reducing the first range of input values by a predetermined percentage. For example, if the DPMO is larger than the specified DMPO limit, the first range is divided by two. After selecting the second range of input values, method 300 proceeds to step 304 of FIG. 3A .
- the design space estimate module 104 sets a step size to a predetermined percentage of a size of the first range of input values.
- the design space estimate module 104 predicts values of the one or more output responses based on the second range of input values to determine a second performance metric for each of the one or more output responses.
- the design space estimate module 104 creates the calculated range of input values for each of the one or more input factors by adjusting the third range of input values based on a comparison of the third performance metric and the predetermined performance metric.
- the design space estimate module 104 can set the step size to 20% of the first range of input values, selecting the second range of input values by expanding the first range by the step size.
- the design space estimate module 104 can perform Monte Carlo simulations as described above with reference to step 304 of FIG. 3A to determine a DPMO for each of the one or more output responses.
- the calculated range is created by adjusting the second range of input values based on a comparison of the second performance metric and the predetermined performance metric as described in FIG. 3C .
- FIG. 3C is a flow chart illustrating a portion of the method 300 of FIG. 3A for a stepwise region expansion for calculating a calculated range of input values, according to an illustrative embodiment of the invention.
- the design space estimate module 104 determines whether the second performance metric equal the predetermined metric. If the second performance metric equals the predetermined metric, the method 300 proceeds to step 354 .
- the design space estimate module 104 sets the calculated range of input values to the second range of input values.
- the method 300 proceeds to step 356 .
- the design space estimate module 104 determines whether the method has reached a maximum number of iterations. If the maximum number of iterations has been reached, the method proceeds to step 354 and sets the calculated range of input values to the second range of input values. If the maximum number of iterations has not been reached, the method proceeds to step 358 . Steps 352 and 356 are termination criteria to prevent the method 300 of FIGS. 3A-3C from infinitely looping.
- the design space estimate module 104 determines if the second performance metric is below the predetermined performance metric.
- the method 300 proceeds to step 328 of FIG. 3B . If the second performance metric is not below the predetermined performance metric, the method proceeds to step 360 .
- the design space estimate module 104 sets a third range of input values equal to the last valid range of input values (the last range of input values that was associated with a performance metric below the predetermined performance metric).
- the design space estimate module 104 reduces the step value by a predetermined percentage. The method 300 continues to box 328 of FIG. 3B , selecting a fourth range of the input values by expanding the third range of input values based on the step size, and so on.
- the design space estimate module 104 continues to expand and test the range until one or more of the termination criteria are satisfied (steps 352 and 356 ). If the DPMO is higher than the DPMO limit, the design space estimate module 104 shrinks the factor range back to the last valid setting (step 360 ) (e.g., the first range), divides the step size by two (step 362 ), and continues to expand and test the range until one or more the termination criteria are satisfied (steps 328 of FIG. 3B and on as described with reference to FIG. 3B ). Method 300 terminates either when the calculated range achieves the desired DPMO for the Monte Carlo simulations of the calculated range, or the maximum number of iterations occurs, resulting on the best calculated range achievable for expanding all of the one or more input factors at the same time.
- FIG. 4A is a flow chart illustrating a method 400 for calculating a modified range of input values, according to an illustrative embodiment of the invention.
- the design space estimate module 104 selects an input factor from the one or more input factors.
- the design space estimate module 104 sets a step size to a predetermined percentage of a size of the calculated range of input values.
- the design space estimate module 104 selects a second range of input values by expanding the calculated range based on the step size.
- the design space estimate module 104 predicts values of the one or more output responses based on the second range of input values to determine a second performance metric for each of the one or more output responses.
- the design space estimate module 104 creates the modified range of input values for the input factor by adjusting the second range of input values based on a comparison of the second performance metric and the predetermined performance metric.
- the modified range is created based on a comparison of the second performance metric and the predetermined performance metric as shown in FIG. 4B .
- FIG. 4B is a flow chart illustrating a portion of the method 400 for a stepwise region expansion for calculating a modified range of input values, according to an illustrative embodiment of the invention.
- the design space estimate module 104 determines whether the second performance metric equals the predetermined metric. If the second performance metric equals the predetermined metric, the method proceeds to step 452 .
- the design space estimate module 104 sets the modified range equal to the second range of input values. If the second performance metric does not equal the predetermined metric, the method proceeds to step 454 .
- the design space estimate module 104 determines whether method 400 has reached a maximum number of iterations. If the maximum number of iterations has been reached, method 400 proceeds to step 452 and sets the modified range to the second range of input values. If the maximum number of iterations has not been reached, the method proceeds to step 456 . Steps 450 and 454 are termination criteria to prevent the method 400 of FIGS. 4A-4B from infinitely looping.
- the design space estimate module 104 determines whether the second performance metric is below the predetermined performance metric. If the second performance metric is below the predetermined metric, the method 400 proceeds to box 406 of FIG. 4A .
- the method 400 proceeds to step 458 .
- the design space estimate module 104 sets the modified range of input values equal to the last valid range of input values (the last range of input values that was associated with a performance metric below the predetermined performance metric).
- the design space estimate module 104 associates a completion flag with the input factor.
- the design space estimate module 104 determines whether there are remaining input factors from the one or more input factors that do not have a completion flag associated with the input factor. If there are remaining input factors, the method proceeds to box 402 of FIG. 4A . If there are not any remaining input factors, the method 400 proceeds to step 464 . At step 464 , the design space estimate module 104 reduces the step percentage by a predetermined percentage, and continues to box 406 to select a new range of input values by expanding the second range of input values based on the modified step size, and so on.
- the method 400 is performed individually on each of the one or more input factors (step 402 ).
- the design space estimate module 104 sets the step size to 20% of the factor range (20% of the calculated range of input values) of the input factor (step 404 ).
- the design space estimate module 104 expands the factor range of the input factor with the step size (e.g., adds the step size to the current factor range of the input factor) (step 406 ).
- the design space estimate module 104 performs Monte Carlo simulations as described above, adding in random errors if appropriate (step 408 ).
- the design space estimate module 104 shrinks the factor range of the input factor back to the last setting of the factor range with a DPMO below the DPMO limit ( 456 ), and associate the input factor with a completion flag. If the DPMO is lower than the DPMO limit, the method (method 400 ) is repeated by expanding the factor range and comparing the resulting DPMO for the Monte Carlo simulations (repeat steps 406 through 410 , where step 410 includes steps 450 through 464 ). The process is repeated for all of the remaining input factors until they are all associated with a completion flag.
- the design space estimate module 104 clears the completion flags from the input factors, divides the step size by a predetermined amount (e.g., divides the step size by two), and repeats the method (e.g., repeats steps 406 through 410 ).
- a modified range is calculated individually for each of the input factors (e.g., via method 400 of FIGS. 4A-4B ), and the PDSE is calculated with a large number of Monte Carlo simulations for better precision of the PDSE.
- the predetermined performance metric e.g., the limiting value
- the DPMO is, for example, the highest DPMO value tolerated for an individual output response, or the highest DPMO value for all of the output responses. In some examples other limiting criteria are used instead of DPMO, such as a percentage outside of the specification limit range for the output responses, a process capability index (Cpk), or medians and quartiles of the output responses.
- a user desires to optimize the operation of a car engine for the output responses of fuel consumption (fuel) measured in mg/stroke, nitrogen oxide emission (NOx) measured in mg/sec and soot emission (soot) measured in mg/sec.
- fuel consumption fuel
- NOx nitrogen oxide emission
- soot emission soot emission
- the three input factors that can be controlled for this example are air intake (air) measured in kg/h, exhaust gas recirculation percentage (EGR %, which is a NOx reduction technique that works by recirculating a portion of an engine's exhaust back to the engine cylinders), and needlelift (NL) measured in degrees before top dead-centre (° BTDC).
- the design space estimate module 104 predicts a PDSE for the engine based on these three input factors and three output responses to provide a measure of how much the three input factors can be varied (e.g., either individually or simultaneously) while still fulfilling the criteria for the three output responses.
- a user can set specific regions on which to perform the calculations, the experimented range of input values (e.g., steps 204 , 206 and 208 of FIG. 2 ). For this example, the calculations are performed on air between 240 kg/h and 284 kg/h, EGR % between 6% and 12%, and NL between ⁇ 5.78° BTDC and 0° BTDC. There are criteria for the output responses.
- the specification limit range for fuel is between a minimum output response value of 200 mg/stroke to a maximum output response value of 240 mg/stroke.
- the specification limit range for NOx is up to 25 mg/sec (e.g., there is no minimum value for the specification limit, which is zero in this example).
- the specification limits for soot is up to 0.5 mg/sec.
- Fuel has a target response value of 220 mg/stroke, NOx has a target response value of 10 mg/sec, and soot has a target response value of 0.2 mg/sec.
- CCD Central Composite Design
- MLR multiple linear regression
- Table 1 shows an exemplary regression model for fuel.
- Column one labeled “Fuel” includes the different components of the model, which are the constants, air, EGR, needlelift (NL), and air ⁇ air.
- Column two labeled “Coeff. SC” (coefficients, scaled and centered), includes a coefficient for each fuel component.
- Column three labeled “Std. Err.” (standard error), includes a standard error for each fuel component.
- a confidence interval is a statistics term used to indicate the reliability of an estimate. The confidence interval is used to get to, for example, 95% confidence interval (e.g., air is 8.78614+/ ⁇ 1.46984).
- the regression model for fuel also includes the following parameters.
- the number (N) of CCF experiments is 17.
- the degree of freedom (DF) is 12 (17 experiments minus 5 entries in the table).
- the residual standard deviation (RSD) is 2.0249, which is connected to R 2 , where the closer R 2 is to one, then the closer RSD is to zero.
- the confidence level of the model is 0.95.
- Table 2 shows an exemplary regression model for NOx.
- Column one, labeled “NOx” includes the different components of the model, which are the constants, air, EGR, NL, air ⁇ air, EGR ⁇ EGR, NL ⁇ NL, and EGR ⁇ NL.
- Table 2 includes the same four remaining columns as Table 1, namely “Coeff. SC”, “Std. Err.”, “P”, and “Conf. Int (+/ ⁇ )”.
- Table 3 shows an exemplary regression model for soot.
- Column one labeled “soot” includes the different components of the model, which are the constants, air, EGR, and NL.
- Table 3 includes the same four remaining columns as Table 1, namely “Coeff. SC”, “Std. Err.”, “P”, and “Conf. Int (+/ ⁇ )”.
- the estimated optimal value is calculated for each input factor that gives a predicted output response value that is the smallest distance to the target response criterion for each of the output responses. These estimated optimal values are the start points for the PDSE calculations. The largest region of variability is calculated for air, EGR and needlelift (e.g., step 204 of FIG. 2 ). In some examples, the calculation is built to assume that each input factor varies with a normal distribution around its specific estimated optimal value.
- FIG. 5A is a diagram illustrating a graphical display 500 for displaying an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- FIGS. 5A-C and 5 E while some of the numerical entries are displayed with commas, other numbering conventions can be used. For example, the U.S. convention of using decimals can be used in place of commas (e.g., 250,88 is equivalent to 250.88, and either convention can be used when displaying values in the graphical displays of the system).
- Display 500 includes an input factor table 502 and an output response table 504 .
- the input factor table 502 includes eight columns, the input factor column 502 A, the low column 502 B, the optimum column 502 C, the high column 502 D, the standard deviation column 502 E, the role column 502 F, the distribution column 502 G, and the estimated acceptable range column 502 H.
- the input factor column 502 A displays the input factors for the PDSE.
- the low column 502 B displays a value for each input factor that is the lowest input factor value of the combined largest region of variability.
- the optimum column 502 C displays a value for each input factor that is the estimated optimal value for each input factor.
- the high column 502 D displays a value for each input factor that is the highest input factor value of the combined largest region of variability.
- the standard deviation column 502 E lists the standard deviation for the input factors in the PDSE.
- the role column 502 F displays the role of the input factor (locked or free).
- the distribution column 502 G displays the distribution of the model.
- the input factor table 502 includes a row for each input factor air 506 A, EGR % 506 B and needlelift 506 C, collectively input factor rows 506 , which are listed in input factor column 502 A.
- the estimated acceptable range column 502 H includes a graph 508 A, 508 B and 508 C for each input factor row 506 A, 506 B and 506 C, respectively (collectively, graphs 508 ).
- Each graph 508 in the estimated acceptable range column 502 H includes an experimental region of input values, with a low value 510 A a high value 510 B, generally the experimented region of input values 510 .
- Each graph 508 includes an estimated optimal value 512 , which is indicated in the optimum column 502 C for each input factor row 506 (e.g., the air input factor row 506 A has an estimated optimal value 512 of 261 kg/h).
- the estimated optimal value 512 is the input factor value for each input factor that best estimates the target response criterion for each output factor.
- Each graph 508 includes an individual largest region of variability 514 for each input factor row 506 A, which is the region of values for the associated input factor that satisfy the PDSE while any remaining input factors are set to their associated estimated optimal value.
- Each graph 508 includes a combined largest region of variability 516 for each input factor row 506 A, which is between the value displayed in the low column 502 B and the high column 502 D for each input factor row 506 (e.g., the combined largest region of variability 516 for the air input factor row 506 A is between a low value 510 A of 250.88 kg/h and a high value 510 B of 271.12 kg/h).
- the values of each of the one or more input factors can be anywhere within the corresponding combined largest region of variability 516 for the input factor and satisfy the predetermined performance metric for each output response.
- the output response table 504 includes seven columns, the output response column 504 A, the min column 504 B, the target column 504 C, the max column 504 D, the criterion column 504 E, the DPMO column 504 F, and the predicted response profile column 504 G.
- the output response table 504 includes a row for each output response fuel 520 A, NOx 520 B and soot 520 C, collectively output response rows 520 , which are listed in output response column 504 A.
- the predicted response profile column 504 G includes a graph 522 A, 522 B and 522 C for each output response row 520 A, 520 B and 520 C, respectively (collectively, graphs 522 ).
- Each graph includes a specification limit range comprising a minimum output response value 524 A displayed in the minimum column 504 B and a maximum output response value 524 B displayed in the maximum column 504 D, collectively the specification limit range 524 (e.g., the fuel output response row 520 A has a specification limit range 524 between a minimum output response value 524 A of 230 mg/stroke and a maximum output response value 524 B of 260 mg/stroke).
- Each graph 522 includes a target response criterion 526 , which is indicated by the corresponding entry for each output response row 520 in the target column 504 C (e.g., the fuel output response row 520 A has a target value of 240 mg/stroke).
- Each graph includes a distribution of output response values 528 for each of the one or more output response rows 520 , wherein the combined largest regions of variability (e.g., the modified ranges of input values) are determinative of the distribution of output response values (e.g., as indicated for each input factor row 506 A by the low column 502 B and the high column 502 D).
- a prespecified number of output response values within the distribution of output response values 528 are within the specification limit range 524 for the output response (e.g., a predetermined performance metric, DPMO, is satisfied for the output response).
- the criteria (e.g., the experimented range of input values 510 for each input response, the specification limit ranges 524 for each output response, and the target response criterion 526 for each output response) constrain the prediction of the PDSE.
- the design space estimate module 104 searches for the best input factor value that results in output response values with the smallest distance to the target response criterion 512 , which are the estimated optimal values 512 for each input factor.
- a user can configure different settings for the calculation of the estimated optimal values 510 as shown in FIG. 5B .
- FIG. 5B is a diagram illustrating a graphical display 550 for configuring parameters used to calculate an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- the graphical display 550 includes an input factor table 552 , an output response table 554 , and a results table 556 .
- Input factor table 552 displays the options a user can configure for the input factors, and includes a low limit column, a high limit column, and a sensitivity range column.
- Output response table 554 displays the options a user can configure for the output responses, and includes a criteria column, a weight column, a minimum value column, a target value column, and a maximum value column.
- Numerical values can be entered into the columns for the input factor table 552 and the output response table 554 .
- a user can select entries from a drop-down menu in the criteria column of the output response table 554 , including “Target”, “Minimize”, “Maximize”, “Predicted”, or “Excluded”.
- “Target” indicates the output response is the desired response.
- “Minimize” indicates the design space estimate module 104 minimizes the output response (e.g., minimized based on a maximum accepted value criteria specified for the output response).
- “Maximize” indicates the design space estimate module 140 maximizes the output response (e.g., maximized based on a minimum accepted value criteria specified for the output response).
- Predicted indicates the design space estimate module 104 does not consider the output response as active in the PDSE (e.g., the design space estimation module 104 can still predict the output response).
- Excluded indicates the design space estimate module 140 excludes the output response from the PDSE (e.g., the design space estimation module 104 does not predict the output response).
- the results table 556 includes nine columns, numbered one through nine.
- the first three columns display the calculated input factor values for each of the input factors.
- the second three columns display the calculated output response values for each of the output responses.
- the seventh column displays the number of iterations performed for the calculation, which a user can configure with the iteration box 558 and the iteration slider 560 .
- the eighth column displays the log (D) value for the calculation.
- the ninth column displays the DPMO for the calculation.
- a user can select the absolute limits checkbox 562 which cause the design space estimate module 104 to not accept a prediction outside the criterias.
- a user can select the analyze sensitivity checkbox 564 which causes the design space estimate module 104 to add a Monte Carlo simulation with small disturbances on the input factor settings for the specific solution.
- a DPMO is calculated using a Monte Carlo simulation with small disturbances.
- Entries in the results table 556 can be similar.
- the ninth column displays a DPMO value for a Monte Carlo simulation with a variation of +/ ⁇ 10% of the experimental region of input values 510 about the proposed estimated optimal value.
- a lower DPMO value indicates the selection of input factor value results in output responses that are less sensitive to Monte Carlo simulation errors.
- the design space estimate module 104 searches for the PDSE from the estimated optimal values 512 for each input factor (e.g., via steps 204 through 210 of FIG. 2 ).
- the search for the largest regions of variability is performed stepwise based on each input factor's sensitivity towards the output response profile, which is displayed as the individual largest region of variability 514 for each input factor.
- the calculated combined largest region of variability 516 of the PDSE for each input factor is displayed in display 500 .
- the combined largest region of variability 516 displayed is the 95% region of a normal distribution for each input factor.
- a user can adjust how the combined largest region of variability 516 is displayed (e.g., changing the percentage of the region and/or the type of distribution).
- the calculations (e.g., monte carlo simulations) generate the predicted response profile 504 G displayed for each output response.
- the design space estimate module 104 allows a user to adjust the outer bounds for the combined largest region of variability 516 for one or more input factors. After a user adjusts the outer bounds of one or more combined largest regions of variability 516 , the design space estimate module 104 recalculates the combined largest regions of variability 516 for the remaining input factors. For example, if a user limits a combined largest region of variability 516 to values within the calculated low and high values of the combined largest region of variability (e.g., by clicking on the end of a displayed combined largest region of variability 516 and sliding the mouse to the left or right), the remaining combined largest regions of variability 516 is expanded due to the constraint placed on the modified input factor.
- a user adjusts an input factor, for example, if the role displayed in the role column 502 F is set to “Free.” For example, a user limits the variability of the combined largest region of variability 516 for the air input factor to +/ ⁇ 10 kg/h (251 kg/h-271 kg/h), which is within the low value of 250.88 kg/h and the high value 271.12 kg/h of the combined largest region of variability 516 for the air input factor row 506 A.
- the design space estimate module 104 can expand the combined largest regions of variability 516 for the NOx and soot input factors due to the limits placed on the air input factor.
- the user can set the predetermined performance metric (e.g., the DPMO) for either an individual response or all responses simultaneously.
- the user can configure the DPMO to take into account the constraints for the input factor so that the combined largest regions of variability 516 for the remaining input factors can increase.
- a calculation allows a user to know how much the remaining input factors can vary when the constrained input factor is kept at controlled values. For example, if a user knows the temperature of a manufacturing process, the user sets the constraints for the temperature input factor to discover the tolerances for other input factors of the manufacturing process (e.g., pressure). If temperature is easy to control, the PDSE is adjusted to indicate to the user the maximum amount the harder factors to control can vary.
- the PDSE output is used to interpret each input factor setting (e.g., a user can determine which factor is the most critical for the PDSE). If, for example, a user can't fulfill the constraints for a PDSE, the user changes the output response constraints or exclude one or more of the output responses from the PDSE model.
- FIG. 5C is a diagram illustrating a graphical display 600 for displaying an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- the graphical display 600 illustrates a sensitivity analysis between each input factor and output response.
- the graphical display 600 includes five columns, a name column 602 A, an optimum column 602 B, and a column for each of the output response values: a fuel column 602 C, a NOx column 602 D, and a soot column 602 E.
- the graphical display 600 includes an entry for each input factor, air 604 A, EGR % 604 B, and needlelift 604 C, which is displayed in the name column 602 A.
- Each table entry in the fuel column 602 C, the NOx column 602 D, and the soot column 602 E includes a graph 606 A, 606 B, and 606 C, respectively, for each input factor (collectively graph 606 ).
- Each graph 606 includes an experimental region of input values, with a low value 608 A a high value 608 B, generally the experimented region of input values 608 .
- Each graph 606 includes an estimated optimal value 610 , which is indicated in the optimum column 602 B for each input factor row 604 (e.g., the air input factor row 506 A has an estimated optimal value 610 of 261 kg/h).
- Each graph 606 includes an individual largest region of variability 612 and a combined largest region of variability 614 for each input factor row 604 .
- Each graph 606 displays the results individually for each input factor and output response.
- a user can use the graphs 606 to analyze the PDSE. For example, a user discerns problematic points of the PDSE by analyzing the graphs 606 .
- the output response e.g., the fuel column 602 C, the NOx column 602 D, and the soot column 602 E
- the soot graphs 606 (for each input response 604 ) display the smallest combined largest regions of variability 614 . Of the three graphs 606 for soot, the smallest combined largest regions of variability 614 is for EGR %.
- EGR % the most critical factor to control for soot is EGR % because it has the smallest combined largest region of variability 614 .
- FIG. 5D is a diagram illustrating a graphical display 650 for displaying an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- Graphical display 650 includes three graphs based on the PDSE for each output factor (e.g., the three graphs are alternative embodiments to display the information displayed for each output factor in the predicted response profile 504 G of FIG. 5A ), the predicted response profile for fuel 652 A, the predicted response profile for NOx 652 B, and the predicted response profile for soot 652 C, collectively graphs 652 .
- the vertical axes 654 of each graph 652 indicates the count of output response values.
- each graph 652 indicates the number of bins for each input factor (e.g., the possible values for each input factor).
- Each graph includes a minimum bar 658 (if appropriate).
- the fuel graph 652 A includes a minimum bar 658 at 230 bins (e.g., mg/stroke), which corresponds to the value displayed in the entry for the fuel row 520 A in the minimum column 504 B in the output response table 504 of FIG. 5A .
- Each graph includes a maximum bar 660 (if appropriate).
- the fuel graph 652 A includes a maximum bar 660 at 260 bins, which corresponds to the value displayed in the entry for the fuel row 520 A in the maximum column 504 D in the output response table 504 .
- Each graph includes a target bar 662 (if appropriate).
- the fuel graph 652 A includes a maximum bar 662 at 240 bins, which corresponds to the value displayed in the entry for the fuel row 520 A in the target column 504 C in the output response table 504 .
- Each graph further includes bars to indicate the first quartile 664 , the median 666 , and the third quartile 668 of the calculated output response values 670 for the corresponding output response.
- a tablet manufacturing PDSE is made (e.g., in the pharmaceutical industry) for a mixture design.
- there are three input factors e.g., constituents
- cellulose e.g., lactose, and phosphate.
- the three input factors are varied according to a modified simplex centroid mixture design (a type of constrained mixture design) to produce tablets.
- the output response is the release of the active substance.
- the criteria placed on the release output response is a minimum output response value of 300 and a target output response value of 350.
- the PDSE informs the user how the three input factors influence the release of the active substance.
- FIG. 5E is a diagram illustrating a graphical display 700 for displaying an automated predictive design space estimate, according to an illustrative embodiment of the invention.
- the graphical display 700 displays the PDSE for the tablet manufacturing example.
- Graphical display 700 includes an input factor table 702 and an output response table 704 .
- the input factor table 702 includes eight columns, the input factor column 702 A, the low column 702 B, the optimum column 702 C, the high column 702 D, the standard deviation column 702 E, the role column 702 F, the distribution column 702 G, and the estimated acceptable range column 702 H.
- the input factor column 702 A displays the input factors for the PDSE.
- the low column 702 B displays a value for each input factor that is the lowest input factor value of the combined largest region of variability.
- the optimum column 702 C displays a value for each input factor that is the estimated optimal value for each input factor.
- the high column 702 D displays a value for each input factor that is the highest input factor value of the combined largest region of variability.
- the standard deviation column 702 E lists the standard deviation for the input factors in the PDSE.
- the role column 702 F displays the role of the input factor (locked or free).
- the distribution column 702 G displays the distribution of the model.
- the input factor table 702 includes a row for each input factor cellulose 706 A, lactose 706 B and phosphate 706 C, collectively input factor rows 706 , which are listed in input factor column 702 A.
- the estimated acceptable range column 502 H includes a graph 708 A, 708 B and 708 C for each input factor row 706 A, 706 B and 706 C, respectively (collectively, graphs 708
- Each graph 708 in the estimated acceptable range column 702 H includes an experimental region of input values, with a low value 710 A a high value 710 B, generally the experimented region of input values 710 .
- Each graph 708 includes an estimated optimal value 712 , which is indicated in the optimum column 702 C for each input factor row 706 .
- Each graph 708 includes an individual largest region of variability 714 for each input factor row 706 A and a combined largest region of variability 716 for each input factor row 706 A, which is between the value displayed in the low column 702 B and the high column 702 D for each input factor row 706 .
- the output response table 704 includes eight columns, the output response column 704 A, the min column 704 B, the target column 704 C, the max column 704 D, the criterion column 704 E, the cpk column 704 F, the DPMO column 704 G, and the predicted response profile column 704 H.
- the output response table 704 includes a row for the output response release 720 .
- the predicted response profile column 704 H includes a graph 722 for the output response releast 720 .
- the graph 722 includes a specification limit range comprising a minimum output response value 724 displayed in the minimum column 704 B with no maximum value since none is displayed in the maximum column 704 D, which is the specification limit range.
- the graph 722 includes a target response criterion 726 and a distribution of output response values 728 .
- the target response criterion 712 is estimated with a modified simplex centroid design with 10 runs. From the target response criterion 712 for each input factor, the design space estimate module 104 calculates the PDSE and displays (e.g., via display 114 ) the combined largest region of variability 714 for each input factor. For this example, the design space estimate module 104 is configured to calculate the PDSE based on each input factor varying with a triangular distribution around its associated estimated optimal value 712 . The combined largest regions of variability 714 are calculated with the mixture restriction. The predicted response profile 704 H for the release shows the distribution of output response values 728 for input factor values within the combined largest regions of variability.
- the above-described techniques can be implemented in digital and/or analog electronic circuitry, or in computer hardware, firmware, software, or in combinations of them.
- the implementation can be as a computer program product, i.e., a computer program tangibly embodied in a machine-readable storage device, for execution by, or to control the operation of, a data processing apparatus, e.g., a programmable processor, a computer, and/or multiple computers.
- a computer program can be written in any form of computer or programming language, including source code, compiled code, interpreted code and/or machine code, and the computer program can be deployed in any form, including as a stand-alone program or as a subroutine, element, or other unit suitable for use in a computing environment.
- a computer program can be deployed to be executed on one computer or on multiple computers at one or more sites.
- Method steps can be performed by one or more processors executing a computer program to perform functions of the invention by operating on input data and/or generating output data. Method steps can also be performed by, and an apparatus can be implemented as, special purpose logic circuitry, e.g., a FPGA (field programmable gate array), a FPAA (field-programmable analog array), a CPLD (complex programmable logic device), a PSoC (Programmable System-on-Chip), ASIP (application-specific instruction-set processor), or an ASIC (application-specific integrated circuit). Subroutines can refer to portions of the computer program and/or the processor/special circuitry that implement one or more functions.
- FPGA field programmable gate array
- FPAA field-programmable analog array
- CPLD complex programmable logic device
- PSoC Programmable System-on-Chip
- ASIP application-specific instruction-set processor
- ASIC application-specific integrated circuit
- processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital or analog computer.
- a processor receives instructions and data from a read-only memory or a random access memory or both.
- the essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and/or data.
- Memory devices such as a cache, can be used to temporarily store data. Memory devices can also be used for long-term data storage.
- a computer also includes, or is operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks.
- a computer can also be operatively coupled to a communications network in order to receive instructions and/or data from the network and/or to transfer instructions and/or data to the network.
- Computer-readable storage devices suitable for embodying computer program instructions and data include all forms of volatile and non-volatile memory, including by way of example semiconductor memory devices, e.g., DRAM, SRAM, EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and optical disks, e.g., CD, DVD, HD-DVD, and Blu-ray disks.
- the processor and the memory can be supplemented by and/or incorporated in special purpose logic circuitry.
- the above described techniques can be implemented on a computer in communication with a display device, e.g., a CRT (cathode ray tube), plasma, or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse, a trackball, a touchpad, or a motion sensor, by which the user can provide input to the computer (e.g., interact with a user interface element).
- a display device e.g., a CRT (cathode ray tube), plasma, or LCD (liquid crystal display) monitor
- a keyboard and a pointing device e.g., a mouse, a trackball, a touchpad, or a motion sensor, by which the user can provide input to the computer (e.g., interact with a user interface element).
- feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, and/or tactile input.
- feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback
- input from the user can be received in any form, including acoustic, speech, and/or tactile input.
- the above described techniques can be implemented in a distributed computing system that includes a back-end component.
- the back-end component can, for example, be a data server, a middleware component, and/or an application server.
- the above described techniques can be implemented in a distributed computing system that includes a front-end component.
- the front-end component can, for example, be a client computer having a graphical user interface, a Web browser through which a user can interact with an example implementation, and/or other graphical user interfaces for a transmitting device.
- the above described techniques can be implemented in a distributed computing system that includes any combination of such back-end, middleware, or front-end components.
- the computing system can include clients and servers.
- a client and a server are generally remote from each other and typically interact through a communication network.
- the relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
- the components of the computing system can be interconnected by any form or medium of digital or analog data communication (e.g., a communication network).
- Examples of communication networks include circuit-based and packet-based networks.
- Packet-based networks can include, for example, the Internet, a carrier internet protocol (IP) network (e.g., local area network (LAN), wide area network (WAN), campus area network (CAN), metropolitan area network (MAN), home area network (HAN)), a private IP network, an IP private branch exchange (IPBX), a wireless network (e.g., radio access network (RAN), 802.11 network, 802.16 network, general packet radio service (GPRS) network, HiperLAN), and/or other packet-based networks.
- IP carrier internet protocol
- RAN radio access network
- 802.11 802.11 network
- 802.16 general packet radio service
- GPRS general packet radio service
- HiperLAN HiperLAN
- Circuit-based networks can include, for example, the public switched telephone network (PSTN), a private branch exchange (PBX), a wireless network (e.g., RAN, bluetooth, code-division multiple access (CDMA) network, time division multiple access (TDMA) network, global system for mobile communications (GSM) network), and/or other circuit-based networks.
- PSTN public switched telephone network
- PBX private branch exchange
- CDMA code-division multiple access
- TDMA time division multiple access
- GSM global system for mobile communications
- Devices of the computing system and/or computing devices can include, for example, a computer, a computer with a browser device, a telephone, an IP phone, a mobile device (e.g., cellular phone, personal digital assistant (PDA) device, laptop computer, electronic mail device), a server, a rack with one or more processing cards, special purpose circuitry, and/or other communication devices.
- the browser device includes, for example, a computer (e.g., desktop computer, laptop computer) with a world wide web browser (e.g., Microsoft® Internet Explorer® available from Microsoft Corporation, Mozilla® Firefox available from Mozilla Corporation).
- a mobile computing device includes, for example, a Blackberry®.
- IP phones include, for example, a Cisco® Unified IP Phone 7985G available from Cisco System, Inc, and/or a Cisco® Unified Wireless Phone 7920 available from Cisco System, Inc.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Hardware Design (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Architecture (AREA)
- Software Systems (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Feedback Control In General (AREA)
Abstract
Description
TABLE 1 | ||||
Fuel | Coeff. SC | Std. Err. | P | Conf. int (±) |
Constant | 243.042 | 0.77712 | 7.68757e−025 | 1.69319 |
Air | 8.78614 | 0.674565 | 1.93059e−008 | 1.46974 |
EGR | 2.66658 | 0.656038 | 0.00156851 | 1.42938 |
NL | −12.2052 | 0.646809 | 2.74769e−010 | 1.40927 |
Air * Air | −5.716 | 1.08892 | 0.00020466 | 2.37255 |
TABLE 2 | ||||
NOx | Coeff. SC | Std. Err. | P | Conf. int (±) |
Constant | 19.4543 | 0.200454 | 6.63998e−015 | 0.453461 |
Air | 0.816983 | 0.152378 | 0.00045539 | 0.344707 |
EGR | −7.77984 | 0.148415 | 1.68126e−012 | 0.335741 |
NL | 1.93977 | 0.146473 | 3.31136e−007 | 0.331349 |
Air * Air | −0.571193 | 0.298143 | 0.087639 | 0.674452 |
EGR * EGR | 1.29447 | 0.283578 | 0.001357 | 0.641503 |
NL * NL | 0.812071 | 0.283058 | 0.0185093 | 0.640327 |
EGR * NL | −0.941485 | 0.166246 | 0.000308381 | 0.376079 |
TABLE 3 | ||||
Soot | Coeff. SC | Std. Err. | P | Conf. int (±) |
Constant | −0.279945 | 0.0288614 | 2.56008e−007 | 0.0623513 |
Air | −0.157893 | 0.0389473 | 0.00136603 | 0.0841406 |
EGR | 0.46588 | 0.0380323 | 1.6319e−008 | 0.0821639 |
NL | 0.283157 | 0.0375012 | 4.18648e−006 | 0.0810165 |
Claims (23)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/336,623 US8412356B2 (en) | 2009-05-14 | 2011-12-23 | Methods and apparatus for automated predictive design space estimation |
US13/775,708 US8577480B2 (en) | 2009-05-14 | 2013-02-25 | Methods and apparatus for automated predictive design space estimation |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/466,098 US8086327B2 (en) | 2009-05-14 | 2009-05-14 | Methods and apparatus for automated predictive design space estimation |
US13/336,623 US8412356B2 (en) | 2009-05-14 | 2011-12-23 | Methods and apparatus for automated predictive design space estimation |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/466,098 Continuation US8086327B2 (en) | 2009-05-14 | 2009-05-14 | Methods and apparatus for automated predictive design space estimation |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/775,708 Continuation-In-Part US8577480B2 (en) | 2009-05-14 | 2013-02-25 | Methods and apparatus for automated predictive design space estimation |
Publications (2)
Publication Number | Publication Date |
---|---|
US20120150508A1 US20120150508A1 (en) | 2012-06-14 |
US8412356B2 true US8412356B2 (en) | 2013-04-02 |
Family
ID=42321117
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/466,098 Active 2030-08-05 US8086327B2 (en) | 2009-05-14 | 2009-05-14 | Methods and apparatus for automated predictive design space estimation |
US13/336,623 Active US8412356B2 (en) | 2009-05-14 | 2011-12-23 | Methods and apparatus for automated predictive design space estimation |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/466,098 Active 2030-08-05 US8086327B2 (en) | 2009-05-14 | 2009-05-14 | Methods and apparatus for automated predictive design space estimation |
Country Status (9)
Country | Link |
---|---|
US (2) | US8086327B2 (en) |
JP (1) | JP5555316B2 (en) |
KR (2) | KR101408833B1 (en) |
CN (1) | CN102804188A (en) |
DE (1) | DE112010002673T9 (en) |
GB (1) | GB2482452A (en) |
SG (2) | SG185983A1 (en) |
TW (1) | TW201102839A (en) |
WO (1) | WO2010132224A2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2016073581A1 (en) * | 2014-11-04 | 2016-05-12 | Samuelson Douglas A | Machine learning and robust automatic control of complex systems with stochastic factors |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8725469B2 (en) | 2011-03-03 | 2014-05-13 | Mks Instruments, Inc. | Optimization of data processing parameters |
JP5977373B2 (en) * | 2012-02-10 | 2016-08-24 | アーベーベー・テクノロジー・アーゲー | Systems and methods for automatically processing workflows in automation and / or electrical engineering projects |
WO2014130172A1 (en) * | 2013-02-25 | 2014-08-28 | Mks Instruments, Inc. | Methods and apparatus for automated predictive design space estimation |
TWI570584B (en) * | 2013-05-20 | 2017-02-11 | 新思科技有限公司 | A simulation method based on semi-local ballistic mobility model and computer system using the same |
TWI603210B (en) | 2016-12-13 | 2017-10-21 | 財團法人工業技術研究院 | System and method for predicting remaining lifetime of a component of equipment |
US11561690B2 (en) | 2018-04-22 | 2023-01-24 | Jmp Statistical Discovery Llc | Interactive graphical user interface for customizable combinatorial test construction |
US11328106B2 (en) | 2018-04-22 | 2022-05-10 | Sas Institute Inc. | Data set generation for performance evaluation |
US10878345B2 (en) * | 2018-04-22 | 2020-12-29 | Sas Institute Inc. | Tool for hyperparameter tuning |
CN113574325B (en) * | 2019-03-15 | 2022-12-27 | 3M创新有限公司 | Method and system for controlling an environment by selecting a control setting |
WO2020263791A1 (en) * | 2019-06-28 | 2020-12-30 | Covestro Llc | Methods for graphical depiction of a value of a property of a material |
CN115066658B (en) * | 2020-02-28 | 2024-05-24 | 3M创新有限公司 | Deep Causal Learning for Advanced Model Predictive Control |
CN115087992B (en) | 2020-02-28 | 2024-03-29 | 3M创新有限公司 | Deep causal learning for data storage and processing capacity management |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5408405A (en) | 1993-09-20 | 1995-04-18 | Texas Instruments Incorporated | Multi-variable statistical process controller for discrete manufacturing |
US5710700A (en) | 1995-12-18 | 1998-01-20 | International Business Machines Corporation | Optimizing functional operation in manufacturing control |
US6453246B1 (en) | 1996-11-04 | 2002-09-17 | 3-Dimensional Pharmaceuticals, Inc. | System, method, and computer program product for representing proximity data in a multi-dimensional space |
US20030028358A1 (en) | 2001-08-06 | 2003-02-06 | Xinhui Niu | Method and system of dynamic learning through a regression-based library generation process |
WO2004019147A2 (en) | 2002-08-20 | 2004-03-04 | Tokyo Electron Limited | Method for processing data based on the data context |
US7191106B2 (en) | 2002-03-29 | 2007-03-13 | Agilent Technologies, Inc. | Method and system for predicting multi-variable outcomes |
US7606685B2 (en) | 2006-05-15 | 2009-10-20 | S-Matrix | Method and system that optimizes mean process performance and process robustness |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5607441A (en) * | 1995-03-24 | 1997-03-04 | Ethicon Endo-Surgery, Inc. | Surgical dissector |
US5667480A (en) * | 1995-10-20 | 1997-09-16 | Ethicon Endo-Surgery, Inc. | Method and devices for endoscopic vessel harvesting |
US5928135A (en) * | 1996-08-15 | 1999-07-27 | Ethicon Endo-Surgery, Inc. | Method and devices for endoscopic vessel harvesting |
US6725112B1 (en) * | 1999-10-29 | 2004-04-20 | General Electric Company | Method, system and storage medium for optimizing a product design |
EP2047806B1 (en) * | 2001-12-27 | 2011-10-05 | Olympus Corporation | Sheath with devices for endoscopic blood vessel harvesting |
JP4528962B2 (en) * | 2004-07-16 | 2010-08-25 | 国立大学法人電気通信大学 | Design support method |
JP5121132B2 (en) * | 2005-11-02 | 2013-01-16 | オリンパスメディカルシステムズ株式会社 | Endoscope system and operation assist device for endoscope |
US7451122B2 (en) * | 2006-03-29 | 2008-11-11 | Honeywell International Inc. | Empirical design of experiments using neural network models |
CN100565534C (en) * | 2006-09-30 | 2009-12-02 | 普诚科技股份有限公司 | Optimum parameter regulation method and system |
KR100828135B1 (en) * | 2006-12-13 | 2008-05-08 | 이은규 | Endoscopic dissector |
US20080249556A1 (en) * | 2007-04-06 | 2008-10-09 | Ken Yamatani | Dissection apparatus and dissection method |
JP4894709B2 (en) * | 2007-10-04 | 2012-03-14 | 株式会社Ihi | Product design support system and operation method for product design support in computer |
TW200921445A (en) * | 2007-11-08 | 2009-05-16 | Airoha Tech Corp | Circuit analysis method |
-
2009
- 2009-05-14 US US12/466,098 patent/US8086327B2/en active Active
-
2010
- 2010-05-03 DE DE112010002673T patent/DE112010002673T9/en not_active Expired - Fee Related
- 2010-05-03 GB GB1119387.7A patent/GB2482452A/en not_active Withdrawn
- 2010-05-03 KR KR1020117030004A patent/KR101408833B1/en not_active IP Right Cessation
- 2010-05-03 CN CN2010800302891A patent/CN102804188A/en active Pending
- 2010-05-03 WO PCT/US2010/033364 patent/WO2010132224A2/en active Application Filing
- 2010-05-03 SG SG2012082996A patent/SG185983A1/en unknown
- 2010-05-03 KR KR1020147002339A patent/KR101391318B1/en not_active IP Right Cessation
- 2010-05-03 SG SG2011083532A patent/SG176043A1/en unknown
- 2010-05-03 JP JP2012510848A patent/JP5555316B2/en not_active Expired - Fee Related
- 2010-05-12 TW TW099115150A patent/TW201102839A/en unknown
-
2011
- 2011-12-23 US US13/336,623 patent/US8412356B2/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5408405A (en) | 1993-09-20 | 1995-04-18 | Texas Instruments Incorporated | Multi-variable statistical process controller for discrete manufacturing |
US5710700A (en) | 1995-12-18 | 1998-01-20 | International Business Machines Corporation | Optimizing functional operation in manufacturing control |
US6453246B1 (en) | 1996-11-04 | 2002-09-17 | 3-Dimensional Pharmaceuticals, Inc. | System, method, and computer program product for representing proximity data in a multi-dimensional space |
US20030028358A1 (en) | 2001-08-06 | 2003-02-06 | Xinhui Niu | Method and system of dynamic learning through a regression-based library generation process |
US7191106B2 (en) | 2002-03-29 | 2007-03-13 | Agilent Technologies, Inc. | Method and system for predicting multi-variable outcomes |
WO2004019147A2 (en) | 2002-08-20 | 2004-03-04 | Tokyo Electron Limited | Method for processing data based on the data context |
US7606685B2 (en) | 2006-05-15 | 2009-10-20 | S-Matrix | Method and system that optimizes mean process performance and process robustness |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2016073581A1 (en) * | 2014-11-04 | 2016-05-12 | Samuelson Douglas A | Machine learning and robust automatic control of complex systems with stochastic factors |
Also Published As
Publication number | Publication date |
---|---|
SG176043A1 (en) | 2011-12-29 |
GB2482452A (en) | 2012-02-01 |
US20120150508A1 (en) | 2012-06-14 |
KR20140022110A (en) | 2014-02-21 |
CN102804188A (en) | 2012-11-28 |
KR20120031013A (en) | 2012-03-29 |
KR101408833B1 (en) | 2014-06-19 |
US20100292812A1 (en) | 2010-11-18 |
JP5555316B2 (en) | 2014-07-23 |
SG185983A1 (en) | 2012-12-28 |
KR101391318B1 (en) | 2014-05-07 |
TW201102839A (en) | 2011-01-16 |
US8086327B2 (en) | 2011-12-27 |
DE112010002673T9 (en) | 2013-05-08 |
GB201119387D0 (en) | 2011-12-21 |
DE112010002673T5 (en) | 2012-12-06 |
JP2012527040A (en) | 2012-11-01 |
WO2010132224A2 (en) | 2010-11-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8412356B2 (en) | Methods and apparatus for automated predictive design space estimation | |
US8577480B2 (en) | Methods and apparatus for automated predictive design space estimation | |
US7831416B2 (en) | Probabilistic modeling system for product design | |
US7877239B2 (en) | Symmetric random scatter process for probabilistic modeling system for product design | |
US7788070B2 (en) | Product design optimization method and system | |
Wei et al. | Variable importance analysis: A comprehensive review | |
US20070061144A1 (en) | Batch statistics process model method and system | |
US8209156B2 (en) | Asymmetric random scatter process for probabilistic modeling system for product design | |
US20060229753A1 (en) | Probabilistic modeling system for product design | |
Smith et al. | Nonlinear clustering in models with primordial non-Gaussianity: the halo model approach | |
JP2011523753A (en) | Recommendation system by fast matrix factorization using infinite dimensions | |
Huang et al. | Adaptive random test case generation for combinatorial testing | |
CN116127695A (en) | Production line construction method and system based on comprehensive performance evaluation | |
CN113610266B (en) | Method and device for predicting failure of automobile part, computer equipment and storage medium | |
DeGennaro et al. | Model structural inference using local dynamic operators | |
Willis et al. | Multi-objective simulation optimization through search heuristics and relational database analysis | |
Menga et al. | Anisotropic meta‐models for computationally expensive simulations in nonlinear mechanics | |
US8566064B2 (en) | Estimating polynomial generating device, estimating device, estimating polynomial generating method, and estimating method | |
US20240103920A1 (en) | Method and system for accelerating the convergence of an iterative computation code of physical parameters of a multi-parameter system | |
US10496948B1 (en) | Computer trend visualization using quadratic simplified closed form linear regression | |
WO2014130172A1 (en) | Methods and apparatus for automated predictive design space estimation | |
Zhou et al. | An active learning variable-fidelity metamodeling approach for engineering design | |
Cingovska et al. | Protein Function Prediction by Clustering of Protein-Protein Interaction Network | |
Wu et al. | Computationally Enhanced Approach for Chance-Constrained OPF Considering Voltage Stability | |
Baquela et al. | Operations Research Perspectives |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MKS INSTRUMENTS, INC., MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WIKSTROM, ERNST CONNY;SUNDSTROM, HANS GEORG JOAKIM;NORDAHL, TORD AKE BORJE;SIGNING DATES FROM 20090710 TO 20090720;REEL/FRAME:027527/0361 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: BARCLAYS BANK PLC, NEW YORK Free format text: SECURITY AGREEMENT;ASSIGNORS:MKS INSTRUMENTS, INC.;NEWPORT CORPORATION;REEL/FRAME:038663/0139 Effective date: 20160429 Owner name: DEUTSCHE BANK AG NEW YORK BRANCH, NEW YORK Free format text: SECURITY AGREEMENT;ASSIGNORS:MKS INSTRUMENTS, INC.;NEWPORT CORPORATION;REEL/FRAME:038663/0265 Effective date: 20160429 |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: SARTORIUS STEDIM BIOTECH GMBH, GERMANY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MKS INSTRUMENTS, INC;REEL/FRAME:043630/0870 Effective date: 20170403 |
|
AS | Assignment |
Owner name: MKS INSTRUMENTS AB, SWEDEN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SARTORIUS STEDIM BIOTECH GMBH;REEL/FRAME:043888/0860 Effective date: 20170928 |
|
AS | Assignment |
Owner name: SARTORIUS STEDIM DATA ANALYTICS AB, SWEDEN Free format text: CHANGE OF NAME;ASSIGNOR:MKS INSTRUMENTS AB;REEL/FRAME:044256/0882 Effective date: 20170529 |
|
AS | Assignment |
Owner name: NEWPORT CORPORATION, CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK AG NEW YORK BRANCH;REEL/FRAME:048226/0095 Effective date: 20190201 Owner name: MKS INSTRUMENTS, INC., MASSACHUSETTS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:DEUTSCHE BANK AG NEW YORK BRANCH;REEL/FRAME:048226/0095 Effective date: 20190201 |
|
AS | Assignment |
Owner name: MKS INSTRUMENTS, INC., MASSACHUSETTS Free format text: PARTIAL RELEASE OF SECURITY INTEREST;ASSIGNOR:BARCLAYS BANK PLC;REEL/FRAME:049728/0509 Effective date: 20170403 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |
|
AS | Assignment |
Owner name: ELECTRO SCIENTIFIC INDUSTRIES, INC., OREGON Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:BARCLAYS BANK PLC;REEL/FRAME:062739/0001 Effective date: 20220817 Owner name: NEWPORT CORPORATION, MASSACHUSETTS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:BARCLAYS BANK PLC;REEL/FRAME:062739/0001 Effective date: 20220817 Owner name: MKS INSTRUMENTS, INC., MASSACHUSETTS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:BARCLAYS BANK PLC;REEL/FRAME:062739/0001 Effective date: 20220817 |