JP7195554B2 - Evaluation apparatus and evaluation method, display apparatus and display method, exposure apparatus and exposure method, exposure system, device manufacturing apparatus, and computer program - Google Patents

Description

Evaluation apparatus and evaluation method for evaluating performance of exposure apparatus, display apparatus and display method for displaying information related to performance of exposure apparatus, exposure apparatus equipped with evaluation apparatus or display apparatus, exposure method including evaluation method or display method , an exposure system equipped with this evaluation device or display device, a device manufacturing method using this exposure method, and a computer program.

An exposure apparatus is used to manufacture devices such as integrated circuits. Patent Literature 1 describes a technique for detecting an abnormality in an exposure apparatus.

U.S. Patent Publication No. 2008/0128644

A first aspect of the evaluation apparatus is an evaluation apparatus for evaluating the performance of an exposure apparatus, comprising: an input unit for inputting variables of a plurality of items each indicating an operation state of the exposure apparatus; a first change amount obtained from each variable of the plurality of items when the operating state is in a first state and each variable of the plurality of items when the operating state is in a second state different from the first state; a memory storing a first relationship representing a relationship between the amount of change in the performance of the exposure apparatus according to one amount of change; a controller that obtains an index value representing the performance of the exposure apparatus by using each variable of the plurality of items in different third states and the first relationship in the memory;

A first aspect of the display device is a display device that displays information about the performance of the exposure apparatus, and is a display form that expresses the influence on the performance of the exposure apparatus and the first aspect of the evaluation apparatus of the present invention described above. and a display unit for displaying the index value.

A first aspect of the evaluation method is an evaluation method for evaluating the performance of an exposure apparatus, comprising: inputting variables for each of a plurality of items each indicating an operating state of the exposure apparatus; A first change amount obtained from each variable of the plurality of items when the operating state is in a first state and each variable of the plurality of items when the operating state is in a second state different from the first state, and the first A first relationship representing a relationship between the amount of change in performance of the exposure apparatus according to the amount of change, and the plurality of items when the operating state is a third state different from the first and second states. obtaining an index value representing the performance of the exposure apparatus using each variable.

A first aspect of the display method is a display method for displaying information about the performance of the exposure apparatus, and the index and displaying the value.

A first aspect of the exposure apparatus is an exposure apparatus that exposes an object, and includes the first aspect of the evaluation apparatus described above.

A second aspect of the exposure apparatus is an exposure apparatus that exposes an object and includes the first aspect of the display device described above.

A first aspect of the exposure method is an exposure method for exposing an object, and includes the first aspect of the evaluation method described above.

A second aspect of the exposure method is an exposure method for exposing an object, and includes the first aspect of the display method described above.

A first aspect of the exposure system includes the first aspect of the evaluation apparatus described above and the exposure apparatus.

A second aspect of the exposure system includes the first aspect of the display device described above and the exposure device.

In a first aspect of the device manufacturing method, the object is exposed using the first or second aspect of the exposure method described above, a pattern is transferred to the object, the exposed object is developed, and the An exposure pattern layer corresponding to a pattern is formed, and the object is processed through the exposure pattern layer.

A first aspect of a computer program causes a computer to execute processing by a controller included in the evaluation device or the display device described above.

FIG. 1 is a block diagram showing the configuration of the exposure system of this embodiment. FIG. 2 is a configuration diagram showing an example of the configuration of the exposure apparatus of this embodiment. FIG. 3 is a configuration diagram showing an example of the configuration of the evaluation device of this embodiment. FIG. 4 is a flow chart showing the overall flow of the evaluation operation performed by the evaluation device (that is, the operation of evaluating the performance of the exposure device). FIG. 5 is a flow chart showing the flow of the first correlation structure estimation operation. FIG. 6 is a plan view showing the basic display form of the correlation structure. FIG. 7 is a plan view showing a first specific example of the display form of the correlation structure. FIG. 8 is a plan view showing a second specific example of the display form of the correlation structure. FIGS. 9A to 9D are plan views showing display examples of the correlation structure, respectively. FIG. 10 is a flow chart showing the flow of the second correlation structure estimation operation. FIG. 11 is a flow chart showing the flow of the third correlation structure estimation operation. FIG. 12 is a matrix showing the classification of elements in the precision matrix. 13(a) and 13(b) are graphs showing examples of scores, respectively. FIG. 14 is an example of a database showing correspondence between scores and performance. FIGS. 15(a) to 15(c) are plan views each showing an example of a display form of performance. FIG. 16 is a graph showing the score SC before weighting processing, weighting coefficients for weighting processing, and score SC before weighting processing. FIG. 17 is a matrix showing an example of log data in the second modified example. FIG. 18 is a flow chart showing the flow of the device manufacturing method.

Hereinafter, embodiments will be described with reference to the drawings. However, the present invention is not limited to the embodiments described below.
(1) Configuration of exposure system SYS
First, the configuration of the exposure system SYS of the present embodiment will be described with reference to FIGS. 1 to 3. FIG. For convenience of explanation, the configuration of the exposure system SYS will be described below, and then the configurations of the exposure apparatus 1 and the evaluation apparatus 2 included in the exposure system SYS will be described in order.
(1-1) Configuration of exposure system SYS
The configuration of the exposure system SYS of this embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the exposure system SYS of this embodiment.

As shown in FIG. 1, the exposure system SYS includes an exposure apparatus 1 and an evaluation apparatus 2 that are connected via a wired or wireless communication interface 3 . The exposure apparatus 1 and evaluation apparatus 2 are installed in the same factory. However, while the exposure apparatus 1 is installed inside the factory, the evaluation apparatus 2 may be installed outside the factory. For example, the evaluation apparatus 2 may be installed at the manufacturer of the exposure apparatus. Moreover, the evaluation apparatus 2 may be plural. At least one of the plurality of evaluation devices 2 may be installed inside the factory and at least one may be installed outside the factory.

An exposure device 1 exposes an object. Hereinafter, for convenience of explanation, the object is assumed to be a substrate 141 (see FIG. 2) such as a semiconductor substrate coated with a resist. In this case, the exposure apparatus 1 transfers a device pattern (for example, a semiconductor element pattern) to the substrate 141 by exposing the substrate 141 . In other words, the exposure apparatus 1 is assumed to be an exposure apparatus for manufacturing devices such as semiconductor elements. However, as will be described later, the exposure apparatus 1 may be any exposure apparatus that exposes an arbitrary object or irradiates an arbitrary object with light.

The evaluation device 2 evaluates the performance of the exposure device 1 . For example, the evaluation device 2 determines whether the current performance of the exposure device 1 is good or bad. For example, the evaluation device 2 predicts whether the future performance of the exposure device 1 is good or bad. The performance to be evaluated may include performance related to device manufacturing by the exposure apparatus 1 (so-called productivity). Performance to be evaluated may include performance related to stable maintenance of productivity. Examples of such performance include performance of the exposure apparatus 1 as a whole (eg, throughput and yield), accuracy of the exposure apparatus 1 as a whole (eg, overlay accuracy, synchronization accuracy, line width accuracy, etc.), , the performance (for example, accuracy) of each component constituting the exposure apparatus 1 .

The performance of the exposure apparatus 1 fluctuates depending on abnormalities that occur in the exposure apparatus 1 . Therefore, "evaluation of performance of exposure apparatus 1" performed by evaluation apparatus 2 may include "detection of abnormality actually occurring in exposure apparatus 1". The "evaluation of the performance of the exposure apparatus 1" performed by the evaluation apparatus 2 may include "prediction of anomalies that may occur in the exposure apparatus 1 in the future".

The exposure apparatus 1 and evaluation apparatus 2 are connected via a wired or wireless network 4 to an external host computer 5 installed outside the factory. The host computer 5 controls, for example, multiple exposure apparatuses 1 and multiple evaluation apparatuses 2 installed in a single factory. Alternatively, the host computer 5 controls, for example, multiple exposure apparatuses 1 and multiple evaluation apparatuses 2 installed in multiple factories.

The evaluation device 2 is connected via a wired or wireless network 6 to a management device 7 installed outside the factory. The management device 7 manages evaluation results by the evaluation device 2, for example.
(1-2) Configuration of Exposure Apparatus 1
Next, an example of the configuration of the exposure apparatus 1 of this embodiment will be described with reference to FIG. FIG. 2 is a configuration diagram showing an example of the configuration of the exposure apparatus 1 of this embodiment.

In the following description, the positional relationship of various components forming the exposure apparatus 1 will be described using an XYZ orthogonal coordinate system defined by mutually orthogonal X, Y and Z axes. In the following description, for convenience of explanation, each of the X-axis direction and the Y-axis direction is the horizontal direction (that is, a predetermined direction in the horizontal plane), and the Z-axis direction is the vertical direction (that is, the direction perpendicular to the horizontal plane). and substantially in the vertical direction). Further, the directions of rotation (in other words, tilt directions) about the X-, Y-, and Z-axes are referred to as the .theta.X direction, the .theta.Y direction, and the .theta.Z direction, respectively.

As shown in FIG. 2, the exposure apparatus 1 includes a mask stage 11, an illumination system 12, a projection optical system 13, a substrate stage 14, a plurality of sensors 15, and a controller 16.

The mask stage 11 can hold the mask 111 on which the device pattern to be transferred to the substrate 141 is formed. The mask stage 11 holds the mask 111 and is movable along a plane (for example, the XY plane) including an illumination area irradiated with the exposure light EL emitted from the illumination system 12 . For example, the mask stage 11 is moved by operation of a drive system 112 including motors. The mask stage 11 is movable along at least one of the X-axis direction, the Y-axis direction, the Z-axis direction, and the θX direction, the θY direction, and the θZ direction by the operation of the drive system 112 .

The illumination system 12 emits exposure light EL. The exposure light EL may be, for example, bright lines (eg, g-line, h-line, i-line, etc.) emitted from a mercury lamp. The exposure light EL may be, for example, far ultraviolet light (DUV light: Deep Ultra Violet light) such as KrF excimer laser light (wavelength 248 nm). The exposure light EL may be, for example, vacuum ultraviolet light (VUV light: Vacuum Ultra Violet light) such as ArF excimer laser light (wavelength 193 nm) or F2 laser light ₍ wavelength 157 nm). A portion of the mask 111 is irradiated with the exposure light EL emitted by the illumination system 12 .

The projection optical system 13 projects the exposure light EL transmitted through the mask 111 onto the substrate 141 to form an image of the device pattern formed on the mask 111 on the substrate 141 .

The substrate stage 14 holds the substrate 141 . The substrate stage 14 holds the substrate 141 and is movable along a plane (for example, the XY plane) including the projection area onto which the exposure light EL is projected by the projection optical system 13 . For example, the substrate stage 14 is moved by operation of a drive system 142 including motors. The substrate stage 14 is movable along at least one of the X-axis direction, the Y-axis direction, the Z-axis direction, and the θX direction, the θY direction, and the θZ direction by the operation of the drive system 142 . In addition, the substrate stage 14 may release the holding of the substrate 141 .

Each of the plurality of sensors 15 detects the state (operating state) of the detection target corresponding to each sensor 15 . Each of the multiple sensors 15 outputs a detection result to the control device 16 .

The object to be detected is each operation block of the exposure apparatus 1 (for example, at least one of the mask stage 11, the mask 111, the drive system 112, the illumination system 12, the projection optical system 13, the substrate stage 14, the substrate 141, and the drive system 142). one). That is, at least some of the multiple sensors 15 may detect the state of each operation block. For example, at least some of the plurality of sensors 15 may detect the temperature of each operating block. For example, at least a part of the plurality of sensors 15 can detect the position of each motion block (for example, position in the X-axis direction, position in the Y-axis direction, position in the Z-axis direction, position in the θX direction, position in the θY direction, and position in the θY direction). At least one of the positions in the θZ direction) may be detected.

The object to be detected can be any space within the exposure apparatus 1 (for example, the space between the mask 111 and the illumination system 12, the space between the mask 111 and the projection optical system 13, the space between the projection optical system 13 and the substrate 141). at least one of a space between, a space in which the mask 111 is arranged, a space in which the substrate 141 is arranged, etc.). That is, at least some of the plurality of sensors 15 may detect the state of any space within the exposure apparatus 1 . For example, at least some of the multiple sensors 15 may detect the temperature of any space. For example, at least some of the multiple sensors 15 may detect the humidity of any space. For example, at least some of the multiple sensors 15 may detect pressure (in other words, atmospheric pressure) in any space. For example, at least some of the sensors 15 may detect the cleanliness of any space.

The object to be detected may contain the exposure light EL. That is, at least some of the multiple sensors 15 may detect the state of the exposure light EL. For example, at least some of the multiple sensors 15 may detect the intensity of the exposure light EL.

The control device 16 controls the operation of the exposure device 1 as a whole. The control device 16 outputs a control command for operating each operation block of the exposure apparatus 1 to each operation block. Each operation block operates according to control commands.

For example, the controller 16 may output to the drive system 112 a control command to move the mask stage 11 in the first direction by the first movement amount within the first time. In other words, the control device 16 may output a control command including control parameters (control target values) of "first direction", "first time" and "first movement amount" to the drive system 112. . In this case, the mask stage 11 is moved in the first direction by the first movement amount within the first time by the operation of the drive system 112 .

For example, the control device 16 may output to the illumination system 12 a control command to cause the illumination system 12 to emit the exposure light EL of the second intensity for the second time. That is, the control device 16 may output to the illumination system 12 a control command including the control parameters "second intensity" and "second time." In this case, the illumination system 12 emits the exposure light EL of the second intensity for the second time.

For example, the controller 16 may output to the drive system 142 a control command to move the substrate stage 14 in the third direction by the third movement amount within the third time. That is, the control device 16 may output to the drive system 142 a control command including the control parameters "third direction", "third time" and "third movement amount". In this case, the substrate stage 14 is moved by the third movement amount in the third direction within the third time by the operation of the drive system 142 .

Note that the control command may include control parameters such as time and speed.

The control device 16 transmits to the evaluation device 2 log data used by the evaluation device 2 to evaluate the performance of the exposure device 1 . Here, log data may refer to data recording that a computer or communication device has executed (or failed to execute) a certain process. For example, the control device 16 may transmit the detection results of the multiple sensors 15 to the evaluation device 2 as at least part of the log data. For example, the control device 16 may transmit control commands to the evaluation device 2 as at least part of the log data. For example, the control device 16 uses other data such as intermediate data generated in the exposure device 1 in the process of the exposure device 1 analyzing the detection results of the plurality of sensors 15 or generating control commands as at least part of the log data. data may be transmitted to the evaluation device 2 .

Note that the exposure apparatus 1 may further include other operation blocks different from the operation blocks described above. In this case, the control device 16 may output control commands for operating the other operation blocks to the other operation blocks. Furthermore, if the exposure apparatus 1 further includes other operation blocks different from the operation blocks described above, at least some of the plurality of sensors 15 may detect the states of the other operation blocks. .

For example, if the exposure apparatus 1 is an immersion exposure apparatus, the exposure apparatus 1 may further include, as another operation block, a liquid supply device that supplies liquid to the immersion space via the liquid immersion member. good. In this case, at least some of the plurality of sensors 15 may detect the state of the liquid supply device (in particular, the state of the liquid that the liquid supply device supplies to the liquid immersion space). For example, at least some of the plurality of sensors 15 indicate the temperature, purity, pressure, and amount of liquid supplied by the liquid supply device as well as substances dissolved in the liquid (oxygen, carbon dioxide, etc.) as the state of the liquid supply device. At least one of may be detected.

For example, if the exposure apparatus 1 is an immersion exposure apparatus, the exposure apparatus 1 may further include, as another operation block, a liquid recovery device that recovers liquid from the liquid immersion space via the liquid immersion member. good. In this case, at least some of the multiple sensors 15 may detect the state of the liquid recovery device (in particular, the state of the liquid recovered from the immersion space by the liquid recovery device). For example, at least some of the plurality of sensors 15 may detect at least one of the temperature, purity, and amount of liquid recovered by the liquid recovery device.

For example, the exposure apparatus 1 may further include a measurement stage including a measuring instrument as another operation block. At least some of the plurality of sensors 15 may be sensors of a measuring machine. In this case, at least some of the multiple sensors 15 may detect the state of the measurement stage (for example, at least one of the temperature and position described above).
(1-3) Configuration of evaluation device 2
Next, an example of the configuration of the evaluation device 2 of this embodiment will be described with reference to FIG. FIG. 3 is a configuration diagram showing an example of the configuration of the evaluation device 2 of this embodiment.

As shown in FIG. 3 , the evaluation device 2 includes a CPU (Central Processing Unit) 21 , a memory 22 , an input section 23 , an operation device 24 and a display device 25 .

The CPU 21 controls operations of the evaluation device 2 . The CPU 21 performs an evaluation operation for evaluating the performance of the exposure apparatus 1, as will be detailed later.

The memory 22 stores a computer program for causing the CPU 21 to perform evaluation operations. The memory 22 also temporarily stores temporary data generated while the CPU 21 is performing evaluation operations.

Log data used by the CPU 21 to perform the evaluation operation is input to the input unit 23 from the exposure apparatus 1 . Log data is a group of data that the exposure apparatus 1 (particularly, the control apparatus 16) transmits to the evaluation apparatus 2. FIG. For example, the log data includes at least one of detection results of the multiple sensors 15, control commands, intermediate data, and the like.

As described above, the detection results of the plurality of sensors 15 indicate the state of the detection target (for example, each operation block of the exposure apparatus 1 and the space within the exposure apparatus 1). Considering that the object to be detected relates to the exposure apparatus 1 , it can be said that the state of the object to be detected indicates the operating state of the exposure apparatus 1 . Since the control command defines the operation of each operation block of the exposure apparatus 1, it can be said that the control command indicates the operation state of the exposure apparatus 1. FIG. It can be said that the intermediate data also indicates the operating state of the exposure apparatus 1 because it is data related to the detection results or control commands. Therefore, log data is a group of data indicating the operating state of exposure apparatus 1 . In other words, the log data corresponds to a set of multiple types of variables each indicating the operating state of the exposure apparatus 1 (that is, variables relating to multiple types of status items each indicating the operating state of the exposure apparatus 1). For this reason, the log data may include as many sets of variables as the status item “detection result of a certain sensor 15 ” for the number of sensors 15 . The log data may include as many sets of variables as the number of types of control commands, which indicate the status items "contents of a certain control command". The log data may include a set of variables indicating status items such as "contents of certain intermediate data" in the same number as the types of intermediate data.

The operation device 24 receives a user's operation on the evaluation device 2 . The operation device 24 may include at least one of a keyboard, mouse and touch panel, for example. The CPU 21 may perform the evaluation operation described above based on the user's operation received by the operation device 24 . However, the evaluation device 2 does not have to include the operation device 24 .

The display device 25 can display desired information. For example, the display device 25 may display information indicating the status of the evaluation device 2 . For example, display device 25 may display any information related to the evaluation operation. For example, the display device 25 may display the evaluation result by the evaluation device 2 . However, the evaluation device 2 may not include the display device 25 .

In the present embodiment, the evaluation device 2 evaluates the performance of the exposure device 1 based on the change in the relationship between the variables of the plurality of state items forming the log data acquired by the input unit 23 . The relationship between the variables of the multiple state items that make up the log data may be a correlation (correlation structure). The performance to be evaluated may include performance that is difficult or impossible to evaluate from the log data itself, but that can be evaluated from changes in the correlation structure of the log data. The above-described overlay accuracy and synchronization accuracy (for example, synchronization accuracy between the movement mode of the mask stage 11 and the movement mode of the substrate stage 14, and the X-axis direction, Y-axis direction and Z-axis direction of the mask stage 11 or the substrate stage 14) While it is difficult or impossible to evaluate from the log data itself, it is difficult or impossible to evaluate performance such as the synchronization accuracy of the movement mode along the axial direction, etc., but the performance that can be evaluated from changes in the correlation structure of the log data An example. However, the performance to be evaluated may include performance that can be evaluated from the log data itself.

In addition to or instead of at least one of the detection results of the plurality of sensors 15, control commands, intermediate data, etc., the input unit 23 stores arbitrary data indicating the operating state of the exposure apparatus 1 as log data. (i.e., as a variable).

Moreover, the evaluation device 2 may be a device that constitutes a part of the control device 16 provided in the exposure device 1 . In other words, the evaluation device 2 may be a device forming part of the exposure device 1 . Alternatively, the evaluation device 2 may be a device physically or functionally separated from the control device 16 provided in the exposure apparatus 1 (or the exposure apparatus 1 itself).

(2) Evaluation Operation by Evaluation Device 2 Subsequently, the evaluation operation performed by the evaluation device 2 (that is, the operation for evaluating the performance of the exposure device 1) will be described with reference to FIGS. 4 to 15. FIG. In the following, for convenience of explanation, after explaining the overall flow of the evaluation operation, each step constituting the evaluation operation will be explained in order.

(2-1) Overall Flow of Evaluation Operation by Evaluation Device 2 First, the overall flow of evaluation operation performed by the evaluation device 2 will be described with reference to FIG. FIG. 4 is a flowchart showing the overall flow of the evaluation operation performed by the evaluation device 2 (that is, the operation of evaluating the performance of the exposure device 1).

As shown in FIG. 4, the input unit 23 receives log data from the exposure apparatus 1, which is a set of multiple types of variables each indicating the operating state of the exposure apparatus 1 (step S1). The log data input to the input unit 23 is stored in the memory 22 .

The log data may include a first data set that is an array (for example, a chronological array) of detection results of a certain sensor 15 . In this case, the logged data includes a first data set corresponding to each sensor 15 . In other words, the log data includes the first data sets for the number of sensors 15 . For example, if the exposure apparatus 1 includes a temperature sensor that detects the temperature of the mask 111 and a temperature sensor that detects the temperature of the substrate 141, the log data is a variable indicating the temperature of the mask 111 (that is, the temperature of the mask 111 The first data set is an array of variables indicating the temperature of the substrate 141 (that is, the variables of the item of the temperature of the substrate 141). include.

The log data may include a second data set that is an array of variables that indicate certain control commands (or control parameters included in the control commands). In this case, the log data includes a second data set corresponding to each control command (or each control parameter). That is, the log data includes the second data sets as many as the number of types of control commands (or control parameters). For example, when the exposure apparatus 1 generates a first control command for moving the mask stage 11 and a second control command for moving the substrate stage 14, the log data is a variable ( That is, the second data set is an array of variables of the item of the state of the first control command), and the second data set is an array of the variables indicating the second control command (that is, the variables of the item of the state of the second control command). including datasets.

The log data may include a third data set obtained by arranging variables indicating other arbitrary data in chronological order. In this case, the log data includes a third data set corresponding to each arbitrary data. That is, the log data includes the third data set as many as the number of other arbitrary data types.

Therefore, the log data can be expressed as a matrix of variables x, as shown in Equation 1.

In Equation 1, "d" means the total number of types of variables x that constitute log data (that is, the total number of state items I). Therefore, "x _k1 (where k1 is an integer greater than or equal to 1 and less than or equal to d)" corresponds to the variable x of item I _k1 in a certain state. That is, the set of variables x _k1 in each column (x _k1 ⁽¹⁾ , x _k1 ^{( 2} _{) , .} is equivalent to the set of variables x _k1 ) of . For example, “x ₁ ” is the variable x indicating the detection result of the first sensor 15, “x ₂ ” is the variable x indicating the detection result of the second sensor 15, and “x ₃ ” is the first A variable x indicating the control command, and "x ₄ " may be a set of variables x indicating the first intermediate data.

On the other hand, in Formula 1, "N" means the number of samples of variable x of item I in a certain state included in the log data. For example, “x ^(k2) (where k2 is an integer of 1 or more and N or less) means a variable at time k2. For example, “x ₁ ⁽¹⁾ ” is the first is a variable indicating the detection result of the sensor 15, "x ₁ ⁽²⁾ " is a variable indicating the detection result of the first sensor 15 at the second time, and "x ₁ ⁽³⁾ " is the variable at the third time It may be a variable indicating the detection result of the first sensor 15 .

After that, the CPU 21 estimates a correlation structure R between the variables x of the items I of a plurality of states constituting the log data input in step S1 (step S2). The "correlation structure R" is the correlation (typically, the correlation coefficient) between the variable x of item I in one state and the variable x of item I in another state, and the variable x of item I in all states. It is obtained by calculating for x. Therefore, the correlation structure R can be expressed as a matrix of correlation coefficients r, as shown in Equation 2. In Expression 2, "r _ij " is the variable x _i (where i is an integer equal to or greater than 1 and d or less) of the state item I _i and the variable x _j of the state item I _j (where j is , an integer greater than or equal to 1 and less than or equal to d).

Especially in this embodiment, the CPU 21 creates a correlation structure R in which the number of significant correlation coefficients (that is, correlation coefficients indicating that there is some kind of correlation and is not zero) r is a predetermined amount or less. presume. In other words, the CPU 21 estimates a correlation structure R in which the number of insignificant correlation coefficients (that is, correlation coefficients indicating no correlation that are zero) r is greater than a predetermined amount. That is, the CPU 21 estimates a sparse correlation structure R. Details of the operation for estimating the sparse correlation structure R will be described later with reference to FIGS. 5 to 12. FIG.

Subsequently, the CPU 21 detects a change V in the correlation structure R estimated in step S2 (hereinafter referred to as "correlation change V" as appropriate) (step S3). “Correlation change V” is the amount of change (i.e., the difference) v is obtained by calculating for all correlation coefficients r. Therefore, the correlation change V can be expressed as a matrix of the variation v of the correlation coefficient r, as shown in Equation 3. In Expression 3, “v _ij ” indicates the amount of change between the correlation coefficient r _ij included in the first correlation structure R and the correlation coefficient r _ij included in the second correlation structure R.

In this embodiment, in order to detect the correlation change V, the CPU 21 estimates the correlation structure R two or more times. Two or more pieces of log data are input to the input unit 23 in order to estimate the correlation structure R two or more times. Typically, log data are sequentially input to the input unit 23 . Each time log data is input to the input unit 23, the CPU 21 estimates the correlation structure R of the input log data. As a result, the CPU 21 estimates the correlation structure R for the number of times log data is input to the input unit 23 .

For example, the input unit 23 receives first log data composed of a variable x indicating the operating state of the exposure apparatus 1 during the first period in which the first substrate 141 is being exposed, and then the second log data. Second log data may be input that includes a variable x that indicates the operating state of the exposure apparatus 1 during the second period in which the substrate 141 is being exposed. That is, new log data may be input to the input unit 23 each time the substrate 141 to be exposed is changed. Alternatively, for example, the input unit 23 receives third log data composed of a variable x indicating the operating state of the exposure apparatus 1 during the third period in which the first shot area on the substrate 141 is exposed. After that, the fourth log data composed of the variable x indicating the operating state of the exposure apparatus 1 during the fourth period of exposing the second shot area on the substrate 141 may be input. That is, new log data may be input to the input unit 23 each time the shot area to be exposed changes. Alternatively, the input unit 23 receives fifth log data composed of a variable x indicating the operating state of the exposure apparatus 1 during an arbitrary fifth period, and then, after that, the exposure apparatus 1 during an arbitrary sixth period. Sixth log data may be input, which consists of a variable x indicating the operating state of the . For example, the input unit 23 receives log data composed of a variable x indicating the operating state of the exposure apparatus 1 at a first time during the period when one shot area on the substrate 141 is being exposed. Log data composed of a variable x that indicates the operating state of the exposure apparatus 1 at a second time during the exposure of the upper shot area may be input.

After that, the CPU 21 determines (or predicts, the same shall apply hereinafter) whether the performance of the exposure apparatus 1 is good or bad based on the correlation change V detected in step S3 (step S4). In this embodiment, the CPU 21 calculates a score SC according to each amount of change v that constitutes the correlation change V. FIG. The CPU 21 determines whether the performance of the exposure apparatus 1 is good or bad based on the score SC.

After that, the display device 25 displays the judgment result (prediction result) of the performance of the exposure apparatus 1 in step S4 as necessary (step S5). Details of the operation for displaying the determination result will be described later with reference to FIG. 15 .

(2-2) Estimation Operation of Correlation Structure R Subsequently, the “estimation operation of correlation structure R (step S2 in FIG. 4)” among the evaluation operations performed by the evaluation device 2 will be described with reference to FIGS. do. In this embodiment, the evaluation device 2 performs the first estimation operation, the second estimation operation, or the third estimation operation as the correlation structure R estimation operation. The first to third estimation operations will be described below in order.

In the estimation operation described below, the CPU 21 estimates a sparse correlation structure R using a sparse structure learning method. Furthermore, in the estimation operation described below, an algorithm called Graphical Lasso is adopted as an algorithm for realizing the sparse structure learning method.

However, other algorithms (for example, Cluster Graphical Lasso, etc.) may be adopted as algorithms for realizing the sparse structure learning method. Furthermore, the CPU 21 may estimate the sparse correlation structure R using a method different from the sparse structure learning method.

(2-2-1) First estimation operation of correlation structure R First, the first estimation operation of correlation structure R will be described with reference to FIG. FIG. 5 is a flow chart showing the flow of the first correlation structure R estimation operation.

As shown in FIG. 5, the CPU 21 sets a regularization parameter ρ that defines a regularization term in the sparse structure learning method (step S211). Here, the CPU 21 may set the regularization parameter ρ to any value or desired value (eg, default value). Alternatively, the CPU 21 may set the regularization parameter ρ according to a user's instruction specifying the regularization parameter ρ.

Note that the number of significant correlation coefficients r included in the correlation structure R decreases as the regularization parameter ρ increases. In other words, as the regularization parameter ρ increases, the number of insignificant (that is, zero) correlation coefficients r included in the correlation structure R increases. That is, as the regularization parameter ρ increases, the sparsity of the correlation structure R improves (in other words, the dimension of the correlation structure R decreases).

Thereafter, the CPU 21 performs arithmetic processing based on the sparse structure learning method employing Graphical Lasso on the log data acquired in step S1 of FIG. 4 (step S212).

Arithmetic processing based on the sparse structure learning method employing Graphical Lasso is a computational processing for identifying a precision matrix Λ that satisfies Equation 4 using log data. In other words, the CPU 21 specifies the precision matrix Λ by performing arithmetic processing based on the sparse structure learning method on the log data (step S213). In Expression 4, "S" indicates a variance-covariance matrix (sample covariance matrix) of log data. The variance-covariance matrix S corresponds to the inverse matrix of the precision matrix Λ. The element S _ij at the i-th row and the j-th column of the variance-covariance matrix S can be specified by Equation (5). In Expression 4, "tr(SΛ)" is a function indicating the diagonal sum of the matrix SΛ. In Equation 5, 'u _i ' and 'u _j ' can be identified by Equations 6 and 7, respectively.

After that, the CPU 21 normalizes (in other words, standardizes or normalizes) the precision matrix Λ specified in step S213 using Equation 8 (step S214). As a result, the CPU 21 can generate a partial correlation matrix of log data (step S214). This partial correlation matrix corresponds to the correlation structure R. Note that normalization is performed using Equation 8. In Equation 8, let the element of the i-th row and j-th column of the precision matrix Λ before normalization be Λ _ij .

After that, the CPU 21 controls the display device 25 so as to display the correlation structure R calculated in step S214 in a desired display form as necessary (step S215). As a result, the display device 25 displays the correlation structure R calculated in step S214 in a desired display form.

Here, the display form of the correlation structure R will be described with reference to FIGS. 6 to 8. FIG. FIG. 6 is a plan view showing the basic display form of the correlation structure R. FIG. FIG. 7 is a plan view showing a first specific example of the display form of the correlation structure R. FIG. FIG. 8 is a plan view showing a second specific example of the display form of the correlation structure R. FIG.

As shown in FIG. 6, the display device 25 displays the correlation structure R by displaying display objects such as nodes n and edges e.

Node n is a display object corresponding to item I in a certain state. Therefore, the number of nodes displayed by the display device 25 matches the number of status items I (that is, the number of types of variable x). FIG. 6 shows a node n ₁ corresponding to the state item I ₁ , a node n ₂ corresponding to the state item I ₂ , a node n ₃ corresponding to the state item I ₃ , and a node n corresponding to the state item I ₄ . ₄ , node _n5 corresponding to state item _I5 , node _n6 corresponding to state item _I6 , node _n7 corresponding to state item _I7 , node _n8 corresponding to state item _I8 , . . , and a node n _d corresponding to the state item I _d is displayed.

An edge e is a display object that associates (in other words, connects or connects) two nodes n. In particular, the edge e is the correlation coefficient r between the two state items I corresponding to the two nodes n (that is, the variable x of item I in one state and the variable x of item I in the other state). It is a display object that associates two nodes n in a display form according to the correlation coefficient r) between them. FIG. 6 shows an example where an edge e is a linear display object that associates two nodes n.

As an example of the display form of the edge e according to the correlation coefficient r, the edge e that associates the two nodes n corresponding to the items I in the two states in which the correlation coefficient r is zero is not displayed, while the correlation coefficient r There is a display form in which an edge e that associates two nodes n corresponding to two states of item I in which r is not zero is displayed. For example, FIG. 6 shows edge e ₁₄ that associates node n ₁ and node n ₄ , edge e ₁₆ that associates node n ₁ and node n ₆ , edge e ₁₇ that associates node n ₁ and node n ₇ , node n ₄ shows an example in which edge e ₄₅ that associates node n 4 with node n ₅ and edge e _6d that associates node _{n 6} with node nd are displayed. In this case, the correlation coefficient r ₁₄ , the correlation coefficient r ₁₆ , the correlation coefficient r ₁₇ , the correlation coefficient r ₄₅ and the correlation coefficient r _6d are not zero, while the other correlation coefficients r are zero. .

As an example of the display form of the edge e according to the correlation coefficient r, the appearance of the edge e that associates two nodes n can be changed according to the correlation coefficient r between the two state items I corresponding to the two nodes n. There is a display form of changing. For example, the edge e may be displayed such that the appearance of the edge e changes depending on the magnitude of the absolute value of the correlation coefficient r. Specifically, for example, the edge e may be displayed such that the edge e becomes thicker or darker as the absolute value of the correlation coefficient r increases. For example, the edge e may be displayed such that the appearance of the edge e changes depending on the sign of the correlation coefficient r. Specifically, for example, the edge e may be displayed such that the appearance of the edge e when the correlation coefficient r is negative differs from the appearance of the edge e when the correlation coefficient r is positive. good. Also, the edge e may be displayed such that the color of the edge e changes according to the magnitude of the correlation coefficient r.

In this embodiment, the display device 25 may display the correlation structure R in a display form that associates (in other words, associates) the correlation structure R with the exposure apparatus 1 .

As an example of a display form that associates the correlation structure R with the exposure apparatus 1, there is a display form that associates the correlation structure R with a plurality of groups obtained by classifying a plurality of operation blocks constituting the exposure apparatus 1 according to predetermined criteria. can give. The operation block may have an independent function, be replaceable or detachable, and may be configured into a larger system.

Specifically, as described above, the correlation structure R is displayed using nodes n and edges e. Node n corresponds one-to-one to item I in a state. A certain state item I indicates the state of a certain operation block of the exposure apparatus 1, and therefore can be said to be related to the certain operation block. That is, each node n is associated with a corresponding operational block of exposure apparatus 1 . In this display form, nodes n related to operation blocks belonging to the same group according to a predetermined standard are displayed as a group of nodes so as to be close to each other (in other words, collectively). It should be noted that node n may correspond to item I in a certain state on a one-to-N or M-to-1 basis (N and M are arbitrary numbers).

As an example of a display form that associates the correlation structure R with the exposure apparatus 1, the correlation structure R is classified into a plurality of spaces that can be physically, functionally, or conveniently divided within the exposure apparatus 1 according to predetermined criteria. A display form associated with a plurality of groups obtained by the above can be mentioned. Specifically, since item I of a certain state indicates the state of a certain space in the exposure apparatus 1, it can be said that it is related to the certain space. That is, each node n is associated with a certain space within the exposure apparatus 1 . In this display mode, the nodes n related to the space belonging to the same group according to a predetermined standard are displayed as a group of nodes so as to be close to each other (in other words, collectively).

The “predetermined criteria” used for classifying the plurality of operation blocks constituting the exposure apparatus 1 or the plurality of spaces that can be divided within the exposure apparatus 1 are, for example, the characteristics (in other words, specifications, etc.) of the exposure apparatus 1. It is a standard that considers

The characteristics of exposure apparatus 1 may depend on the functions of a plurality of operation blocks that constitute exposure apparatus 1 . Therefore, the “predetermined criterion” may be a criterion that considers the functions of the plurality of operation blocks that constitute the exposure apparatus 1 . In this case, the correlation structure R is displayed so as to be associated with a plurality of groups obtained by classifying the plurality of operation blocks constituting the exposure apparatus 1 according to the functional difference of the plurality of operation blocks. Specifically, nodes n related to operation blocks for realizing a certain function are displayed as a group of nodes so as to be close to each other (in other words, collectively).

For example, the mask stage 11 is an operation block for realizing a first function, the illumination system 12 is an operation block for realizing a second function, and the projection optical system 13 is an operation block for realizing a third function. , the substrate stage 14 and the substrate 141 are operation blocks for realizing the fourth function, and the driving systems 112 and 142 are operation blocks for realizing the fifth function. Suppose that it can be classified as In this case, a node n corresponding to the state item I indicating the state of the mask stage 11 and a node n corresponding to the state item I indicating the state of the mask 111 are displayed as a group of nodes. Furthermore, the node n corresponding to the item I of the state indicating the state of the illumination system 12 is displayed as a group of nodes. Furthermore, the node n corresponding to the item I of the state indicating the state of the projection optical system 13 is displayed as a group of nodes. Furthermore, a node n corresponding to the status item I indicating the status of the substrate stage 14 and a node n corresponding to the status item I indicating the status of the substrate 141 are displayed as a group of nodes. Furthermore, a node n corresponding to the status item I indicating the status of the drive system 112 and a node n corresponding to the status item I indicating the operating status of the drive system 142 are displayed as a group of nodes.

As a result, as shown in FIG. 7, a plurality of nodes n respectively corresponding to a plurality of status items I are displayed so as to be differentiated in units of node groups including nodes n related to the same function. FIG. 7 shows a node group including three nodes n related to function F#1, a node group including one node n related to function F#2, and two nodes related to function F#3. n, 5 nodes n associated with function F#4, 9 nodes n associated with function F#5, and 3 nodes n associated with function F#6. It shows an example in which a node group including node n, a node group including four nodes n related to function F#7, and a node group including one node n related to function F#8 are displayed. .

A plurality of operation blocks for realizing the same function have a relatively strong tendency to operate in the same way for realizing the same function. Therefore, the variable x of the plurality of state items I, which respectively indicate the states of a plurality of operation blocks for realizing the same function, tends to have a relatively strong positive correlation. Furthermore, each variable x of a plurality of state items I related to a certain operation block for realizing a certain function has a relatively strong tendency to change in the same way because it indicates the state of the same operation block. . Therefore, the variable x of the item I of multiple states related to a certain operation block for realizing a certain function also tends to have a relatively strong positive correlation. Therefore, as shown in FIG. 7, relatively many edges e that associate multiple nodes n related to the same function are displayed.

On the other hand, a plurality of operation blocks for implementing different functions have a relatively weak tendency to operate in the same way because they implement different functions. Therefore, between the variable x of the state item I indicating the state of an action block for realizing a certain function and the variable x of the state item I indicating the state of another action block for realizing another function , there is a relatively strong tendency of no correlation. For this reason, as shown in FIG. 7, not many edges e that associate multiple nodes n, each associated with a different function, are displayed.

The characteristics of the exposure apparatus 1 may depend on the positions at which the plurality of operation blocks forming the exposure apparatus 1 are arranged. Therefore, the “predetermined reference” may be a reference that takes into consideration the arrangement of the plurality of operation blocks that make up the exposure apparatus 1 . In this case, the correlation structure R is displayed so as to be associated with a plurality of groups obtained by classifying the plurality of operation blocks constituting the exposure apparatus 1 according to the difference in arrangement of the plurality of operation blocks. Specifically, in this display form, the nodes n associated with the action blocks arranged at the same or close positions are displayed as a group of nodes so as to be close to each other (in other words, collectively). be.

For example, the mask stage 11, the mask 111, the drive system 112, and the illumination system 12 are arranged in the upper region within the exposure apparatus 1, and the projection optical system 13 is arranged in the middle region within the exposure apparatus 1, It is assumed that the substrate stage 14 , the substrate 141 and the drive system 142 can be classified as arranged in the lower region within the exposure apparatus 1 . In this case, the node n corresponds to the state item I indicating the state of the mask stage 11, the node n corresponds to the state item I indicating the state of the mask 111, and the state item I indicates the state of the driving system 112. A node n corresponding to a state item I indicating the state of the node n and the illumination system 12 is displayed as a group of nodes. Furthermore, the node n corresponding to the item I of the state indicating the state of the projection optical system 13 is displayed as a group of nodes. Further, a node n corresponding to the state item I indicating the state of the substrate stage 14, a node n corresponding to the state item I indicating the state of the substrate 141, and a node corresponding to the state item I indicating the state of the drive system 142. n is displayed as a group of nodes.

As a result, as shown in FIG. 8, a plurality of nodes n corresponding to items I of a plurality of states are displayed so as to be distinguished in units of node groups containing nodes n related to the same or close positions. be done. Note that FIG. 8 shows a node group including two nodes n associated with position P#1 near the upper region, a node group including two nodes n associated with position P#2 near the upper region, and a node group including two nodes n associated with position P#2 near the upper region. A node group containing two nodes n associated with nearby position P#3, a node group containing four nodes n associated with position P#4 near the central region, and a node group containing four nodes n associated with position P#5 near the central region. , a node group including three nodes n associated with position P#6 near the central region, and a node group including four nodes n associated with position P#7 near the central region An example is shown in which a group and a node group containing 8 nodes n associated with position P#8 near the bottom region are displayed.

The states of a plurality of operation blocks arranged at the same or close positions tend to change in the same manner. Therefore, the variable x of multiple state items I associated with multiple operation blocks located at the same or adjacent positions has a relatively strong tendency to have a positive correlation. Furthermore, the variable x of the multiple state items I associated with a certain action block placed at a certain position has a relatively strong tendency to change in the same way because it indicates the state of the same action block. Therefore, the variable x of the multiple state item I associated with an action block placed at a certain position also has a relatively strong tendency to have a positive correlation. For this reason, as shown in FIG. 8, a relatively large number of edges e that associate a plurality of nodes n related to action blocks arranged at the same or close positions are displayed.

On the other hand, the motion states of a plurality of motion blocks arranged at positions separated from each other have a relatively weak tendency to change in the same manner. Therefore, the variable x of the state item I indicating the state of an action block placed at a certain position and the state item I indicating the state of another action block placed at another position away from the certain position There is a relatively strong tendency that there is no correlation with the variable x. For this reason, as shown in FIG. 8, not many edges e that associate a plurality of nodes n that are arranged at positions apart from each other are displayed.

It should be noted that the characteristics of the exposure apparatus 1 may also depend on the function or arrangement of multiple spaces within the exposure apparatus 1 . For this reason, the "predetermined criteria" may be criteria that take into account the functions of a plurality of spaces within the exposure apparatus 1 or the arrangement of the spaces. In this case, the correlation structure R may be associated with a plurality of groups obtained by classifying the plurality of spaces in the exposure apparatus 1 according to at least one difference between the functions of the plurality of spaces and the arrangement of the spaces. good.

Alternatively, the properties of the exposure apparatus 1 may depend on other properties of the operating blocks or spaces of the exposure apparatus 1 (ie properties other than layout and function). Therefore, the “predetermined criterion” may be a criterion that takes into consideration other characteristics of the plurality of components of the exposure apparatus 1 or the plurality of spaces. In this case, the correlation structure R associates a plurality of groups obtained by classifying a plurality of motion blocks or a plurality of spaces of the exposure apparatus 1 according to differences in other characteristics of the plurality of motion blocks or the plurality of spaces. may be

According to such a display form, a user viewing the correlation structure R displayed on the display device 25 can preferably and easily recognize the node n having correlation. In other words, the user can easily visually recognize whether or not the variable x of item I in which state has a correlation with the variable x of item I in which state. As a result, for example, the user can easily visually recognize whether or not the state of the motion block or space related to which function or arrangement (position) of the exposure apparatus 1 changes in the same way. can be done.

For example, FIG. 9A shows a display example of the correlation structure R when the temperature of the operation block located in the lower region of the exposure apparatus 1 increases. When the temperature of the operation blocks in the lower area increases, the temperatures of the plurality of operation blocks in the lower area also increase, while the temperature of the operation blocks located in the upper area is less likely to increase. Therefore, the temperature correlation between the plurality of operation blocks located in the lower area is a positive correlation (that is, the correlation coefficient has a value of +1 or close to +1). On the other hand, the temperature correlation between the operating blocks located in the lower area and the operating blocks located in the upper area tends to disappear (that is, the correlation coefficient tends to have a value of zero or close to zero). Therefore, as shown in FIG. 9A, a relatively large number of edges e that associate a plurality of nodes n associated with a plurality of motion blocks located in the lower area are displayed. On the other hand, as shown in FIG. 9(a), the edge e that associates the node n related to the action block located in the lower area with the node n related to the action block located in the upper area is displayed so many times. never. Therefore, by visually recognizing the correlation structure R shown in FIG. 9(a), the user can easily recognize that the states (here, temperatures) of the operation blocks located in the lower region are similarly changing. be able to.

For example, FIG. 9B shows a display example of the correlation structure R when the temperature of the operation block located in the upper region of the exposure apparatus 1 increases. In this case, a relatively large number of edges e that associate a plurality of nodes n associated with a plurality of motion blocks located in the upper region are displayed. On the other hand, the edge e that associates the node n associated with the action block located in the upper area with the node n associated with the action block located in the lower area is not displayed so often. Therefore, by visually recognizing the correlation structure R shown in FIG. 9B, the user can easily recognize that the states (here, temperatures) of the operation blocks located in the upper region change in the same way. be able to.

Note that FIG. 9C shows a display example of the correlation structure R when the temperature of the action block located in the lower region increases under the condition that the regularization parameter ρ is not appropriately set. Here, in a situation where the regularization parameter ρ is not set appropriately, the regularization parameter ρ is set to a value smaller than the desired value (for example, the regularization parameter ρ set in the example shown in FIG. 9A). Assume the set situation. When the regularization parameter ρ is set to a value smaller than the desired value, the number of insignificant (i.e., zero not) the number r of correlation coefficients increases. For this reason, as shown in FIG. 9C, not only the edge e that associates the plurality of nodes n related to the plurality of motion blocks located in the lower region, but also the node n related to the motion blocks located in the lower region. Edges e that associate nodes n associated with action blocks located in the upper region may also be displayed correspondingly more.

On the other hand, FIG. 9(d) shows a display example of the correlation structure R when the temperature of the action block located in the upper region increases under the condition that the regularization parameter ρ is not appropriately set. Here, in a situation where the regularization parameter ρ is not set appropriately, the regularization parameter ρ is set to a desired value (for example, a value smaller than the regularization parameter ρ set in the example shown in FIG. 9B If the regularization parameter ρ is set to a value smaller than the desired value, then the correlation structure R is more affected than if the regularization parameter ρ is set to the desired value The number r of included insignificant (that is, non-zero) correlation coefficients increases, so as shown in FIG. Not only the associating edges e, but also the edges e associating the node n associated with the action block located in the upper region and the node n associated with the action block located in the lower region may be displayed correspondingly more. .

In this way, if the regularization parameter ρ is not set appropriately, the user cannot determine which function is to be realized or which motion blocks are arranged at which positions, or whether the state of the space changes in the same way. It may not be possible to easily visually recognize whether or not Therefore, when specifying the correlation structure R, the CPU 21 may set an appropriate regularization parameter ρ that can facilitate visual recognition of the correlation structure R by the user. Each of the second and third estimation operations to be described later corresponds to an operation example in which an operation for setting an appropriate regularization parameter ρ is added to the first estimation operation.

(2-2-2) Second Estimation Operation of Correlation Structure R Subsequently, the second estimation operation of the correlation structure R will be described with reference to FIG. FIG. 10 is a flow chart showing the flow of the second correlation structure R estimation operation. Note that the second estimation operation differs from the first estimation operation in that a cross validation method is used to determine the regularization parameter ρ. Other operations of the second estimation operation may be the same as other operations of the first estimation operation. Therefore, the same step numbers are assigned to the same operations as those performed by the first estimation operation, and detailed descriptions thereof are omitted.

As shown in FIG. 10, the CPU 21 sets a regularization parameter ρ (step S211).

Thereafter, the CPU 21 divides the log data acquired in step S1 of FIG. 4 into K (where K is an integer equal to or greater than 2) pieces of divided data (step S222).

After that, the CPU 21 selects K−1 divided data. Furthermore, the CPU 21 performs arithmetic processing based on the sparse structure learning method employing Graphical Lasso on the selected K−1 divided data (step S223). As a result, the CPU 21 identifies the precision matrix Λ. After that, the CPU 21 calculates the likelihood of the precision matrix Λ specified from the K−1 pieces of divided data using the remaining unselected piece of divided data (step S224). The CPU 21 performs the above operations of steps S223 and S224 for all combinations of K−1 pieces of divided data selectable from K pieces of divided data (step S225). Therefore, the CPU 21 performs the operations of steps S223 and S224 described above K times (step S225). As a result, the CPU 21 calculates K likelihoods. After that, the CPU 21 calculates an average of K likelihoods (step S226).

After that, the CPU 21 determines whether or not the average of likelihoods has been calculated a first predetermined number of times or more (step S227).

As a result of the determination in step S227, when it is determined that the average of likelihoods has not been calculated for the first predetermined number of times or more (step S227: No), the CPU 21 performs steps S211 to S226 again. At this time, the CPU 21 changes the regularization parameter ρ. That is, the CPU 21 sets a value that has not yet been set to the regularization parameter ρ (step S211). After that, the CPU 21 completes the calculation of the average likelihood using the changed regularization parameter ρ.

On the other hand, as a result of the determination in step S227, when it is determined that the average likelihood has been calculated for the first predetermined number of times or more (step S227: Yes), the CPU 21 sets the average likelihood to the maximum. is selected (step S228). Thereafter, the CPU 21 uses the regularization parameter ρ selected in step S228 to perform arithmetic processing based on the sparse structure learning method employing Graphical Lasso on the log data acquired in step S1 of FIG. 4 (step S229 ). Thereafter, in the second estimation operation as well, the operations from step S213 to step S215 are performed in the same manner as in the first estimation operation.

According to the second estimation operation, the CPU 21 can narrow down the regularization parameter ρ to an appropriate value using a cross-validation technique. Therefore, the CPU 21 can estimate the correlation structure R more appropriately. Note that the CPU 21 may narrow down the regularization parameter ρ to an appropriate value using an alternative estimation method or a test sample method.

(2-2-3) Third Estimating Operation of Correlation Structure R Subsequently, the third estimating operation of the correlation structure R will be described with reference to FIG. FIG. 11 is a flow chart showing the flow of the third correlation structure R estimation operation. Note that the third estimation operation differs from the first estimation operation in that the regularization parameter ρ is determined in consideration of the characteristics of the exposure apparatus 1 . Other operations of the third estimation operation may be the same as other operations of the first estimation operation. Therefore, the same step numbers are assigned to the same operations as those performed by the first estimation operation, and detailed descriptions thereof are omitted.

As shown in FIG. 11, the CPU 21 sets a regularization parameter ρ (step S211).

Thereafter, the CPU 21 performs arithmetic processing based on the sparse structure learning method employing Graphical Lasso on the log data acquired in step S1 of FIG. 4 (step S223). As a result, the CPU 21 identifies the precision matrix Λ.

After that, the CPU 21 calculates the edge density D based on the specified accuracy matrix Λ (step S232).

Here, the edge density D will be explained. As described above, the correlation structure R is estimated from the accuracy matrix Λ. Furthermore, as mentioned above, the correlation structure R is represented using nodes n and edges e. “Edge density D” in this embodiment means the density of edges e used to display the correlation structure R. FIG. However, the edge e that associates the two nodes n corresponding to the two-state item I for which the correlation coefficient r is zero is not displayed, while the edge e corresponding to the two-state item I for which the correlation coefficient r is not zero is not displayed. A display form in which an edge e that associates two nodes n is displayed is a prerequisite for calculating the edge density D. FIG. Therefore, the edge density e can also be said to be a parameter that indicates the ratio of significant (that is, non-zero) correlation coefficients r to all the elements forming the correlation structure R.

Furthermore, the “edge density D” in this embodiment is not simply the density of the edges e, but the density of the edges e calculated in consideration of the characteristics of the exposure apparatus 1 . Specifically, as described above, nodes n related to motion blocks (or spaces) belonging to the same group according to a predetermined criterion (for example, a criterion considering function or arrangement (position)) are grouped is displayed as a group of nodes. As a result, there is a relatively strong tendency for edges e that associate different nodes n belonging to different node groups not to be displayed. In other words, the density of edges e that associate different nodes n belonging to different node groups becomes relatively small. On the other hand, edges e that associate different nodes n belonging to the same node group are relatively likely to be displayed. That is, the density of edges e that associate different nodes n belonging to the same node group is relatively large. In this way, the edge density D has some correlation with the group (that is, node group) that classifies the node n. The "edge density D" in this embodiment is the density of the edge e calculated in consideration of the fact that the group (that is, node group) for classifying the node n and the edge density D have some kind of correlation. be. In other words, the "edge density D" of the present embodiment is an edge density considering a plurality of groups obtained by classifying the operation blocks (or spaces) of the exposure apparatus 1 according to a predetermined criterion considering the characteristics of the exposure apparatus. is the density of e (in other words, the density of edges e between different groups).

For example, the CPU 21 can calculate the edge density D using Equation 9. Note that “Λ _ij ” in Expression 9 indicates the element of the i-th row and j-th column of the precision matrix Λ. “sgn(Λ _ij )” in Equation 9 is +1 when Λ _ij >0, 0 when Λ _ij =0, and −1 when Λ _ij <0. “C _k ” in Equation 9 is a part of the elements of the precision matrix Λ that indicates the correlation coefficient r of the items I of multiple states belonging to the same group (for example, the same function or the same arrangement) (that is, the correlation It is a matrix generated by extracting the relational coefficient r). "C _k '" in Equation 9 is an element that indicates the correlation coefficient r of items I of a plurality of states belonging to the same group among the elements of the precision matrix Λ and is different from the elements that make up the matrix C _k is a matrix generated by extracting . Specifically, as shown in FIG. 12, the accuracy matrix Λ is composed of a matrix C ₁ composed of elements indicating correlation coefficients r of items I of a plurality of states belonging to the first group, and A matrix C ₂ composed of elements indicating the correlation coefficient r of a plurality of state items I belonging to , . It contains a matrix Cs composed of elements indicating the correlation coefficient _r of I and a matrix composed of elements indicating the correlation coefficient r of items I of multiple states belonging to different groups. Each of “C _k ” and “C _k ′” in Equation 9 is sequentially selected from matrix C ₁ to matrix C _s shown in FIG.

However, depending on the type of performance of the exposure apparatus 1 to be evaluated by the evaluation operation, there may be an item I that greatly contributes to the evaluation and an item I that does not contribute much to the evaluation. In this case, when calculating the edge density D, the CPU 21 does not need to consider the variable x of the item I that does not contribute much to the evaluation. That is, the CPU 21 may calculate the edge density D in consideration of the type of performance of the exposure apparatus 1 to be evaluated by the evaluation operation. For example, if the matrix _{C2 among} the matrices C1 to Cs shown in _FIG . It is not necessary to choose the matrix C ₂ as each of ' _k ' and 'C _k ''.

After that, the CPU 21 determines whether or not the edge density D has been calculated a second predetermined number of times or more (step S233).

As a result of the determination in step S233, when it is determined that the edge density D has not been calculated more than the second predetermined number of times (step S233: No), the CPU 21 repeats steps S211 to S232. At this time, the CPU 21 changes the regularization parameter ρ. That is, the CPU 21 sets a value that has not yet been set to the regularization parameter ρ (step S211). After that, the CPU 21 completes the calculation of the edge density D using the changed regularization parameter ρ.

On the other hand, as a result of the determination in step S233, when it is determined that the edge density D has been calculated more than the second predetermined number of times (step S233: Yes), the CPU 21 sets the threshold value δ (step S234).

The threshold δ is a parameter used to set an appropriate edge density D and, consequently, an appropriate regularization parameter ρ. Specifically, as described with reference to FIGS. 9(a) to 9(d), when the regularization parameter ρ is set appropriately, the regularization parameter ρ is not set properly. By comparison, the edge density D becomes smaller. When the edge density D is relatively small (see FIG. 9(a) or 9(b)), the edge density D is relatively large (see FIG. 9(c) or 9(d)). In comparison, the user can easily visually recognize changes in the operating state of the exposure apparatus 1 . For this reason, the edge density D may be as small as possible so that the user can easily visually recognize the change in the operating state of the exposure apparatus 1 .

Therefore, the CPU 21 allows the user to visually recognize the change in the operating state of the exposure apparatus 1 when the edge density D is small enough to allow the user to easily visually recognize the change in the operating state of the exposure apparatus 1 . The threshold value δ is set so as to be a value capable of distinguishing a situation in which the edge density D is so large that it cannot be easily recognized.

Here, as shown in FIG. 12 , each element of the precision matrix Λ includes an element indicating the correlation coefficient r of the multiple state items I belonging to the same group, and an element showing the correlation coefficient r of the multiple state items I belonging to different groups. As described above, the element indicating the correlation coefficient r is included. For this reason, the correlation structure R also has an element indicating the correlation coefficient r of the items I in a plurality of states belonging to the same group, and an element indicating the correlation coefficient r of the items I in a plurality of states belonging to different groups. including. Here, the absolute value of the element indicating the correlation coefficient r of the items I of multiple states belonging to the same group is relatively large. On the other hand, the absolute value of the element indicating the correlation coefficient r of items I in a plurality of states belonging to different groups becomes relatively small or zero. That is, considering the characteristics of the exposure apparatus 1 (for example, grouping of components or spaces considering characteristics), the accuracy matrix Λ (or the correlation structure R) may be a sparse matrix in the first place. In other words, considering the characteristics of the exposure apparatus 1, the edge density D may be a reasonably small value in the first place. In other words, it can be said that the grouping of operation blocks or spaces considering the function or arrangement (position), which is one of the factors that determine the characteristics of the exposure apparatus 1, specifies the default edge density D substantially.

Therefore, the CPU 21 sets the threshold δ in consideration of the characteristics of the exposure apparatus 1 . For example, the ratio of the elements showing the correlation coefficient r of the items I in a plurality of states belonging to different groups to all the elements constituting the accuracy matrix Λ is substantially determined by considering the characteristics of the exposure apparatus 1. The ratio corresponds to the default edge density D. Therefore, the CPU 21 may set the threshold value δ in consideration of the ratio of the elements showing the correlation coefficient r of the items I of a plurality of states belonging to different groups to all the elements constituting the precision matrix Λ. good. For example, the CPU 21 sets the edge density D corresponding to the ratio of the elements showing the correlation coefficient r of the items I of the plurality of states belonging to different groups to all the elements constituting the accuracy matrix Λ as the threshold value δ. You may For example, the CPU 21 sets a predetermined margin for the edge density D corresponding to the ratio of the elements showing the correlation coefficient r of the items I of the plurality of states belonging to different groups to all the elements constituting the precision matrix Λ. A value obtained by adding or subtracting may be set as the threshold value δ.

Alternatively, in addition to or instead of considering the characteristics of the exposure apparatus 1, the CPU 21 may set the threshold value δ in consideration of the type of performance of the exposure apparatus 1 to be evaluated by the evaluation operation. For example, as an example of the performance of the exposure apparatus 1, there may be a performance in which the accuracy of pass/fail judgment varies according to the edge density D. FIG. In this case, the CPU 21 considers the relationship between the edge density D and the accuracy of determining the quality of the type of performance to be evaluated, and determines the edge density that enables the quality of the performance to be evaluated to be appropriately determined. The threshold value δ may be set so as to be a value capable of distinguishing D from the edge density D at which it is difficult to appropriately determine the quality of the performance to be evaluated.

Specifically, for example, as an example of the performance of the exposure apparatus 1, it is possible to appropriately determine the quality based on the correlation structure R with a relatively low edge density D, while the correlation structure R with a relatively high edge density D There may be performances for which it is difficult to make good or bad judgments based on the structure R. When evaluating such performance, the CPU 21 may set a relatively small value as the threshold value δ. Alternatively, for example, as an example of the performance of the exposure apparatus 1, there may be a performance that can be suitably judged based on the correlation structure R with the relatively large edge density D. When evaluating such types of performance, the CPU 21 may set a relatively large value as the threshold value δ.

Referring back to FIG. 11, after that, the smallest regularization parameter ρ among the regularization parameters ρ with which the edge density D becomes smaller than the threshold value δ is selected (step S236). Thereafter, the CPU 21 uses the regularization parameter ρ selected in step S236 to perform arithmetic processing based on the sparse structure learning method employing Graphical Lasso on the log data acquired in step S1 of FIG. 4 (step S237 ). Thereafter, in the third estimation operation as well, the operations from step S213 to step S215 are performed in the same manner as in the first estimation operation.

According to the third estimation operation, the CPU 21 can narrow down the regularization parameter ρ to an appropriate value in consideration of the characteristics of the exposure apparatus 1 . Therefore, the CPU 21 can estimate the correlation structure R more preferably.

(2-3) Detection Operation of Correlation Change V Subsequently, the detection operation of the correlation change V (step S3 in FIG. 4) among the evaluation operations performed by the evaluation device 2 will be described.

As described above, the CPU 21 calculates the amount of change v between a certain correlation coefficient r included in the first correlation structure R and the same type of correlation coefficient r included in the second correlation structure R as all The correlation change V is detected by calculating the correlation coefficient r. When three or more correlation structures R are estimated, the CPU 21 may detect changes V in two correlation structures R estimated in order along the time series among the three or more correlation structures R. good. For example, when the first correlation structure R, the second correlation structure R, the third correlation structure R, and the fourth correlation structure R are estimated in this order, the CPU 21 calculates the first and second correlation structures A change V in R, a change V in the second and third correlation structures R, and a change V in the third and fourth correlation structures R may be detected.

However, the CPU 21 may detect the correlation change V without estimating the correlation structure R from the log data acquired in step S1 of FIG. 4 (that is, without performing the operation of step S2 in FIG. 4). . The CPU 21 may use a probability density ratio estimation method (DRE: Density Ratio Estimation) as a technique for detecting the correlation change V from the log data. In this case, since the correlation structure R does not need to be estimated, the accuracy of the detection result of the correlation change V is not affected by the estimation accuracy of the correlation structure R. Therefore, the CPU 21 can detect the correlation change V with relatively high accuracy. Furthermore, the CPU 21 can detect nonlinear correlation change V by adopting a polynomial basis function as a basis function for the probability density ratio estimation method. Furthermore, the CPU 21 can preferably detect the correlation change V even when the correlation structure R from which the correlation change V is detected is dense (that is, there are relatively many significant elements). .

On the other hand, when the correlation change V is detected by estimating the correlation structure R, there is an advantage that the structure of the correlation structure R becomes clear. Therefore, the CPU 21 compares the advantage obtained by detecting the correlation change V by estimating the correlation structure R and the advantage obtained by detecting the correlation change V without estimating the correlation structure R. , a more suitable technique for detecting the correlation change V may be employed.

Similar to the correlation structure R, the detected correlation change V becomes a matrix of d rows and d columns. Therefore, the CPU 21 may control the display device 25 so as to display the correlation change V in a desired display form in the same manner as the correlation structure R, if necessary. The display form of the correlation change V may be the same as the display form of the correlation structure R. FIG. Therefore, detailed description of the display form of the correlation change V is omitted.

(2-3) Performance Judgment Operation Next, among the evaluation operations performed by the evaluation device 2, the judgment operation of the performance of the exposure device 1 (step S4 in FIG. 4) will be described. As described above, the CPU 21 calculates the score SC corresponding to each amount of change v that constitutes the correlation change V, based on the correlation change V, in order to determine whether the performance of the exposure apparatus 1 is good or bad.

For example, the CPU 21 may calculate the score SC for each amount of change v. In this case, the CPU 21 may employ each variation v itself as the score SC. That is, the CPU 21 may employ the change amount v _ij itself as the score SC _ij corresponding to the change amount v _ij . Alternatively, the CPU 21 may calculate the score SC corresponding to each variation v by performing arbitrary arithmetic processing based on each variation v. In this case, the CPU 21 calculates the square of the total number of status items I (that is, d×d) scores SC.

Alternatively, the CPU 21 may calculate the score SC for each status item I in order to reduce the total number of scores SC. Specifically, the amount of change v _i1 to the amount of change v _{id are respectively} the correlation coefficients r _i1 to It shows the change of r _id . Therefore, the CPU 21 may calculate the score SC _i of the item I _i in a certain state based on the amount of change v _i1 to the amount of change v _id related to the variable x _i of the item I _i in a certain state. For example, the CPU 21 may use Equation 10 to calculate the score SC _i of the item I _i in a certain state.

Alternatively, the CPU 21 may use any other method to calculate the score SC corresponding to each amount of change v that constitutes the correlation change V. FIG.

Here, an example of the score SC will be described with reference to FIGS. 13(a) and 13(b). FIGS. 13A and 13B are graphs showing examples of scores SC, respectively.

FIG. 13A shows an example of the score SC for each item I when the temperature of the operation block located in the upper region of the exposure apparatus 1 increases. As shown in FIG. 13(a), the score SC of item I in the state relating to the upper region is greater than the score SC of item I in the state relating to the middle region or the lower region. Therefore, the score SC indicates that the state (here, temperature) of the action blocks located in the upper region changes in the same way.

FIG. 13B shows an example of the score SC for each item I when the temperature of the operation block located in the lower region of the exposure apparatus 1 increases. As shown in FIG. 13(b), the score SC of item I in the state related to the lower region is greater than the score SC of the item I in the state related to the upper or middle region. Therefore, the score SC indicates that the state (here, temperature) of the action blocks located in the lower region changes in the same way.

The display device 25 may display the score SC in any display form. For example, the display device 25 may display the score SC using graphs shown in FIGS. 13(a) and 13(b). For example, the display device 25 can classify a plurality of operation blocks constituting the exposure apparatus 1 according to the score SC, similar to the correlation structure R or the correlation change V, according to the difference in function or arrangement position of the plurality of operation blocks. A score SC may be displayed so as to be associated with a plurality of obtained groups. For example, the display device 25 may display the scores SC of items I for states associated with a certain function or a certain arrangement (location) in close proximity to each other. FIGS. 13(a) and 13(b) show an example of displaying the scores SC of items I in a state related to a certain arrangement position close to each other.

After calculating the score SC, the CPU 21 determines whether the performance of the exposure apparatus 1 is good or bad based on the calculated score SC. In this embodiment, the CPU 21 determines the quality of the performance of the exposure apparatus 1 based on the database (in other words, table or function) DB indicating the correspondence between the score SC and the performance of the exposure apparatus 1 and the calculated score SC. judge. For example, the CPU 21 determines whether the performance of the exposure apparatus 1 is good or bad based on the calculated score SC and a database DB indicating the correspondence between the score SC and index data indicating the performance of the exposure apparatus 1 qualitatively or quantitatively. judge. However, the CPU 21 determines whether the performance of the exposure apparatus 1 is good or bad based on the calculated score SC using another method.

Here, an example of the database DB showing the correspondence relationship between the score SC and the performance will be described with reference to FIG. 14 . FIG. 14 is an example of a database DB showing correspondence between scores SC and performance.

As shown in FIG. 14, the database DB stores a combination of the score SC ₁ of the state item I ₁ , the score SC ₂ of the state item I ₂ , . . . , and the score SC _d of the state item _Id , It is associated with index data indicating the quality of performance corresponding to the combination. For example, in the database DB, if the score SC ₁ is S#11, the score SC ₂ is S#21, and the score SC _d is S#d1, the performance of the exposure apparatus 1 is P #11 . . . 1 (eg, all performance remains good). For example, in the database DB, if the score SC ₁ is S#21, the score SC ₂ is S#22, and the score SC _d is S#d3, the performance of the exposure apparatus 1 is P# 22 . For example, in the database DB, if the score SC ₁ is S#21, the score SC ₂ is S#21, and the score SC _d is S#d1, the performance of the exposure apparatus 1 is P# 21 . Similarly, each record constituting the database DB also indicates how the performance of the exposure apparatus 1 is now (or how it has changed at present) or in the future when the score SC reaches a certain value. indicates whether it changes to

The database DB may have any data structure as long as the CPU 21 can determine the quality of the performance of the exposure apparatus 1 using at least part of the score SC calculated by the CPU 21. good.

In this embodiment, the database DB is constructed in advance by the evaluation device 2 (or by a database construction device different from the evaluation device 2, the same below) before the evaluation operation is performed. The constructed database DB is stored in the memory 22 in advance. However, the database DB may be constructed sequentially by the evaluation device 2 in parallel with the evaluation operation. The database DB may be sequentially updated by the evaluation device 2 in parallel with the evaluation operation.

The evaluation device 2 may construct (or update, the same shall apply hereinafter) the database DB based on the result of the exposure operation actually performed by the exposure device 1 . Specifically, for example, the evaluation apparatus 2 may acquire, as log data, the variable x and the performance actually measured when the exposure apparatus 1 performs a certain exposure operation. Alternatively, for example, the evaluation apparatus 2 may acquire, as log data, the variable x and the performance actually measured when another exposure apparatus of the same model as the exposure apparatus 1 performs a certain exposure operation. In this case, the evaluation device 2 may construct the database DB by calculating the score SC based on the acquired log data and associating the calculated score SC with the performance acquired as the log data. The “exposure operation” referred to here may be an exposure operation for the substrate 141 used as an actual device. Alternatively, the “exposure operation” may be an exposure operation for the substrate 141 for test exposure.

The evaluation device 2 may perform the simulation in consideration of the characteristics of the exposure device 1 described above. For example, the evaluation apparatus 2 may perform the simulation after considering the functions or arrangement (position) of the plurality of operation blocks that constitute the exposure apparatus 1 described above. For example, the evaluation device 2 may perform the simulation after considering the functions or layouts (positions) of the plurality of spaces in the exposure device 1 described above. For example, the evaluation device 2 may perform the simulation after considering the operation blocks of the exposure device 1 described above or other characteristics of the space. An example of simulation considering the characteristics of the exposure apparatus 1 will be described below.

For example, when an exposure operation is performed, movement of the mask stage 11 along the X-axis direction, movement of the mask stage 11 along the Y-axis direction, and movement of the mask stage 11 along the Z-axis direction are originally independently controlled. Therefore, the variable x of the item I indicating the position of the mask stage 11 along the X-axis direction, the variable x of the item I indicating the position of the mask stage 11 along the Y-axis direction, and the variable x of the item I indicating the position of the mask stage 11 along the Y-axis direction There should be no correlation with the variable x of the state item I, which indicates the position along the Z-axis. The evaluation device 2 may construct the database DB in consideration of such a positional relationship along each axis of the mask stage 11 . As a result, the CPU 21 performs an evaluation operation based on such a database DB, so that the variable x of the item I indicating the position of the mask stage 11 along the X-axis direction and the variable x of the mask stage 11 along the Y-axis direction of the mask stage 11 If the score SC indicates that there is a correlation between the variable x of the state item I indicating the position along the Z-axis direction of the mask stage 11 and the variable x of the state item I indicating the position along the Z-axis direction of the mask stage 11 , it can be determined that the synchronization accuracy of the movement of the mask stage 11 along each axis is not good.

For example, when the state of a specific portion of an operation block is detected by a plurality of sensors 15, the variable x of the plurality of state items I indicating the detection results of the plurality of sensors should have a positive correlation. is. The evaluation device 2 may construct the database DB in consideration of the presence of operation blocks including specific portions whose states are detected by such a plurality of sensors 15 . As a result, the CPU 21 performs an evaluation operation based on such a database DB to determine whether the correlation between the variables x of the items I of the plurality of states indicating the detection results of the plurality of sensors 15 is small or absent. When the score SC indicates, it can be determined that some abnormality has occurred in the operation block including the specific portion or that some abnormality has occurred in the plurality of sensors 15 that detect the state of the specific portion.

For example, there may be multiple motion blocks (or multiple spaces) that should be in the same state. In this case, the variables x of the multiple state item I, which respectively indicate the states of the multiple action blocks, should have a positive correlation. The evaluation device 2 may construct the database DB in consideration of the existence of such a plurality of operation blocks that should originally be in the same state. As a result, the CPU 21 performs an evaluation operation based on such a database DB to score whether or not there is a small or no correlation between the variables x of the items I of a plurality of states respectively indicating the states of a plurality of operation blocks. When SC indicates, it can be determined that some kind of abnormality has occurred in the plurality of operation blocks or that some kind of abnormality has occurred in the plurality of sensors 15 that detect the states of the plurality of operation blocks.

For example, as described above, the temperature of multiple motion blocks (or spaces) in the same or close proximity should change in the same way. That is, the variables x of the multiple state items I, which respectively indicate the temperatures of multiple operating blocks at the same or closely located locations, should have a positive correlation. The evaluation device 2 may construct the database DB in consideration of the temperature relationship of a plurality of operation blocks located at the same or close arrangement positions. As a result, the CPU 21 performs the evaluation operation based on such a database DB, thereby determining the correlation between the variables x of the plurality of state items I, which respectively indicate the temperatures of the plurality of operation blocks at the same or adjacent positions. When the score SC indicates that the is small or not, it is determined that some kind of abnormality has occurred in the plurality of operation blocks or that some kind of abnormality has occurred in the plurality of sensors 15 that detect the states of the plurality of operation blocks. can do.

For example, as described above, the temperature of multiple motion blocks (or spaces) at remote locations should vary independently of each other. That is, there should be little or no correlation between the variables x of the state items I, which respectively indicate the temperatures of the remotely located motion blocks. The evaluation device 2 may construct the database DB in consideration of the temperature relationship of a plurality of operation blocks positioned at positions separated from each other. As a result, the CPU 21 performs an evaluation operation based on such a database DB, so that the CPU 21 can relatively When the score SC indicates that there is a large correlation, it is determined that some kind of abnormality has occurred in the plurality of operation blocks or that some kind of abnormality has occurred in the plurality of sensors 15 that detect the states of the plurality of operation blocks. can do. The same thing can be said about the temperature of the operation block arranged with the heat insulating material interposed therebetween.

As described above, the evaluation device 2 can suitably determine whether the performance of the exposure device 1 is good or bad. In particular, in this embodiment, the evaluation device 2 determines whether the performance is good or bad based on the variable x itself (that is, the log data itself). The quality of the performance is judged based on the results. Therefore, the evaluation device 2 can preferably evaluate the performance that can determine the quality from the correlation change V, while it is difficult or impossible to determine the quality from the variable x itself. For example, depending on the type of performance to be evaluated, when the performance is no longer good, although the variable x itself does not show any abnormal tendency, the correlation change V related to the variable x shows some abnormal tendency. may be The evaluation device 2 of the present embodiment can suitably and quickly determine the quality of such performance (substantially predict the quality of performance).

In addition, since the evaluation device 2 determines whether the performance is good or bad based on the correlation change V of the log data, even if the value of the variable x of the item I in each state constituting the log data is different It can be determined that certain performances of the exposure apparatus 1 have similarly deteriorated. For example, when the operating state of the exposure apparatus 1 changes from the first state to the second state, the combination of the score SC calculated from the actually measured log data (variable x) and the operating state of the exposure apparatus 1 are changed to the first state. Assume that a database DB is constructed that includes a first record that associates the quality of the performance of the exposure apparatus 1 that is actually measured when the state changes to the second state. For convenience of explanation, the first record is a combination of the score SC calculated when the operating state of the exposure apparatus 1 changes from the first state to the second state and a certain performance A of the exposure apparatus 1. Assume that it is associated with index data indicating that only quantitative B has deteriorated. When the operation state of the exposure apparatus 1 changes from the first state to the second state while the evaluation apparatus 2 is performing the evaluation operation using such a database DB, the evaluation apparatus 2 naturally performs the second One record is used to determine whether the performance of the exposure apparatus 1 is good or bad. Therefore, the evaluation device 2 determines that a certain performance A of the exposure device 1 has deteriorated by a predetermined amount B. FIG. On the other hand, suppose that the operation state of the exposure apparatus 1 changes from the third state to the fourth state while the evaluation apparatus 2 is performing evaluation operations using such a database DB. In this case, log data (variable x) input when the operating state of the exposure apparatus 1 is in the first state and log data (variable x) input when the operating state of the exposure apparatus 1 is in the third state. ) shall be different. Similarly, the log data (variable x) when the operating state of the exposure apparatus 1 is in the second state and the log data (variable x) input when the operating state of the exposure apparatus 1 is in the fourth state are be different. On the other hand, the correlation change V of the log data when the operating state of the exposure apparatus 1 changes from the first state to the second state is the log data V when the operating state of the exposure apparatus 1 changes from the third state to the fourth state. It is assumed to be the same as the correlation change V of the data. In this case, if it is an evaluation apparatus of a comparative example that determines whether the performance of the exposure apparatus 1 is good or bad simply based on the log data itself, it can be determined that a certain performance A of the exposure apparatus 1 has deteriorated by a predetermined amount B. Relatively likely not. However, the evaluation apparatus 2 of the present embodiment refers to the first record to determine whether the operation state of the exposure apparatus 1 is changed from the third state to the fourth state. As in the case of changing from the first state to the second state, it can be determined that a certain performance A of the exposure apparatus 1 has deteriorated by a predetermined amount B. FIG. Therefore, the evaluation device 2 can suitably determine whether the performance of the exposure device 1 is good or bad.

Further, in the above-described embodiment, even if the number of variables x (log data) is, for example, several thousand or more, it is possible to stably and suitably determine whether the performance of the exposure apparatus 1 is good or bad. Further, in the above-described embodiment, it is possible to suitably determine whether the performance of the exposure apparatus 1 is good or bad even when the observed value (for example, the variable x) changes dynamically. In the above-described embodiment, the module structure (variable group) of the exposure apparatus 1 may be automatically recognized.

(2-4) Operation for Displaying Judgment Results Subsequently, among the evaluation operations performed by the evaluation device 2, the operation for displaying the judgment results as to whether the performance is good or bad (step S5 in FIG. 4) will be described.

The display device 25 may display the determination result as it is using characters, graphics, or the like. For example, the display device 25 may display characters, graphics, or the like indicating that "some performance A of the exposure apparatus 1 has deteriorated by a predetermined amount B".

The display device 25 may display the determination result in a display format that associates the determination result with the exposure apparatus 1 . As an example of a display form that associates the determination results with the exposure apparatus 1, the determination results are classified into a plurality of operation blocks (or a plurality of spaces, the same shall apply hereinafter) that constitute the exposure apparatus 1 according to the above-described predetermined criteria. A display form associated with a plurality of obtained groups can be mentioned. For example, the determination result indicates whether a certain type of performance of the exposure apparatus 1 is good or bad. Whether the performance of the exposure apparatus 1 is good or bad depends on the state of an operation block of the exposure apparatus 1 . Therefore, the evaluation device 2 can specify which operation block of the exposure device 1 is related to the performance determination result to be displayed. For this reason, the evaluation apparatus 2 performs determination so that the determination result to be displayed is associated with at least one operation block related to the determination result to be displayed among the plurality of operation blocks constituting the exposure apparatus 1. You can display the results. Specifically, as shown in FIG. 15(a), the display device 25 displays the determination result that "a certain performance A of the exposure apparatus 1 has deteriorated by a predetermined amount B" in the central area related to the determination result. The determination result may be displayed so as to be associated with at least one arranged action block. As a result, the user can determine that one of the causes of the determination result that "a certain performance A of the exposure apparatus 1 has deteriorated by a predetermined amount B" is at least one operation located in the central region associated with the determination result. Blocks can be easily recognized. Alternatively, as shown in FIG. 15B, the display device 25 implements the function F#2 related to the determination result that "a certain performance A of the exposure apparatus 1 has deteriorated by a predetermined amount B". The determination result may be displayed so as to be associated with at least one action block for performing. As a result, the user finds that one of the causes of the judgment result that "a certain performance A of the exposure apparatus 1 has deteriorated by a predetermined amount B" is at least one of the causes for realizing the function F#2 associated with the judgment result. It can be easily recognized as being in one operation block.

The display device 25 may change the display form and display mode of the determination result according to the degree of performance deterioration. For example, as shown in FIGS. 15(a) and 15(c), as the degree of performance deterioration increases, the display device 25 displays characters, graphics, etc. (FIGS. 15(a) and 15(c)) indicating the determination result. In c), the determination result may be displayed such that the exclamation mark) is emphasized (larger in FIGS. 15(a) and 15(c)). FIG. 15(a) shows a display example of the determination result when the degree of performance deterioration is relatively large, and FIG. 15(c) shows a display example of when the degree of performance deterioration is relatively small. 4 shows a display example of a judgment result.

(3) Modification Next, a modification of the evaluation operation by the evaluation device 2 of this embodiment will be described.

(3-1) First Modification In the first modification, the evaluation device 2 calculates the score SC corresponding to each amount of change v that constitutes the correlation change V, and then calculates at least part of the calculated score SC. is subjected to a desired weighting process. For example, the evaluation device 2 may perform a weighting process of multiplying at least part of the score SC by a predetermined weighting factor. For example, the evaluation device 2 may perform a weighting process in which calculation based on a predetermined weighting function is performed on at least part of the score SC. After that, the evaluation device 2 determines whether the performance of the exposure device 1 is good or bad based on the weighted score SC.

An example of weighting processing for the score SC will be described below with reference to FIG. 16 . FIG. 16 is a graph showing the score SC before weighting processing, weighting coefficients for weighting processing, and score SC before weighting processing.

The upper graph in FIG. 16 shows the score SC for each item I before the weighting process is performed. The CPU 21 multiplies the score SC shown in the upper part of FIG. 16 by the weighting factor w shown in the middle part of FIG. As a result, the score SC shown in the lower part of FIG. 16 is obtained.

The weighting coefficient w is prepared for each score SC (that is, for each state item I in the example shown in FIG. 16). Therefore, the CPU 21 multiplies the score SC _i corresponding to the status item I _i by the weighting factor _wi . However, FIG. 16 shows an example in which a common weighting factor w is prepared for a plurality of scores SC.

The CPU 21 may set the weighting factor w in consideration of the characteristics of the exposure apparatus 1 . For example, the evaluation device 2 may set the weighting coefficient w after considering the functions or arrangement (position) of the plurality of operation blocks that constitute the exposure device 1 described above. For example, the evaluation device 2 may set the weighting factor w after considering the functions or layouts (positions) of the plurality of spaces in the exposure device 1 described above. For example, the evaluation device 2 may set the weighting factor w after considering other characteristics of the plurality of operation blocks or the plurality of spaces of the exposure device 1 described above. An example of the operation of setting the weighting factor w considering the characteristics of the exposure apparatus 1 will be described below.

The CPU 21 may set a weighting factor w that can increase the importance of the score SC for an action block or space at a certain position. The CPU 21 may set a weighting factor w that can reduce the importance of the score SC for a motion block or space at a certain position. The CPU 21 may set a weighting factor w that can realize a state in which the importance of the score SC for the motion block or space at a certain position is higher than the importance of the score SC for the motion block or space at another position. .

For example, it is assumed that the score SC for the motion block at the second position is greater than the score SC for the motion block at the first position. Here, if the score SC for the motion block at the first position is very small, the very small score SC may be noise. For example, a very small score SC may be noise due to the estimation accuracy of the correlation structure R. Therefore, in this case, the CPU 21 may set a weighting factor w that can reduce the importance of the score SC that may be noise.

Specifically, for example, it is assumed that the temperature of the operation block located in the upper region of the exposure apparatus 1 increases. In this case, as described above, the score SC associated with the motion block located in the upper region will be greater than the score SC associated with the motion block located in the middle region or the lower region. In particular, the scores SC associated with action blocks located in the lower region are extremely small (see upper graph in FIG. 15). That is, the scores SC associated with motion blocks located in the lower region may be noise. In this case, the CPU 21 may set a weighting factor w capable of lowering the importance of the score SC associated with the action block located in the lower area, as shown in the middle graph of FIG.

Alternatively, for example, depending on the arrangement (position) of an action block, the state of that action block may not be affected by the states of other action blocks. That is, depending on the placement position of a certain action block, the variable x of the item I of the state indicating the state of the action block is independent of the variable x of the item I of other states (that is, there is no correlation). ). In this case, it can be said that the variable x of the independent state item I is a relatively important variable for determining whether or not the state of a certain operation block affects the exposure apparatus 1 . Therefore, in this case, the CPU 21 may set a weighting factor w that can increase the importance of the score SC of the item I in the independent state.

Specifically, an operation block that can be a heat source is arranged at a first position, and an operation block at a second position that is close enough to the first position to transmit heat generated by the heat source. , and there is a motion block placed between the first position and the second position, which may be a thermal insulator. In this case, the insulation prevents the heat generated by the heat source from being transferred to the operating block. Therefore, the temperature of the operation block located at the second position is not affected by the operation block (that is, heat source) located at the first position. That is, the variable x of the state item I indicating the temperature of the operation block located at the second position is obtained from the variable x of the state item I indicating the temperature of the operation block (that is, the heat source) located at the first position. being independent. Therefore, in this case, a weighting factor w may be set that can increase the importance of the score SC of the item I indicating the temperature of the operation block positioned at the second position.

The CPU 21 may set the weighting factor w in consideration of the type of performance of the exposure apparatus 1 to be evaluated by the evaluation operation. For example, as described above, depending on the type of performance of the exposure apparatus 1 to be evaluated by the evaluation operation, there may be an item I in a state that greatly contributes to the evaluation and an item I in a state that does not contribute much to the evaluation. Therefore, when evaluating a performance, the importance of the score SC of item I in conditions that contribute significantly to the evaluation is higher than the importance of the score SC of item I in conditions that contribute less to the evaluation. Therefore, the CPU 21 may perform a weighting process based on the type of performance to be evaluated so that the contribution rate of the score SC of the item I that greatly contributes to the evaluation to the evaluation operation is increased. The CPU 21 may perform weighting processing based on the type of performance to be evaluated so that the contribution rate of the score SC of the item I that does not contribute much to the evaluation to the evaluation operation decreases or becomes zero.

Also in the first modified example, the evaluation device 2 can suitably determine whether the performance of the exposure device 1 is good or bad. In particular, in the first modification, the quality of the performance is determined based on the weighted score SC, so the evaluation apparatus 2 can more preferably determine the quality of the performance of the exposure apparatus 1 .

(3-2) Second Modification Next, a second modification of the evaluation operation will be described with reference to FIG. FIG. 17 is a matrix showing an example of log data in the second modified example.

In the above-described embodiment, the log data is composed of the variable x of the item I of a plurality of states, each of which indicates the operating state of the exposure apparatus 1 during a certain period. On the other hand, in the second modified example, in addition to the variable x of the item I of a plurality of states, each of which indicates the operating state of the exposure apparatus 1 during a certain period, the log data each includes a period earlier than the certain period. It includes a variable x of a plurality of status items I indicating the operating status of the exposure apparatus 1 in the table. For example, as shown in FIG. 17, the log data is data indicating variable x of item I of a plurality of states each indicating the operation state of exposure apparatus 1 during the m-th period (where m is an integer equal to or greater than 1). In addition to the portion Lm, it includes a data portion _Lm-1 each indicating the variable x of a plurality of state items I indicating the operating state of the exposure apparatus 1 during the m-1th period prior to the _mth period. . Therefore, the log data becomes a matrix of N rows×2d columns. Using such log data, the CPU 21 estimates the correlation structure R, detects the correlation change V, and determines whether the performance of the exposure apparatus 1 is good or bad.

Also in the second modification, the evaluation device 2 can suitably determine whether the performance of the exposure device 1 is good or bad. In particular, in the second variation, since the log data includes variables x of items I in multiple states observed during multiple periods, the correlation change V is also observed during multiple periods. Fig. 3 shows the change in correlation of variable x for item I in multiple states. For example, the correlation change V is the correlation structure between the variable x of the multiple state item I observed during the m period and the variable x of the multiple state item I observed during the m-1 period. In addition to the change V of R, between the variable x of the multiple state item I observed during the m−1 period and the variable x of the multiple state item I observed during the m−2 period can show the variation V of the correlation structure R of Therefore, the evaluation apparatus 2 can more preferably determine whether the performance of the exposure apparatus 1 is good or bad based on the latest correlation change V and the past correlation change V. FIG.

It should be noted that in the description of the second modified example above, the log data includes the variable x indicating the operating state of the exposure apparatus 1 during the two periods. However, the log data may include the variable x of multiple status items I that indicate the operating status of the exposure apparatus 1 during more than two periods. For example, in the log data, in addition to the data portion Lm indicating the variable x of the item I of a plurality of states, each of which indicates the state of the exposure apparatus 1 during the _m -th period, each of the m-th a data portion L _m-1 indicating the variable x of a plurality of state items I indicating the operation state of the exposure apparatus 1 during the -1 period; A data portion L _m-2 may be included which indicates the variable x of a plurality of status items I indicating the operating status of the exposure apparatus 1 during the -2 period.

In the above description, the exposure apparatus 1 exposes the substrate 141 such as a semiconductor substrate. However, the exposure apparatus 1 may expose any object such as a glass plate, a ceramic substrate, a film member, or mask blanks. The exposure apparatus 1 may be an exposure apparatus for manufacturing liquid crystal display elements or displays. The exposure apparatus 1 may be an exposure apparatus for manufacturing at least one of a thin film magnetic head, an imaging device (for example, CCD), a micromachine, MEMS, a DNA chip, and a mask 111 (or reticle). The exposure device 1 may be an optical tweezers device that captures an object using a light trapping force generated in the object by irradiating the object with the exposure light EL.

A device such as a semiconductor device may be manufactured through each step shown in FIG. The steps for manufacturing the device include a step S201 of designing the function and performance of the device, a step S202 of manufacturing the mask 111 based on the function and performance design, a step S203 of manufacturing the substrate 141 which is the base material of the device, and a mask A step S204 of exposing the substrate 141 with the exposure light EL from the device pattern 111 and developing the exposed substrate 141, a step S205 including device assembly processing (processing processing such as dicing processing, bonding processing, and packaging processing), and inspection. It may include step S206.

The requirements of each of the above embodiments can be combined as appropriate. Some of the requirements of each embodiment described above may not be used. The requirements of each embodiment described above can be replaced with the requirements of other embodiments as appropriate. In addition, to the extent permitted by law, the disclosures of all publications and US patents relating to the exposure apparatus and the like cited in each of the above-described embodiments are incorporated herein by reference.

In addition, the present invention can be modified as appropriate within the range not contrary to the gist or idea of the invention that can be read from the scope of claims and the entire specification. , an exposure apparatus and exposure method, an exposure system, a device manufacturing apparatus, and a computer program are also included in the technical concept of the present invention.

Reference Signs List 1 exposure device 11 mask stage 111 mask 112 drive system 12 illumination system 13 projection optical system 14 substrate stage 141 substrate 142 drive system 15 sensor 16 control device 2 evaluation device 21 CPU
22 memory 23 input unit 24 operation device 25 display device SYS exposure system

Claims (13)

an input unit into which respective variables of a plurality of items each indicating an operating state of the exposure apparatus are input;
A sparse structure learning method is used to determine the correlation between each variable of the plurality of items, and a plurality of nodes corresponding to the plurality of items, respectively, and the correlation between the two variables of the two items. a display unit that displays a display object including the obtained correlation using an edge that associates the two nodes that correspond to the two items in a display form;
with
Each of the plurality of items relates to at least one of a plurality of groups obtained by classifying the plurality of parts constituting the exposure apparatus according to predetermined criteria based on differences in arrangement of the plurality of parts. and
The display unit displays at least two of the nodes respectively corresponding to at least two items related to one group corresponding to at least one portion arranged at the same or close position among the plurality of portions. , a display device that collectively displays as a group of display objects. The exposure apparatus includes a plurality of sensors,
one of the plurality of sensors is capable of detecting the operating state of at least one portion corresponding to the one sensor of the plurality of portions;
the detection results of each of the plurality of sensors are input to the input unit as variables of each of the plurality of items;
2. The display device according to claim 1 , wherein the plurality of nodes respectively correspond to the plurality of sensors . The exposure device includes a first portion of the plurality of portions in an upper region within the exposure device,
The exposure device includes a second portion different from the first portion among the plurality of portions in a middle region within the exposure device located below the upper region,
The exposure device includes a third portion different from the first and second portions among the plurality of portions in a lower region within the exposure device located below the central region,
The display unit collectively displays at least two of the nodes respectively corresponding to at least two items related to a first group corresponding to the first part as a group of display objects in a first display area;
The display unit positions at least two nodes respectively corresponding to at least two items related to a second group corresponding to the second part as a group of display objects below the first display area. collectively displayed in the second display area,
The display unit positions at least two nodes respectively corresponding to at least two items related to a third group corresponding to the third part as a group of display objects below the second display area. Display together in the third display area
The display device according to claim 1 or 2. The first part includes at least one of a mask, a mask stage that holds the mask, a drive system that moves the mask stage, and an illumination system that emits exposure light toward the mask,
the second part includes a projection optical system that projects the exposure light that has passed through the mask onto a substrate;
The third part includes at least one of the substrate, a substrate stage that holds the substrate, and a drive system that moves the substrate stage.
The display device according to claim 3. The display device according to any one of Claims 1 to 4 , wherein the display unit displays the display object in a display mode in which the display object is associated with the exposure device. The display device according to any one of claims 1 to 5 , wherein the plurality of groups includes a plurality of first groups obtained by classifying the plurality of portions based on differences in functions of the plurality of portions. . The display device according to any one of claims 1 to 6 , wherein the plurality of groups includes a plurality of second groups obtained by classifying the plurality of portions based on differences in position of the plurality of portions. . inputting variables for each of a plurality of items, each of which indicates an operating state of the exposure apparatus;
A sparse structure learning method is used to determine the correlation between each variable of the plurality of items, and a plurality of nodes corresponding to the plurality of items, respectively, and the correlation between the two variables of the two items. displaying a display object containing the determined correlation using an edge that associates two of the nodes that correspond to the two items in a display form ;
Each of the plurality of items relates to at least one of a plurality of groups obtained by classifying the plurality of parts constituting the exposure apparatus according to predetermined criteria based on differences in arrangement of the plurality of parts. and
At least two of the nodes each corresponding to at least two items related to one group corresponding to at least one portion arranged at the same or close position among the plurality of portions. are collectively displayed as a group of display objects . An exposure apparatus for exposing an object,
An exposure apparatus comprising the display device according to claim 1 . An exposure method for exposing an object,
An exposure method comprising the display method according to claim 8 . An exposure system comprising the display device according to claim 1 and the exposure device. exposing the object using the exposure method according to claim 10 , transferring a pattern to the object,
developing the exposed object to form an exposed pattern layer corresponding to the pattern;
A device manufacturing method for processing the object through the exposure pattern layer. A computer program that causes a computer to execute the display method according to claim 8 .

Images