JP2016142960A - Imaging system and imaging method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging system and imaging method that can make a size of a small segment large, and can make a blur of an image in each small segment small.SOLUTION: An imaging system includes: an optical system that forms an image of a subject; imaging means that images an imaging range of the subject for each of a plurality of small segments via the optical system; measurement means that measures a height of the subject in an optical axis direction of the optical system before the subject is imaged by the imaging means; processing means that calculates a residual error based on the height of the subject and an approximate height in a plane having a height distribution of the subject approximated to thereby acquire an imaging condition for each small segment; and adjustment means that adjusts a relative position relation between the small segment and the imaging means on the basis of the imaging condition.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an imaging system and an imaging method for capturing a sample and acquiring digital image data.

  In recent years, in the field of pathological diagnosis, an imaging system that captures a microscope image of a specimen (such as a tissue piece of a human body) and acquires, accumulates, and browses a high-definition digital image has attracted attention. This type of imaging system is also called a virtual slide system, and is required to image a wide area. For example, when a 10 mm × 10 mm observation area on a slide is imaged at an optical magnification of 10 times, the size of the imaging area on the imaging surface is 100 mm × 100 mm, that is, about 141 mm in diameter. A large image sensor (image sensor) capable of capturing an image of such an area at a time is costly due to the influence of a decrease in yield due to a pixel defect or the like. For this reason, an imaging region of a subject is divided into a plurality of small sections, images are captured in units of small sections, and the obtained partial images are connected to generate an entire image.

  Here, the subject means a general object to be imaged such as a specimen itself or a preparation. In the preparation, a specimen (tissue piece or the like) to be observed is placed on a slide glass, and this is sandwiched and fixed by a transparent protective member (cover glass). In order to reduce the number of times of imaging and shorten the imaging time, it is desirable to increase the size of the small sections and reduce the number of small sections. In addition, high resolution is required for imaging a subject, and it is necessary to increase the numerical aperture (NA) on the object side of the imaging optical system. However, increasing the numerical aperture on the object side decreases the depth of field. If the surface of the subject has height unevenness, there may be a place with a height exceeding the depth of field in the small section, and the image may be blurred.

  As a technique enabling observation of a wide field of view, Patent Document 1 proposes a system that generates a high-resolution and wide-angle image by combining a plurality of images with different imaging sections. In this system, when an operator first specifies an imaging range on a low-resolution whole image, the arithmetic unit divides the imaging range into small sections and captures a high-magnification image with a microscope for each small section. At this time, an image is taken at the in-focus position using the height information of the subject. The size of the small section is set small so that a portion exceeding the depth of field does not occur in the small section.

  As a technique for increasing the size of the small section, Patent Document 2 discloses a technique for imaging in accordance with the unevenness of the subject. The imaging system includes a first imaging optical system (imaging optical system), a second imaging optical system (re-imaging optical system), and reflecting means disposed on the optical path between them. . By changing at least one of the position of the reflecting means in the optical axis direction and the inclination with respect to the optical axis, imaging is performed in accordance with the unevenness of the subject. In this technique, when the number of small sections increases, the number of second imaging optical systems increases. In order to reduce the number of second imaging optical systems, it is desirable that the number of small sections is small. For this purpose, it is desirable that the size of the small sections is larger.

JP 2007-108223 A Patent No. 5220172

  In Patent Document 1, a boundary used when an imaging region is divided into a plurality of small sections is determined using a position where a subject exists. However, the relative position of the boundary of the small section with respect to the subject is not optimized with respect to the height distribution in the small section. The imaging surface is translated in the direction of the optical axis in order to align the in-focus position with the unevenness of the image surface. However, the imaging surface cannot be tilted relative to the subject in accordance with the unevenness of the subject. For this reason, it is difficult to reduce the change in the focus position of the image due to the unevenness of the subject, and the size of the small section cannot be increased.

  In Patent Literature 2, the unevenness of the subject is prevented from exceeding the depth of field by adjusting the position and inclination of the reflecting means. However, if the amount of unevenness of the subject is large, it is difficult to increase the size of the small section. Also, depending on the boundary divided into small sections, the unevenness of the subject may exceed the depth of field. In order to cope with such a case, it is necessary to optimize the size of the small section, the boundary position of the small section with respect to the subject, and the position adjustment amount of the reflecting means. However, there is no specific disclosure of a method for optimizing the size of a small section or the boundary to be divided into small sections and determining the position and inclination of the reflecting means at the optimized boundary.

  In view of such a problem, an object of the present invention is to provide an imaging system and an imaging method capable of increasing the size of a small section and reducing blurring of an image in each small section.

  An imaging system according to an aspect of the present invention includes an optical system that forms an image of a subject, an imaging unit that captures an imaging range of the subject for each of a plurality of small sections via the optical system, and the imaging unit. Before imaging the subject, a measuring means for measuring the height of the subject in the optical axis direction of the optical system, and a residual based on the height of the subject and an approximate height on a plane approximating the height distribution of the subject. Processing means for obtaining an imaging condition for each of the small sections by calculating a difference; and adjusting means for adjusting a relative positional relationship between the small sections and the imaging means based on the imaging conditions; It is characterized by having.

  According to another aspect of the present invention, an imaging method includes: an optical system that forms an image of a subject; and an imaging unit that captures an imaging range of the subject for each of a plurality of small sections via the optical system. An imaging method for an imaging system, comprising: measuring a height of the subject in the optical axis direction of the optical system; and an approximate height in a plane that approximates the height of the subject and the height distribution of the subject A step of obtaining an imaging condition for each of the small sections by calculating a residual, a step of adjusting a relative positional relationship between the small section and the imaging unit based on the imaging conditions, and the imaging And means for imaging the subsection.

  ADVANTAGE OF THE INVENTION According to this invention, while increasing the size of a small division, the imaging system and imaging method which can make the blur of the image in each small division small can be provided.

It is a principal part schematic of the imaging system which concerns on embodiment of this invention. It is a principal part schematic diagram of the imaging unit which has a reflection member. It is explanatory drawing of a plane fitting. It is a flowchart of an imaging method. It is a figure which shows the setting of an imaging condition. It is a figure which shows the setting method of an imaging condition. 6 is a flowchart of imaging method setting methods 1 and 2; 10 is a flowchart of an imaging condition setting method 3; 10 is a flowchart of an imaging condition setting method 4; It is a figure which shows the height obtained by the measurement of Example 1. FIG. It is a figure which shows having changed the boundary of Example 1. FIG. FIG. 10 is a diagram illustrating a position of a residual exceeding the depth of field according to the second embodiment. It is a figure which shows the value of the residual exceeding the depth of field of Example 2. It is a figure which shows the position of the residual exceeding the depth of field of Example 2, and the boundary which images the residual. It is a figure which shows the value of the residual in the new boundary of Example 2. FIG. It is a figure which shows the height obtained by the measurement of Example 2. FIG. FIG. 10 is a diagram illustrating a position of a residual exceeding the depth of field according to the third embodiment. It is a figure which shows the value of the residual exceeding the depth of field of Example 3. It is a figure which shows the position of the residual exceeding the depth of field of Example 3, and the boundary which images the residual. It is a figure which shows the value of the residual in the new boundary of Example 3. FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the same members are denoted by the same reference numerals, and redundant description is omitted.
(System configuration)
FIG. 1 is a main part schematic diagram of an imaging system according to an embodiment of the present invention. The direction parallel to the optical axis is the z direction, the direction perpendicular to the paper surface is the x direction, and the direction perpendicular to the optical axis and parallel to the paper surface is the y direction. The imaging system includes a measurement imaging device 1, an arithmetic device (processing means) 2, and a display device 3.

  The measurement and imaging apparatus 1 includes a measurement unit (measurement unit) 100, an imaging unit (imaging unit) 300, a conveyance unit 203 that conveys a preparation 103 that is a subject, and a control unit 401, and images a sample on the preparation 103. To generate a digital image. The measurement unit 100 performs surface shape measurement. The surface shape measurement refers to measuring the sample position on the preparation 103 and the height (unevenness) of the subject. The surface shape measurement is performed at a magnification lower than the imaging magnification imaged by the imaging unit 300 before imaging the subject. The measurement unit 100 may have a function of processing measurement data as well as surface shape measurement. For example, it has a function of calculating a spatial frequency of the surface shape with respect to the result of the surface shape measurement and performing a process of removing a component whose spatial frequency exceeds a certain value. With this process, noise having a high spatial frequency due to stage errors or measurement errors can be removed. The result of the surface shape measurement is used when setting the imaging conditions of the imaging unit 300. The imaging unit 300 divides the imaging area of the subject into a plurality of small sections, and performs imaging in units of small sections. Here, consider a case where an image is picked up by a different image sensor for each small section. In the present embodiment, the result of surface shape measurement by the measurement unit 100 is used when determining the position of the boundary that divides the subject into a plurality of small sections. That is, the boundary position (x, y) of the small section with respect to the subject, which is one of the imaging conditions, is set using the height z (x, y) of the sample or cover glass obtained by the surface shape measurement. The imaging conditions are not limited to the boundary positions of the small sections, but include conditions for the position and inclination of the imaging surface described later, and are set so that the entire area of the subject can be accommodated in the depth of field with a smaller number of imaging times. The imaging condition is calculated using the arithmetic device 2. The transport unit 203 transports the preparation 103 whose surface shape measurement has been completed from the measurement unit 100 to the imaging unit 300. The control unit 401 controls the position of the subject of the imaging apparatus and the position of the imaging surface so as to satisfy imaging conditions such as the boundary positions of the small sections.

  The arithmetic device 2 acquires imaging conditions. A function of applying predetermined image processing (development, color conversion, gamma correction, noise removal, compression, etc.) to the image data obtained by using the measurement imaging device 1, a function of stitching the images of the small sections, and It has functions to manage and view images. The arithmetic device 2 may be a general-purpose computer including a CPU, a memory, a storage device, an input device (input means), and the like, or may have a dedicated image processing circuit. The storage device of the arithmetic device 2 stores a program for acquiring shooting conditions. Note that the function of the arithmetic device 2 may be incorporated in the measurement imaging device 1.

  The display device 3 is a display used for browsing images and calculation results. Note that the display device 3 may be integrated with the measurement imaging device 1 and the calculation device 2.

  The measurement unit 100 includes a measurement illumination unit 101, a measurement stage 102, a measurement optical system 104, and a measurement unit 105. The measurement illumination unit 101 has an illumination optical system that guides light from a light source to a preparation 103 installed on the measurement stage 102. The measurement stage 102 holds the preparation 103 and adjusts the position of the preparation 103 with respect to the measurement optical system 104. The measurement optical system 104 is an optical system that guides the light transmitted through the preparation 103 to the measurement unit 105. Since the measurement unit 100 is intended to measure the entire preparation, the measurement optical system 104 may have a low magnification. The measurement unit 105 is a sensor, and measures physical quantities at a plurality of points on the subject based on the light from the measurement optical system 104. For the purpose of measuring the unevenness (surface shape) of the specimen or the cover glass, various distance sensors using reflected light or interference light can be used. In FIG. 1, a transmission type measurement unit is illustrated, but a reflection type measurement unit that irradiates light from the cover glass side and measures reflected light from the subject is used according to the physical quantity to be measured and the type of sensor. May be. A plurality of units or sensors may be used in combination. Measurement data at each measurement point is transmitted to the control unit 401.

  The transport unit 203 transports the preparation 103 whose surface shape measurement has been completed from the measurement unit 100 to the imaging unit 300. The preparation 103 conveyed to the imaging unit 300 is shown as a preparation 303 disposed on the imaging stage 302 of the imaging unit 300. For example, the transport unit 203 may move the measurement stage 102 itself to function as the imaging stage 302. Alternatively, the preparation 103 may be held or sucked by the hand device and moved onto the stage. Further, when imaging a plurality of preparations 103, different preparations may be installed on the measurement stage 102 and the imaging stage 302, respectively. At this time, the surface shape measurement process and the imaging process may be performed in parallel. In addition, when imaging a plurality of slides continuously, the slide 103 accommodated in a stocker (not shown) is transported one by one from the measurement stage 102 to the imaging stage 302 by the transport unit 203. desirable.

  The imaging unit 300 includes an imaging unit 305 including an imaging illumination unit 301, an imaging stage 302, an imaging optical system 304, and an imaging element such as a CCD or a CMOS. The imaging illumination unit 301 has an illumination optical system that guides light from a light source to a preparation 303 installed on a stage 302. As the light source, for example, a halogen lamp, a xenon lamp, or an LED (Light Emitting Diode) can be used. The imaging optical system 304 enlarges the subject image and forms it on the imaging unit 305. The imaging unit 305 includes a plurality of imaging elements. In FIG. 1, three image sensors are provided, but the number of image sensors is not limited to this. The light received by the image sensor is photoelectrically converted to generate and output image data. This image data is transmitted to the arithmetic device 2 via the control unit 401. The size, number, and arrangement of the image sensor are arbitrary. In order to capture N small sections, it is ideal to perform collective imaging using N imaging elements. However, N imaging may be performed with one imaging element, or a plurality (m). You may image N / m times with this image sensor. The imaging time can be shortened by using a plurality of imaging elements.

  The control unit 401 functions as a driving mechanism (adjusting means) together with each driving unit. Specifically, the control unit 401 drives each driving unit based on the imaging information acquired by the arithmetic device 2, so that the preparation 303 and the imaging surfaces of the plurality of imaging elements constituting the imaging unit 305 are relative to each other. Adjust the relative position. The relative positional relationship means the position of the preparation 303 with respect to the imaging surface on the xy plane, the distance between the preparation 303 and the imaging surface in the z direction, and the tilt angle of the preparation 303 with respect to the z-axis with respect to the imaging surface. The position of the preparation 303 with respect to the imaging surface on the xy plane is determined by the condition of the boundary position for dividing the subject into small sections. The distance between the preparation 303 and the imaging surface in the z direction and the inclination angle of the preparation 303 with respect to the imaging surface with respect to the z axis are set such that the imaging data in the small section is within the depth of field.

  The driving unit 201 drives the imaging stage 302 based on the signal transmitted from the control unit 401 and adjusts the relative positions of the preparation 303 and the imaging unit 305. The drive unit 201 translates the imaging stage 302 on the xy plane and finely adjusts the position of the preparation 303. The drive unit 201 may rotate the imaging stage 302 in the xy plane around the z axis, translate in the z direction, or rotate around the x axis or the y axis.

The driving unit 202 drives the imaging unit 305 based on the signal transmitted from the control unit 401. The driving unit 202 is provided independently for each of the plurality of imaging elements constituting the imaging unit 305. The driving unit 202 performs rotational driving around the x axis. In the present embodiment, a drive unit (not shown) is disposed in the xz plane. The drive unit in the xz plane performs rotational driving around the y axis. By doing so, the inclination of the image sensor with respect to the z-axis changes. That is, in the present embodiment, the position and inclination can be adjusted for each of a plurality of image sensors that constitute the imaging unit 305. The drive unit 202 changes the position of the image sensor in the z direction.
(Configuration of imaging unit having a reflecting member)
An example of a configuration in which the imaging unit 300 includes a plurality of imaging units will be described with reference to FIG. In FIG. 2, two imaging units 505 and 508 are provided as an example. The imaging unit is divided into a first imaging optical system 502 and second imaging optical systems 504 and 507. The x, y, and z axes in the figure are local coordinates set separately based on the optical axis direction of the first imaging optical system 502 and the optical axis directions of the second imaging optical systems 504 and 507. To do. Reflecting members 503 and 506 are arranged at the image position of the first imaging optical system 502. The reflecting member 503 is between the first imaging optical system 502 and the second imaging optical system 504, and the reflecting member 506 is between the first imaging optical system 502 and the second imaging optical system 507. Has been placed. The positions corresponding to the object points of the second imaging optical systems 504 and 507 correspond to the image points of the first imaging optical system 502. The positions corresponding to the object points of the second imaging optical systems 504 and 507 correspond to different positions in the image of the first imaging optical system 502. Since the image of the first imaging optical system 502 is an image of the subject on the imaging stage 501, the plurality of second imaging optical systems 504 and 507 re-image each different area of the subject. As described above, by using the plurality of second imaging optical systems, the imaging region of the second imaging optical system can be divided into a plurality of small sections and collectively imaged. In the present embodiment, the imaging stage 501 is driven by the drive unit 509 based on the surface shape measurement result, and the subject on the imaging stage 501 is translated in the xy plane. In this way, the boundary position of the small section with respect to the subject is optimized. In addition, the tilts of the reflecting members 503 and 506 and the position with respect to the optical axis (z axis) are changed by the driving units 512 and 513. In this way, by changing the position of the boundary, the inclination of the reflecting members 503 and 506, and the position with respect to the optical axis, each small section is imaged. In the case of an imaging unit having a reflecting member, the relative position and direction between the subject and the imaging unit can be changed using the reflecting member in this way. Further, not only the reflecting member but also the driving units 510 and 511 may be used to change the positions of the imaging units 505 and 508 to perform imaging of each small section. In changing the position of the imaging unit, the position in the z direction and the position in the imaging plane are mainly changed. Note that at least one of the first and second imaging optical systems may have a zooming function.
(Flat fitting)
The plane fitting will be described with reference to FIG. The plane fitting is calculated by the arithmetic unit 2 when setting the position and inclination of the imaging surface with respect to the optical axis.

  As shown in FIG. 3A, the area of the subject is divided into small sections having a designated size. Each square indicated by a solid black line represents a small section. Gray shading represents the height of the subject acquired by the surface shape measurement, and the lighter the color, the higher the height of the subject. The plane fitting is performed on the height data of the subject in the small section acquired by the surface shape measurement. Heights belonging to different subsections are fitted with different planes. Heights belonging to the same small section are fitted on the same plane. For the plane fitting, a general method such as a least square method can be used. Optimize the boundary position of the small section relative to the subject so that the residual (fitting residual) between the data of the subject in the small section obtained by surface shape measurement and the data after plane fitting is within the depth of field. . For example, FIG. 3B is a diagram in which a small section indicated by R1 in FIG. The data before plane fitting is Z (X, Y), the data after plane fitting is P (X, Y), the residual is E, the slope of the plane in the X direction is G, the slope of the plane in the Y direction is H, the plane When the position in the Z direction is C, the following equations (1) and (2) hold. The values of G, H, and C correspond to the imaging conditions, and the inclination and position of the imaging surface are set so as to correspond to the values of G, H, and C.

P (X, Y) = G.X + H.Y + C (1)
E (X, Y) = | P (X, Y) −Z (X, Y) | (2)
Further, the boundary position of each plane in the plane fitting is the same as the boundary position of the small section, and this also corresponds to the imaging condition. The position of the imaging stage 302 is set so as to correspond to the boundary position. Note that the evaluation of the residual E is not limited to the method using the above equation, and other methods may be used. For example, when the number of residual E (X, Y) data in the small section is as large as Nd, the number of residual data to be evaluated may be reduced to Md for evaluation. For example, the evaluation data is reduced to Md by evaluating R data (Nd> R) collectively for Nd data in one small section. In this case, Md = Nd / R. When Nd / R becomes a decimal, the decimal part is rounded down to an integer. Using the R pieces of data, the root mean square RMS is obtained by Equation (3).

The root mean square RMS is obtained as Md pieces in one small section. Then, the residual of one small section may be evaluated using Md pieces of data.
(Image acquisition procedure)
The imaging method of this embodiment will be described using the flowchart of FIG. In step s1, the height of the subject is measured by surface shape measurement, and height data is acquired. The surface shape measurement is performed using the measurement unit 100. In step s2, an imaging condition is set when the subject imaging range is divided into a plurality of small sections. The imaging conditions include the boundary positions of the small sections, the position of the imaging surface, and the inclination. In addition to this, when the imaging optical system has a zooming function, it may include the size of a small section. The imaging condition is calculated by the arithmetic device 2. In step s3, based on the imaging condition set in step s2, the imaging stage, the imaging surface, and the reflecting member are moved using the drive mechanism, and a partial image of each small section is captured. In step s4, the partial image data captured in step s3 is captured. The arithmetic device 2 executes application of predetermined image processing, stitching of images of small sections, image management / viewing, and the like. The display device 3 is used to display images, analysis results, and the like. Further, the image data is stored in an internal or external storage device.
(Overview of setting imaging conditions)
The imaging condition in step s2 in FIG. 4 is set so that the entire area of the subject can be accommodated in the depth of field with a smaller number of imaging. First, the computing device 2 calculates whether there is a condition for capturing an image so that the entire area of the subject falls within the depth of field with one imaging. If this condition does not exist, it is calculated whether there is a condition for imaging so that the entire area of the subject is within the depth of field when the number of times of imaging is two. Specifically, it is calculated whether or not there is a condition for imaging so that all the areas that did not fit in the depth of field in the first imaging can fall within the depth of field. Since the imaging data is stored in the storage device for the area that has fallen into the depth of field in the first imaging, it is not always necessary to perform imaging in the second time. If there is no condition for capturing the entire area of the subject within the depth of field even in the case of the two imaging, the number of times is increased in the same manner, and the condition for capturing the entire area of the subject within the depth of field is the same. Calculate if there is. If there is no condition to capture the entire area of the subject within the depth of field even if the predetermined number of times of imaging is reached, the area exceeding the depth is displayed. With respect to the displayed area, it is possible to capture a plurality of images while changing the z-direction position of either the imaging stage or the imaging surface, and obtain an image that falls within the depth of field. However, this method increases the imaging time.

  The variables used in the calculation in setting the imaging conditions differ depending on the presence or absence of an imaging stage, imaging surface, drive mechanism for driving the reflecting member, and the presence or absence of a magnification function of the imaging optical system. In the present embodiment, the boundary position of the small section with respect to the subject is set using the height of the sample or the cover glass obtained by the surface shape measurement. For this reason, in this embodiment, the imaging stage always has a mechanism for moving in the xy plane. Regarding the imaging surface, it is desirable to have a mechanism capable of moving in the optical axis z direction and rotating with respect to the z axis. However, for example, there may be a case where there is no mechanism for rotating the imaging surface with respect to the z axis. In this case, the plane fitting may be performed by setting only the shift component C as a variable and fixing the tilt components G and H to zero.

  Setting of imaging conditions will be described with reference to FIG. In step s201, the arithmetic unit 2 calculates the plane fitting using the boundaries of the small sections, the z shift, and the z tilt as variables for all the small sections. The z shift corresponds to the position C in the Z direction on the plane, and the z tilt corresponds to (tilt G in the X direction on the plane, and tilt H in the Y direction on the plane). In step s202, it is determined whether all the residuals of the plane fitting are within the depth of field. If it does not fit, the imaging conditions are set and the process ends. If not, the process proceeds to step s203. In step s203, the arithmetic unit 2 performs plane fitting on the region exceeding the depth. Details will be described later. In step s204, the arithmetic unit 2 determines whether all the residuals of the plane fitting are within the depth of field. If it does not fit, the imaging conditions are set and the process ends. If not, the process proceeds to step s205. In step s205, an area exceeding the depth is displayed and the process ends.

The setting of the imaging conditions has been described for the case where the imaging optical system does not have a scaling function. However, in the present invention, there is no influence even if the imaging optical system has no scaling function. In the case of having a scaling function, the size of the small section is added to the variable in the calculation of plane fitting. By increasing the imaging magnification (magnification magnification) of the imaging optical system, the size of the small section into which the subject is divided is reduced. Here, the size of the imaging surface is assumed to be constant. Since the area of the subject included in the small section is small, the change in the height of the subject in the small section is small. Since the change in height, which is a target for plane fitting, is small, the residual of the plane fitting is also small, and it is easy to obtain an imaging condition that falls within the depth of field. Each calculation step is the same as that shown in FIG. 5 except that the size of the small section is increased.
(Setting imaging conditions)
In setting the imaging conditions, first, the first plane fitting is performed on the entire imaging area (all small sections), and then the second plane fitting is performed on the area where the residual does not fall within the depth of field. There are two cases of the second plane fitting. The first is a case where the z-shift and the z-tilt are changed only in a region where the residual does not fall within the depth of field without changing the boundary position of the first plane fitting in the second plane fitting. The second is a case where, in the second plane fitting, after changing the boundary position of the first plane fitting, the z shift and the z tilt of the region where the residual does not fall within the depth of field are obtained again. . These correspond to the case where the boundary position of the small section with respect to the subject is changed and the case where it is not changed.

  The case where the residual of the entire imaging region is within the depth of field in the first plane fitting is called imaging method setting method 0. The case where the z-shift and the z-tilt are changed without changing the boundary position in the second plane fitting performed for the region where the residual exceeds the depth of field is called imaging condition setting method 1. On the other hand, the case where the boundary position is changed and the z shift and the z tilt are changed in the second plane fitting performed on the region where the residual exceeds the depth of field is called imaging condition setting method 2. When calculating the imaging condition in the arithmetic device 2, first, the imaging condition setting method 0 is calculated. If the residual of the entire imaging area does not fit within the depth of field, for example, the imaging condition setting methods 1 and 2 are calculated in order, and the residual of the entire imaging area falls within the depth of field. If so, the calculation may be terminated. The imaging condition setting methods 0 to 2 correspond to the case where the imaging optical system does not have a zoom function.

  When the imaging optical system has a zoom function, the size of the small section can be changed. A method of imaging the entire imaging region by increasing the magnification of the imaging optical system and reducing the size of the small section is called imaging condition setting method 3. Also, a method of imaging a plurality of times by reducing the size of the small section until there is no region where the residual exceeds the depth of field is called imaging condition setting method 4.

The imaging condition setting methods 0 to 4 will be described with reference to FIG.
(Imaging condition setting method 0)
The imaging condition setting method 0 will be described with reference to FIG. A method for capturing an image of a subject shown in gray divided into nine small sections will be described. In the setting method 0, the residual of the entire imaging region is within the depth of field by optimizing the relative position (x, y) of the boundary with respect to the subject. In this case, for example, in the configuration of FIG. 2, nine second imaging optical systems can be used, and nine small sections can be imaged separately and simultaneously. That is, the number of times of imaging is one.
(Imaging condition setting method 1)
The imaging condition setting method 1 will be described with reference to FIGS. This method is used when the residual of the entire area of the subject in the entire imaging area does not fall within the depth of field in the first imaging. That is, it is used when the residual of one or more height data exceeds the depth of field even if the boundary B1 for the subject is optimized.

First, as shown in FIG. 6B, a boundary B1 in which the number of residuals exceeding the depth of field is minimized is determined, and a region N1 indicated by black where the residual exceeds the depth of field at the boundary B1. , N2 is specified. Then, plane fitting is performed again on the areas N1 and N2 and the vicinity thereof so that the areas N1 and N2 are within the depth of field. The size of the small section at this time is the same size as the small section of the boundary B1. Further, only the inclination and position of the plane are used as fitting variables without changing the boundary B1 of the small section. That is, the boundary B1 remains unchanged, and the small section including the regions N1 and N2 is subjected to the plane fitting again (FIG. 6C). In addition, since areas other than the areas N1 and N2 are not objects of plane fitting, the depth of field may be exceeded.
(Imaging condition setting method 2)
The environmental condition setting method 2 will be described with reference to FIGS. This method differs from the imaging condition setting method 1 in that the boundary is also changed as a fitting coefficient when the plane fitting is performed again.

  First, the boundary B1 that minimizes the number of residuals exceeding the depth of field is determined (FIG. 6D). Then, the plane fitting is performed again so that the regions Er1 and Er2 fall within the depth of field with respect to the regions Er1 and Er2 where the residual exceeds the depth of field at the boundary B1. The size of the small section at this time is the same size as the small section of the boundary B1, but the boundary B1 of the small section is changed. That is, using the boundary as a variable for plane fitting, the small section including the regions Er1 and Er2 is subjected to plane fitting again (FIG. 6 (e)).

It is also conceivable that the target area to be plane-fitted is not limited to the data Er1, Er2 and its vicinity. After determining the boundary B1 that minimizes the number of data exceeding the depth of field, all data of the small sections including the data Er1 and Er2 may be set as the target data for the plane fitting. FIG. 6 (f) shows a case where the boundary B3 is obtained by changing the boundary and calculating all the data of the small sections as the target data for plane fitting. At the boundary B3, the regions Er1 and Er2 whose residuals exceed the depth of field at the boundary B1 are within the depth of field. Further, at the boundary B3, all the residuals included in the small sections R21 and R22 including the region Er1 are within the depth of field. However, in the boundary B3, among the small sections R23 to R26 including the area Er2, in the small section R23, the residual of the area Er21 indicated by the x mark exceeds the depth of field. In this case, in the imaging of the small section R23, the area Er21 may be deleted after the data of the entire area of the small section R23 is acquired. Alternatively, since the area Er21 is acquired when imaging at the boundary B1, the portion of D2 shown in black in FIG. 6G corresponding to the area Er21 is not imaged (so that no light enters). You may image by masking. As described above, the present invention may include a treatment for imaging data having a residual exceeding the depth.
(Imaging condition setting method 3)
The imaging condition setting method 3 will be described with reference to FIG. In this method, the entire area of the subject is imaged at a single boundary by reducing the size of the small section. By using this method, there is a case where the entire region can be imaged by one imaging. On the other hand, there is a case where the entire area of the subject cannot be captured due to the reduction of the imaging area due to the higher magnification of the imaging unit. In this case, it is necessary to move the imaging stage and perform imaging a plurality of times. The numbers shown in FIG. 6H indicate the imaging order. The second imaging is performed by moving the imaging stage in the horizontal direction with respect to the first imaging. Then, the third imaging is performed by moving the imaging stage in the vertical direction with respect to the first imaging. Then, with respect to the first imaging, the imaging stage is moved in the horizontal direction and the vertical direction to perform the fourth imaging. The imaging order need not be in numerical order. In this method, the size of the small section is determined as a single value and does not change depending on the difference in the number of times of imaging. In addition, since the size of the small section is set so that all fitting residuals are within the depth of field, the same region is not duplicated.
(Imaging condition setting method 4)
The imaging condition setting method 4 will be described with reference to FIG. In this method, the same region is imaged twice, and the size of the small section is changed depending on the number of times of imaging. First, a small section having a specific size is set, and a boundary B21 that minimizes the residual exceeding the depth of field is first determined. This is the same as the imaging condition setting method 1. Then, the boundary B22 is determined so that the residual exceeding the depth of field at the boundary B21 falls within the depth of field. The size of the small section set when determining the boundary B22 can be changed. This is different from other imaging condition setting methods. Among the plurality of residuals exceeding the depth of field at the boundary B21, there may be a residual exceeding the depth of field even when the plane fitting is performed instead of the boundary B22. In this case, a boundary B23 is determined so that these residuals fall within the depth of field. The size of the small section set when determining the boundary B23 can be changed. In this way, the boundary is changed so that all the residuals are within the depth of field, and images are taken in duplicate. In the determination of the boundaries B22 and B23, a condition can be set for selecting one in which all the residuals in the small sections are within the depth of field. Further, if only the part that becomes the residual exceeding the depth of field at the boundaries B21 and B22 is set to be within the depth of field at the boundaries B22 and B23, all the residuals in the small sections are covered. A condition can be set that it is not necessary to be within the depth of field. Under the latter condition, the boundary adoption criteria become loose, and it is easy to adopt a boundary having a size of a small partition for each region where there is a residual exceeding the depth of field. However, it is necessary to delete the imaging data of the area where there is a residual exceeding the depth in the small section. For this reason, the imaging method tends to be complicated. If you select a condition that selects all the data in the subdivision within the depth of field, the processing to delete the imaging data in the area where there is a residual exceeding the depth of field becomes unnecessary, and the imaging method It can be simplified. Further, the sampling pitch of the imaging data changes with the change of the size of the small section. For this reason, for example, a process for interpolating the data obtained by the (n−1) -th and n-th imaging so as to have the same sampling pitch is required.
(Flow for setting shooting conditions)
The flow of the imaging condition setting methods 1 to 4 will be described with reference to FIGS. This flow corresponds to the details of step s203 in FIG. For this reason, in the plane fitting in step s201 of FIG. 5, the region where the residual exceeds the depth of field is determined. Then, a region where the residual exceeds the depth of field by the determination in step s202 of FIG. 5 (for example, regions Er1 and Er2 of FIG. 6D) is defined as A1.

  First, the flow of the imaging condition setting method 1 will be described with reference to FIG. In step s101, an area A1 in which the residual exceeds the depth of field in the first imaging (N = 1) is read. In step s102, a boundary that minimizes the number of residuals in the region A1 is determined. In the plane fitting performed when calculating the residual, only the area A1 and its vicinity are set as the target data for the plane fitting. Then, only the z shift and the z tilt are changed without changing the boundary B1. In step s103, it is determined whether or not the residual is within the depth of field in the entire area A1. If it does not fit, the process proceeds to step s104, and if it does not fit, the process proceeds to s105. In step s104, since all residuals are within the depth of field in step s103, the boundary position, the tilt and shift of the imaging surface, and the number of imaging operations are determined. For example, in the case shown in FIGS. 6B and 6C, first imaging is performed at the boundary B1. Next, when the small section including the areas N1 and N2 under the boundary B1 is imaged, the second imaging is performed by changing the relative position and inclination of the object plane and the imaging plane in the optical axis direction. That is, the number of times of imaging is two. In step s105, since all the residuals did not fall within the depth of field in step s103, an increase in the number of imaging is counted in order to capture the residual that did not fall within the depth of field. In step s106, it is determined whether the number of imaging has exceeded a preset upper limit. If it does not exceed the upper limit, the process proceeds to step s101, and thereafter proceeds to the flow described above. If the upper limit is exceeded, the process proceeds to step s107. In step s107, when the number of times of imaging exceeds the upper limit, an area exceeding the depth of field is displayed and the process ends.

  Next, the flow of the imaging condition setting method 2 will be described with reference to FIG. In step s101, the area A1 in which the residual exceeds the depth of field in the first imaging is read. In step s102, a boundary that minimizes the number of residuals in the region A1 is determined. As described above, in the imaging condition setting method 2, the boundary is changed from the boundary B1. In the plane fitting, there are cases where only the area A1 and its vicinity are limited to the target data of the plane fitting, and there are cases where it is not limited. In the case of limitation, for example, it corresponds to the boundary B2 in FIG. 6 (e), and in the case of not limiting, for example, it corresponds to the boundary B3 in FIG. 6 (f). If the boundary residual other than the boundary B1 is calculated when the boundary B1 is obtained in steps s201 and s202 in FIG. 5, the boundary where the region A1 is minimum is read out as the boundary B3 from the result of this calculation. Also good. Further, the boundary B3 may be calculated again. As described above, in the residual of the plane fitting at the boundary B3, even if the residual other than the area A1 (for example, Er21 in FIG. 6F) exceeds the depth of field, the residual in the area A1 is not captured. If it falls within the depth of field, it can be adopted as the boundary B3. In step s103, it is determined whether all the residuals in the area A1 fall within the depth of field in the residuals obtained in step s102. The subsequent flow is the same as the imaging condition setting method 1. In the imaging condition setting method 2, the boundary changes, but the size of the small section does not change.

  Next, the flow of the imaging condition setting method 3 will be described with reference to FIG. In step s301, the area for plane fitting is designated as the entire area. This means that the target data to be plane-fitted is not limited to a part such as a residual exceeding the depth of field. The plane to be fitted is different for each small section. In step s302, the size of the small section is set. In step s303, it is determined whether the number of times of imaging exceeds a predetermined upper limit. Depending on the size of the small section, it may not be possible to capture the entire region at one time. The number of times of imaging is determined according to the size of the small section (imaging magnification). For example, in FIG. 6 (h), it is determined as 4 times. If the upper limit is not exceeded, the process proceeds to step s304, and if the upper limit is exceeded, the process proceeds to step s308. In step s308, an area where there is a residual exceeding the depth of field is displayed and the process ends. In step s304, the boundaries of the small sections are set. In step s305, all the small sections are plane-fitted and the residual is calculated. In step s306, it is determined whether all the residuals are within the depth of field. If all the residuals are within the depth of field, the process proceeds to step s307. If not, the process proceeds to step s302, the size of the small section is reset, and the same flow is repeated. At this time, the area for plane fitting is always the entire area. In step s307, the boundary having the size of the small section common to all regions, the position and the inclination of the plane are determined, and the process ends.

  Next, the flow of the imaging condition setting method 4 will be described with reference to FIG. In step s401, an area A1 exceeding the depth of field of the first imaging is read out. In step s402, the size of the small section is set. In step s403, a boundary for dividing the area A1 into small sections is set. In step s403, the residual of the plane fitting is calculated. In step s405, it is determined whether there is a boundary that makes the residual exceeding the depth of field within the depth of field. At the time of determination, it is possible to set a condition that the data within the small section is limited to a boundary within the depth of field. Under this condition, the process of deleting data having a residual exceeding the depth of field in the small section can be omitted. Further, if data exceeding the depth of field is within the depth of field, it is possible to set a condition that all the data in the small section need not be within the depth of field. If there is no boundary satisfying these conditions, the process returns to step s402. If there is a boundary satisfying these conditions, the process proceeds to step s406. In step s406, it is determined whether all the fitting residuals in the area A1 fall within the depth of field. If it falls within the range, in step s407, the boundary, the position and inclination of the plane, and the number of times of imaging are determined, and the process ends. If not, the process proceeds to step s408. In step s408, the count N of the number of imaging is increased by one. In step s409, it is determined whether the number of imaging has exceeded the upper limit. If the upper limit is exceeded, an area where the fitting residual exceeds the depth of field is displayed in step s410, and the process ends. If the upper limit is not exceeded, the process returns to step s402, and the same flow is repeated thereafter.

  In this embodiment, the imaging condition setting method 0 will be described in detail.

  First, in step s1 of FIG. 4, the height of the object is acquired by surface shape measurement. Complement processing for increasing the number of data by complementing the height obtained by the surface shape measurement may be performed. In this embodiment, the complement process is omitted. The height obtained by the surface shape measurement is shown in FIG. The height is obtained in increments of 0.25 mm.

  Next, the process proceeds to step s2 in FIG. 4 to optimize the size of the small section, the imaging position, and the inclination. In this embodiment, the size of the subject is 10 mm × 10 mm on the object plane. The subject is imaged by dividing it into small sections of 2 mm × 2 mm. An area corresponding to one small section is imaged by one image sensor. The size of the imaging surface of the imaging device is 40 mm × 40 mm, and the imaging magnification is 20 times. There are 9 × 9 = 81 pieces of height data in a 2 mm × 2 mm small section. The boundary can be changed in 81 ways. When the boundary position is optimized with respect to the subject, the boundary is shifted by one small section at the maximum. For this reason, it is assumed that the imaging unit 300 can capture a range of 12 mm × 12 mm in consideration of the size of the subject and the boundary being shifted by one small section at the maximum. When the entire area is imaged, 36 small sections are imaged. In this embodiment, the difference between the height obtained by surface shape measurement and the approximate height obtained by plane fitting is evaluated as a residual. In this embodiment, the depth of field D is calculated by the following equation. k2 is a constant, λ is a used wavelength, and NA is an object-side numerical aperture of the imaging unit. Note that the subject depth may be arbitrarily set by the calculator.

  In this embodiment, since k2 = 1.0, λ = 0.54 μm, and NA = 0.6, the depth of field is D = ± 1.5 μm on the object plane.

  In step s201 of FIG. 5, the boundaries of the small sections are set, and the residual is calculated. In the present embodiment, 81 boundaries are changed, and the whole area is planarly fitted at each boundary, and the number of data whose residual exceeds the depth of field (1.5 μm) is acquired.

  Note that various methods such as changing the boundary (relative position between the subject and the boundary) using, for example, the steepest descent method can be applied. When using the steepest descent method, first, a plane approximation height P having the data position (X, Y), boundary position (m, n), plane inclination and position (G, H, C) as variables is obtained. . Then, a residual E between the height P and the height Z obtained by the surface shape measurement is obtained. Then, the residual E is differentiated by each variable, and each variable is changed by a minute amount in the direction in which the residual changes most greatly in the direction in which the residual becomes smaller. In this way, the set of variables that minimizes the residual is determined.

  In this embodiment, a boundary where the number of residuals exceeding the depth is zero is calculated, and the result is shown in FIG. FIG. 11A shows an initial boundary line superimposed on the surface shape measurement value. FIG. 11B shows the boundary lines of the boundary such that the number of residuals exceeding the depth of field is zero.

  In step s202 of FIG. 5, since all the residuals are within the depth of field, the setting of the imaging conditions is ended. The boundary position and the slope of the fitted plane (G, H in equation (1)) and the position with respect to the optical axis direction (C in equation (1)) are determined. The number of imaging is one.

  Next, divided imaging in step s3 in FIG. 4 is performed. Here, the imaging unit of FIG. 2 is considered. The imaging stage 501 is moved in accordance with the boundary position determined in step s2 in FIG. 4 to move the subject on the xy plane. Further, the drive unit is configured to determine the position and inclination of the reflecting members 503 and 506 corresponding to the inclination of the plane and the positions relative to the optical axis (G, H, and C in Expression (1)) as a result of the plane fitting of each small section. Adjust using 512,513. Both the inclination G in the xz plane and the inclination H in the yz plane are within a range of ± 2 mrad. The height adjustment amount in the z direction is in the range of ± 5 μm on the object side. When adjusting the relative position on the image side, the magnification is taken into consideration. In this way, after adjusting the relative position of the subject and the imaging surface, the second imaging optical systems 504 and 507 are used to capture images independently for each of the small sections. In this manner, the entire subject is divided and imaged at the relative position between the subject position corresponding to the boundary (FIG. 11B) and the image sensor.

  Next, the data capture in step s4 of FIG. 4 is performed. In step s4, the data of the partial image captured in step s3 is tackled. Then, the images of the small sections are synthesized (joined together). If necessary, an image, an analysis result, etc. are displayed using the display device 3 of FIG. Then, the image data is stored in the storage device, and the process ends.

  In the present embodiment, the imaging condition setting method 2 will be described in detail. This corresponds to the case where there is a fitting residual exceeding the depth in step s202 of FIG.

  In this embodiment, the surface shape measurement value is the same as that in the first embodiment. In this embodiment, since k2 = 1.0, λ = 0.54 μm, and NA = 0.73, the depth of field is D = ± 1.0 μm on the object plane.

  As in Example 1, when the boundary was changed in 81 ways, the number of small sections in which three of the boundaries B79 to B81 had a residual exceeding the depth of field was 1, which was the smallest. There is no boundary where the fitting residuals of all sub-parts fall within the depth. That is, the entire region cannot be imaged within the depth of field by the single imaging shown in FIG.

  In step s101 of FIG. 7, the region A1 where the fitting residual exceeds the depth is read out in the first imaging. In the present embodiment, in the first imaging, the number of residuals exceeding the depth of field is the smallest at the three boundaries B79 to B81. Here, considering not only the number of residuals exceeding the depth of field but also the amount of residuals, the boundary B80 having the smallest amount of fitting residuals is selected, and the region A1 of the boundary B80 is read. The positions of the five residuals exceeding the depth of field are indicated by crosses in FIG. FIG. 12 shows a subject 121 and a small section boundary B80. All of the five residuals exceeding the depth of field exist in one small section B80R1. The residual value of the small section B80R1 is shown in FIG. The unit of the numerical value is μm. The five residuals E1 to E5 exceed the depth of field.

  In step s102 of FIG. 7, a boundary where more of the five residuals E1 to E5 are within the depth of field is determined. In the present embodiment, the determination can be made using the boundary residual information already calculated in steps s201 and s202 of FIG. In the present embodiment, all five fitting residuals are fitted within the depth at the boundary B72. FIG. 14 shows the relative positions of the boundary B72 and the boundary B80. The five residuals exceeding the depth of field that existed in one small section at the boundary B80 are distributed in two small sections B72R1 and B72R2 at the boundary B72. FIG. 15A shows the residual value of the small section B72R1 at the boundary B72. The unit of the numerical value is μm. All of the residuals E1 to E3 exceeding the depth of field at the boundary B80 are within the depth of field, and all the residuals of the small sections are within the depth. FIG. 15B shows the residual value of the small section B72R2 at the boundary B72. Both of the residuals E4 and E5 exceeding the depth of field at the boundary B80 are within the depth of field. On the other hand, the seven residuals indicated by the broken lines E21 and E22 exceed the depth. However, these data are data that can be fitted within the depth of field at the boundary B80.

  In step s103 in FIG. 7, it is determined whether or not the residuals in the entire area A1, that is, the five residuals E1 to E5 are within the depth of field. As described above, the five residuals E1 to E5 are all within the depth of field by using the two boundaries of the boundary B80 and the boundary B72.

  In step s104 in FIG. 7, the position and inclination of the boundary and the number of imaging are determined. The position of the boundary is determined by the boundary B80 and the boundary B72. For the position and inclination of the plane, the results obtained by plane fitting (G, H, C in equation (1)) are adopted. In the boundary B80, the position and inclination are different for all 36 small sections. At the boundary B72, it is sufficient to image only the data exceeding the depth at the boundary B80, so the positions and inclinations of the two planes of the small sections B72R1 and B72R2 are sufficient. The number of times of imaging is two times of imaging of the boundary B80 and imaging of the boundary B72.

  Thus, the imaging condition setting method 2 is completed.

  The subsequent flow after divided imaging is the same as that of the first embodiment except that the number of times of divided imaging is changed. Similar to the first embodiment, the subject position and the position and inclination of the reflecting member are adjusted corresponding to the boundary B80 to capture an image. Then, in response to the boundary B72, the subject position and the position and inclination of the reflecting member are newly adjusted, and the second imaging is performed. In the second imaging, only the two sections B72R1 and B72R2 need be imaged as described above. Of course, the entire area may be imaged. In imaging of the small section B72R2, the imaging surface may be masked so that the seven data of the broken line portions E21 and E22 shown in FIG. Alternatively, the seven data of the broken line portions E21 and E22 may be imaged, and the seven data of the broken line portions E21 and E22 may be deleted by capturing the data in step s4 in FIG.

  In the third embodiment, the imaging condition setting method 4 will be described in detail.

  FIG. 16 shows the height obtained by the surface shape measurement. The height is obtained as data in increments of 0.25 mm. The size of the small section is 2 mm × 2 mm as in the first embodiment. One small section includes 9 × 9 = 81 pieces of data. The range that can be imaged by the imaging unit is 12 mm × 12 mm. In the present embodiment, as in the second embodiment, the depth of field is set to ± 1.0 μm on the object plane. Further, the difference between the height obtained by the surface shape measurement and the approximate height obtained by plane fitting is evaluated as a residual.

  FIG. 17 shows a subject 191 and a boundary B301. Subsections (B301R1, B301R2, B301R3) that contain residuals that do not fit within the depth of field are shown in dark gray. The residual values of these three subdivisions are shown in FIG. Residuals that do not fit within the depth of field are shown enclosed in ellipses. Residuals that do not fit within the depth of field are gathered mainly at the edge of the subdivision. There are 18 fitting residuals that do not fit within the depth of field.

  In step s401 of FIG. 9, the area A1 that exceeds the depth of field of the first imaging is read out. The area A1 is the three small sections (B301R1, B301R2, B301R3) in FIG.

  In step s402, the size of the small section is set. In this example, the size of the small section was 1.75 mm × 1.75 mm. This is the size of 8 × 8 = 64 data contained in the small section.

  In step s403, the boundaries of the small sections that divide the area A1 are set, and in step s404, the residual is calculated.

  In step s405, it is determined whether there is a boundary that makes the residual exceeding the depth of field within the depth of field. In this determination, in this embodiment, a condition is set in which all the data in the small section is within the depth of field. This condition will be specifically described. For example, when there are 18 residuals exceeding the depth of field in the area A1, it is assumed that a boundary B31 that can accommodate one of the residuals N31 within the depth of field is obtained. In the boundary B31, when the small section including the residual N31 is R31, the condition is that all the residuals included in the small section R31 must be within the depth of field. The boundary B31 satisfies the condition if the residual (N31) exceeding one depth of field falls within the depth of field in this way. In the present embodiment, the process proceeds to step s406 even if the residuals exceeding the other 17 depths of field do not fall within the depth of field.

  In step s406, it is determined that the residuals that exceed all 18 depths of field in area A1 do not fall within the depth of field. Therefore, the size of the small section is reduced so that the data included in the small section is small, and the size of 1.25 mm × 1.25 mm, and all residuals in the small section are within the depth of field. . Since there are a plurality of boundaries where all the residuals fall within the depth of field, a boundary that minimizes the number of small sections required to image the residual exceeding the depth of field is selected here. This boundary is shown as a boundary B302 in FIG. In the boundary B301, the small sections (region A3) including the residual that does not fall within the depth of field are shown in dark gray in the small sections (B301R1, B301R2, B301R3) as in FIG. By increasing the magnification, the subject area that can be imaged by 36 small sections is reduced to 7.50 mm × 7.50 mm, but the area A3 can be imaged once at the boundary B302. The residual value at the boundary B302 of the region A3 is shown in FIG. The unit is μm. For example, in the small section B302R1, the residual exceeding the depth of field surrounded by the ellipse is −0.53 within the depth, and all the residuals within the small section B302R1 are within the depth of field. . Accordingly, the process proceeds to step s406.

  In step s406, since it is determined that the residuals exceeding all 18 depths of field in area A3 are within the depth of field, the process proceeds to step s407.

  In step s407, the position and inclination of the boundary and the plane, and the number of times of imaging are determined. As the boundaries, boundaries B301 and B302 are used. At each boundary, the plane position and inclination (G, H, C in equation (1)) are determined from the result of calculating the plane fitting. By adjusting the positions of the imaging stage and the reflecting member, the relative position between the subject and the imaging surface is set so as to correspond to the boundary B301, and imaging is performed once. Then, after the imaging magnification is increased from 20 times to 32 times so that the area A3 including 18 residuals exceeding the depth of field corresponds to the boundary B302, the positions of the imaging stage and the reflecting member are adjusted. Thus, the relative position between the subject and the imaging surface is set, and the second imaging is performed. At this time, as described above, the entire area A1 can be imaged by one imaging. That is, the number of times of imaging is two times, one time at the boundary B301 and one time at the boundary B302. The subsequent flow after divided imaging is the same as that of the first embodiment except that the number of times of divided imaging is changed.

  Consider a case where the imaging magnification is set to 32 times by changing the magnification of the second imaging optical systems 504 and 507 without changing the magnification of the first imaging optical system 502 in FIG. At this time, if the magnification is changed by changing the NA on the object side without changing the NA on the image side of the second imaging optical systems 504 and 507, the magnification is picked up by the second imaging optical systems 504 and 507. The amount of movement of the surface is not affected. The subject and the imaging surface are adjusted so as to correspond to the boundary of the boundary B302 by adjusting the movement amount in the optical axis direction of the reflecting members 503 and 506 on the object plane of the second imaging optical systems 504 and 507 and the inclination with respect to the optical axis. The relative position of can be set.

  In the present embodiment, even if the size of the small section is changed, the difference exceeding the depth of field is such that the difference between the height obtained by the surface shape measurement and the approximate height obtained by plane fitting exceeds 1 μm. . However, for example, when the imaging magnification is increased to reduce the size of the small section, this condition may be changed according to the magnification. The amount of residual that can be allowed can be arbitrarily set by the calculator regardless of the depth of field.

  In this embodiment, not only the area A1 but also the entire area of the subject can be imaged within the depth of field at the boundary B302. When the entire region is imaged by imaging using only the boundary B302 without imaging at the boundary B301, imaging is performed four times as can be seen from FIG. This is a solution in the case of the imaging condition setting method 3 shown in FIG. After obtaining the solution of the imaging condition setting method 4, the imaging condition setting method 3 may be performed to select an imaging method with a smaller number of times.

  In the present embodiment, the size of the small section is gradually reduced so that the number of data included in the small section is reduced, but other methods are also conceivable. For example, it is conceivable to obtain period information in the x and y directions from the spatial frequency information of the height data obtained by the surface shape measurement and determine the size based on the period information. It is also conceivable to determine the size so that the amount of change in the height data obtained by the surface shape measurement is less than a certain value. It is also possible to use the gradient method (the steepest descent method). When using the steepest descent method, calculate as follows. A plane approximation height P having the data position (X, Y), the boundary position (m, n), the inclination of the plane, the position (G, H, C), and the size (R) of the small section as variables is obtained. Then, a residual E between P and the height Z obtained by the surface shape measurement is obtained. Next, the residual E is differentiated by each variable, and each variable is changed by a minute amount in the direction in which the residual changes most greatly in the direction in which the residual decreases. In this way, the set of variables that minimizes the residual is determined.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

2 arithmetic unit (processing means)
100 Measuring unit (measuring means)
300 Imaging unit (imaging means)
401 Control unit (adjustment means)

Claims (18)

An optical system for forming an image of a subject;
Imaging means for imaging the imaging range of the subject for each of a plurality of small sections via the optical system;
Measuring means for measuring the height of the subject in the optical axis direction of the optical system before imaging the subject by the imaging means;
Processing means for obtaining an imaging condition for each of the small sections by calculating a residual based on the height of the subject and an approximate height in a plane approximating the height distribution of the subject;
An imaging system comprising: an adjusting unit that adjusts a relative positional relationship between the small section and the imaging unit based on the imaging condition.   The imaging condition includes at least one of a position of the small section with respect to the imaging unit on a plane perpendicular to the optical axis of the optical system, a distance between the small section and the imaging unit, and an inclination of the small section with respect to the imaging unit. The imaging system according to claim 1, wherein   The imaging system according to claim 1, wherein the processing unit acquires an imaging condition in which the residual is within a predetermined depth of field.   The imaging system according to claim 3, wherein the depth of field is determined using a use wavelength, a numerical aperture of the optical system, and an imaging magnification of the optical system.   The imaging system according to any one of claims 1 to 4, wherein the processing unit includes the size of the small section as a variable when calculating the approximate height.   6. The imaging system according to claim 5, wherein the size of the small section is determined using a size of an imaging surface of the imaging unit and an imaging magnification of the optical system.   The imaging system according to claim 5 or 6, wherein the processing means performs either deletion or interpolation of a part of imaging data in the small section.   Using the size of the small section determined using the size of the imaging surface of the imaging means and the imaging magnification of the optical system, the wavelength used, the numerical aperture of the optical system, and the imaging magnification of the optical system The imaging system according to claim 1, further comprising an input unit configured to input at least one of the depths of field determined in this way. The processing means sets a first imaging condition when a first residual based on an approximate height on a plane approximating the height of the subject and the height distribution of the subject exceeds a predetermined depth of field. After the acquisition, the second imaging condition is acquired by calculating a second residual based on an approximate height in a plane approximating the height distribution of the area including the height of the subject and the first residual. ,
The adjusting unit adjusts a relative positional relationship between the small section and the imaging unit based on the first imaging condition, and after being imaged by the imaging unit, based on the second imaging condition. The imaging system according to claim 1, wherein a relative positional relationship between the small section and the imaging unit is adjusted.   The imaging system according to claim 9, wherein the imaging unit does not capture an area different from the area after the positional relationship is adjusted based on the second imaging condition.   The processing means deletes an area different from the area from the subject imaged by the imaging means after the positional relationship is adjusted based on the second imaging condition. Or the imaging system of 10. The optical system includes: a first imaging optical system that images an object; a second imaging optical system that re-images the object imaged by the first imaging optical system; Reflecting means disposed between the first and second imaging optical systems,
The at least one of the position of the reflecting means in the optical axis direction and the inclination of the reflecting means with respect to the optical axis can be changed based on the imaging condition. The imaging system according to item.   The imaging system according to claim 1, wherein the processing unit calculates the residual using a part of the data of the approximate height.   The imaging system according to any one of claims 1 to 13, wherein the processing unit performs interpolation to increase the number of data of the height of the subject.   The imaging system according to any one of claims 1 to 14, wherein the processing unit calculates spatial frequency information using a height of the subject.   16. The residual according to claim 1, wherein the residual is one of a difference between the height of the subject and the approximate height, or a root mean square of a difference between the height of the subject and the approximate height. The imaging system according to claim 1.   The imaging system according to any one of claims 1 to 16, wherein the processing unit calculates the approximate height using one of a least square method and a gradient method. An imaging method for an imaging system, comprising: an optical system that forms an image of a subject; and an imaging unit that captures an imaging range of the subject for each of a plurality of small sections via the optical system,
Measuring the height of the subject in the optical axis direction of the optical system;
Obtaining an imaging condition for each of the small sections by calculating a residual based on the height of the subject and an approximate height in a plane approximating the height distribution of the subject;
Adjusting the relative positional relationship between the small section and the imaging means based on the imaging condition;
And a step of imaging the small section by the imaging means.