JP4833743B2 - Stage motion correction method by image montage processing - Google Patents

Description

  The present invention relates to a stage operation correction method by image montage processing, and more particularly, to correct a stage operation shift when a montage image is created by a transmission electron microscope (TEM).

  The TEM stage can be accurately moved to the specified coordinates, but may be slightly shifted depending on the location. This is caused by operation errors of gears and springs constituting the stage, and is unavoidable with hardware. Therefore, when creating a montage, there is a method to make the shift inconspicuous by increasing the number of shot images that make up the montage, reducing the size per shot, and shortening the distance of stage movement between shot images. I'm taking it.

  As a conventional technique of this kind, the sample is moved so that the electron beam is scanned in each of the sections obtained by virtually partitioning the sample in a lattice pattern in units of the area in which the image is captured in the electron beam scanning area. An apparatus is known that has sample movement control means and creates composite image data by storing image data obtained for each section in the image memory in association with the position of each section (for example, Patent Document 1). reference).

  Further, there is known an apparatus in which an area within a scanning range is repeatedly observed several times so that a part of each scanning area overlaps and the image data of the overlapping parts are connected to match each other. (For example, refer to Patent Document 2).

Also, an image processing apparatus for continuously combining a plurality of digital images taken while continuously moving the shooting place or the shooting direction so that a part of the taken digital images overlaps, and synthesizing one panoramic image Is known (see, for example, Patent Document 3).
JP 2001-250500 A (paragraphs 0008 to 0016, FIGS. 1 and 2) JP-A-6-34313 (paragraphs 0008 to 0012, FIGS. 1 to 5) JP 2004-139219 A (paragraphs 0026 to 0029, FIGS. 1 to 4)

In the above-described conventional technique, the number of photographed images constituting the montage is increased to reduce the size per image, and thus there is a problem that processing takes time.
The present invention has been made in view of such a problem, and provides a stage motion correction method by image montage processing that can shorten the creation time of a montage image and can create a more accurate montage image. It is aimed.

(1) The invention described in claim 1 includes a step 1 for calibrating the stage and the image photographing means, a step 2 for calculating the photographing position so as to fill the entire moving range of the stage, and the stage at the first photographing position. Step 3 for moving the image, Step 4 for obtaining the image data A from the transmission image obtained by photographing the drawing unit of the divided sample by the image photographing means, and converting the obtained image data A from the image photographing coordinate system to the montage image coordinate system. Step 5 for storing in the first buffer, Step 6 for moving the stage to the next photographing position, obtaining image data B from the transmission image photographed by the image photographing means, and photographing the obtained image data B Step 7 for converting from the coordinate system to the montage image coordinate system and storing it in the second buffer, and recording in the first buffer and the second buffer. And Step 8 for obtaining the shift amount of the two images subjected to matching the images, the images of the second buffer stored in the first buffer being shifted by shift amount calculated above, the second buffer Repeat the process from Step 9 where the empty data is entered again and from Step 6 to Step 9, and create a correction map using all the obtained deviations as coordinate data converted from the montage coordinate system to the stage coordinate system. And step 11 for obtaining coordinates for moving the stage using the correction map when performing montage.
(2) The invention described in claim 2 is characterized in that the stage is moved so as to have the same vector when reaching the target position .

(1) According to the first aspect of the present invention, it is possible to provide a bad stage operation correction method by image montage processing that can shorten the creation time of a montage image and can create a more accurate montage image.
(2) According to the second aspect of the present invention, the stage can be moved accurately without the influence of the stage system gear backlash or the like .

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram showing a configuration example of a transmission electron microscope (TEM) for carrying out the present invention. In the figure, reference numeral 1 denotes a lens barrel. The components of the apparatus are included in the lens barrel 1. The inside of the lens barrel 1 is maintained in a high vacuum state. The electron beam EB generated from the electron gun 2 is focused by the focusing lens 3 and irradiated onto the sample 4. The sample 4 is placed on the sample stage 5 and is moved in a two-dimensional direction by the stage moving mechanism 6.

  Reference numeral 7 denotes an objective lens that focuses the image transmitted through the sample 4, and 8 denotes an electron lens that focuses the image focused by the objective lens 7. Reference numeral 9 denotes a CCD camera as an image photographing means for photographing an image transmitted through the sample 4, and 10 denotes a fluorescent screen provided slightly on the lower surface of the CCD camera 9. When the CCD camera 9 is removed, the image is emitted from the phosphor screen 10 and can be viewed as an image from a viewing window.

  On the other hand, an image photographed by the CCD camera 9 is input to the image processing unit 12 to be converted into a digital image, subjected to predetermined image processing, and then input to the control device 13. The control device 13 executes the present invention described later. As the control device 13, for example, a computer is used. The control device 13 controls the stage moving mechanism 6. A memory 14 stores image data and the like. The control device 13 is also used for controlling the operation of each component of the TEM.

  In the apparatus configured as described above, the electron beam EB emitted from the electron gun 2 is focused by the focusing lens 3 and then irradiated to the sample 4. At this time, the transmission image transmitted through the sample 4 is focused by the objective lens 7 and then taken by the CCD camera 9. The captured transmission image is subjected to predetermined image processing by the image processing unit 12 and then input to the control device 13. The controller 13 uses the memory 14 to perform stage operation correction by montage processing as will be described later.

  FIG. 2 is a flowchart showing the operation of the present invention. The operation of the present invention will be described below along this flowchart. First, the stage and the CCD camera are calibrated (S1). This is because it is necessary to recognize how the image taken by the CCD camera in the stage is represented by the stage coordinates. In this calibration, the stage coordinates and the CCD camera coordinates are calibrated so that the coordinate conversion between the captured image coordinates (CCD camera coordinates) and the montage image coordinates can be performed. Specifically, the deviation amount between the stage coordinates and the CCD camera coordinates is obtained in advance.

  FIG. 3 is a diagram showing the relationship between stage coordinates and camera coordinates. In the figure, P is stage coordinates, and Q is camera coordinates. The position of the CCD camera coordinate Q is fixed based on being mechanically attached. As shown in the figure, even when the sample stage 5 moves, the camera coordinates Q are inclined with respect to the stage coordinate system as shown in the figure. Therefore, the image taken by the CCD camera 9 is taken at an angle shown in the figure.

Next, the stage movement method and the photographing position are calculated so as to fill the entire movement range of the stage (S2). Here, the shooting position is calculated using the stage coordinates. A method for moving the stage will be described later. Next, create a montage image. The montage image is created by the following sequence. The control subject is the control device 13 (see FIG. 1).
1) Create a montage image buffer A and a buffer B, and store empty data (S3). FIG. 2 shows the buffer A and the buffer B. Here, the buffer A and the buffer B are provided in the memory 14 connected to the control device 13.
2) Next, the control device 13 operates the stage moving mechanism 6 to move the stage 5 to the first photographing position (S4).
3) Next, an image is taken by the CCD camera 9 to obtain image data (S5).
4) Next, the control device 13 puts the acquired image data of the CCD camera 9 into the buffer A (S6). At that time, the control device 13 performs coordinate conversion between the captured image and the montage image, assuming that the coordinates of the stage are correct. Then, the image data converted from the CCD camera coordinate system to the montage coordinate system is stored in the buffer A. The photographed image is a CCD camera coordinate system, and the montage image is a montage coordinate system. The montage coordinate system is a coordinate system determined by coordinates and image magnification.
5) Next, the control device 13 operates the stage moving mechanism 6 to move the stage 5 to the next photographing position (S7).
6) Next, an image is taken by the CCD camera 9 to obtain image data (S8).
7) Next, the control device 13 puts the acquired image data into the buffer B (S9). At that time, the control device 13 performs coordinate conversion between the captured image and the montage image, assuming that the coordinates of the stage are correct. In this case, the image data converted from the CCD camera coordinate system to the montage coordinate system is stored in the buffer B.
8) Next, the control device 13 calculates matching between the images of the buffer A and the buffer B (S10). In this case, the image matching is performed focusing on only the area having data in both the images of the buffer A and the buffer B, and the deviation between the two images is obtained. How to determine the deviation will be described later. This shift is stored in the memory 14.
9) Next, the control device 13 shifts the coordinates of the buffer B by the shift amount obtained in 8) and puts it in the buffer A (S11). In this case, empty data is again stored in the buffer B.
10) Repeat steps 5) to 9).

  Next, the control device 13 checks whether it has moved to all areas (S12). If the entire area is not reached, the process returns to step S7 to move the stage. When the entire area is reached, a correction map is created (S13). Here, since the shift amount stored in the memory 14 is a montage coordinate system, the conversion into the stage coordinate system is saved in the memory 14 and used as a correction map. Details of the correction map will be described later. When performing montage, the stage is moved using the correction map. A method for obtaining coordinates using the correction map will be described later.

As described above, according to the present invention, the creation time of a montage image can be shortened, and a more accurate montage image can be created.
Next, a method for moving the stage will be described. FIG. 4 is a diagram showing the relationship between the stage movement and the CCD camera imaging region. Reference numeral 20 denotes a stage moving area. The first shooting area when the stage advances as shown by an arrow is 21, and the last shooting area is indicated by 22. Reference numeral 23 denotes a photographing region of one frame of the CCD camera, which is shifted and overlapped. The arrow → in the figure indicates the moving direction of the stage moving mechanism 6. The stage moving mechanism 6 is moved so as to have the same vector when reaching the target position in consideration of stage backlash. By doing so, it is possible to move the stage accurately without the influence of the stage system gear backlash or the like.

Next, how to determine the deviation will be described. The areas where both the image A and the image B have data are referred to as an image A ′ and an image B ′, respectively. There are several methods for finding a matching position, and two of them will be described below. Specifically, the control device 13 performs the following method.
(Method 1) The cross-correlation coefficient between the image A ′ and the image B ′ is obtained, and the minimum position is set as the most matching position. If such a method is used, matching can be performed by obtaining the cross-correlation coefficient between the images A ′ and B ′ of the area that the two images A and B have in common.
(Method 2) The image B ′ is moved pixel by pixel to find a position that most closely matches the image A ′. When the coordinates are shifted by the pixel which is the image B ′, the difference in luminance at the same coordinate point between the image A ′ and the image B ′ is obtained. Do this at all coordinate points and add the differences. The moving position of the image B ′ when the sum of the differences is minimized is set as the most matching position.

  If such a method is used, the image A ′ and B ′ in the area where both the images A and B have both data, the image B ′ is moved one pixel at a time, and is most matched with the image A ′. Matching can be performed by finding a position to perform.

  FIG. 5 is a schematic diagram in which a difference in luminance at each coordinate point is obtained. (A) is the difference in luminance between the image 25 and the image 26, and the difference is as shown in 27. Since there is no difference in luminance for the common area, the result shown in 27 is obtained. (B) shows the difference in luminance between the image 28 and the image 29, and the difference is as shown at 30. Since the image 28 and the image 29 are exactly the same image and there is no difference in luminance, the result shown in 30 is obtained.

  Next, a method for using the correction map will be described. The correction map is a matrix having a value indicating a deviation amount at each position where the stage 5 is moved. Each stage coordinate (x, y) has a deviation amount (dx, dy).

The correction map has only a deviation amount of the position to which the stage 5 has moved when the correction map is created. Therefore, when it is desired to move the stage to a position that has not moved at the time of creating the correction map, the amount of deviation at that position is interpolated from the amount of deviation at the coordinates of the four nearest points among the coordinates possessed by the correction map. FIG. 6 is an explanatory diagram of shift interpolation. The following are all shown in stage coordinates. The coordinates to be moved are (x, y). The coordinates of the four points on the correction map are as follows.
1. Coordinates (cx ₁ , cy ₁ ), deviation (dx ₁ , dy ₁ )
2. Coordinates (cx ₂ , cy ₂ ), displacement (dx ₂ , dy ₂ )
3. Coordinates (cx ₃ , cy ₃ ), displacement (dx ₃ , dy ₃ )
4). Coordinates (cx ₄ , cy ₄ ), displacement (dx ₄ , dy ₄ )
Is the nearest four points.

A point (cxl ₁₂ , cyl ₁₂ ) on the line connecting points 1 and 2 and a point (cxl ₃₄ , cxl ₃₄ ) on the line connecting points 3 and 4 are defined so as to be a straight line passing through (x, y). The shift (dxa, dya) of (cxl ₁₂ , cyl ₁₂ ) is obtained from linear interpolation between point 1 and point 2,

And Similarly, the shift (dxb, dyb) of (cxl ₃₄ , cyl ₃₄ ) is obtained from linear interpolation between point 3 and point 4,

And The shift of (x, y) is similarly obtained from the linear interpolation of (cxl ₁₂ , cyl ₁₂ ) and (cxl ₃₄ , cyl ₃₄ ).

In this way, the coordinates (x, y) to be moved can be obtained.
As described above in detail, according to the present invention, the creation time of a montage image can be shortened, and a more accurate montage image can be created. Further, when a position that the user wants to observe is found on the montage image, the stage can be accurately moved to that position.

  In the above-described embodiment, a CCD camera is used as the image photographing means. However, the present invention is not limited to this, and any device can be used as long as it can photograph an image. .

  FIG. 7 is an explanatory diagram of the effect of the present invention. (A) is a montage image without correction, and (b) is a montage image when correction according to the present invention is performed. In the case of the montage image shown in (a), the influence by the section 21 appears and is not connected well. On the other hand, in the case of the present invention, a montage image is accurately obtained.

It is a figure which shows the structural example of the transmission electron microscope for implementing this invention. It is a flowchart which shows operation | movement of this invention. It is a figure which shows the relationship between a stage coordinate and a camera coordinate. It is a figure which shows the relationship between a stage movement and the imaging region of a CCD camera. It is the schematic diagram which calculated | required the difference of the brightness | luminance of each coordinate point. It is explanatory drawing of a shift | offset | difference interpolation. It is explanatory drawing of the effect of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens barrel 2 Electron gun 3 Focusing lens 4 Sample 5 Sample stage 6 Stage moving mechanism 7 Objective lens 8 Electronic lens 9 CCD camera 10 Phosphor screen 11 Peep window 12 Image processor 13 Controller 14 Memory

Claims (2)

Step 1 for calibrating the stage and the image capturing means;
Step 2 for calculating the shooting position so as to fill the entire range of movement of the stage;
Step 3 to move the stage to the first shooting position;
Step 4 for obtaining the image data A from the transmission image obtained by photographing the drawing unit of the divided sample by the image photographing means;
Step 5 of converting the acquired image data A from the image capturing coordinate system to the montage image coordinate system and storing it in the first buffer;
Step 6 for moving the stage to the next photographing position and acquiring the image data B from the transmission image photographed by the image photographing means;
Step 7 of converting the acquired image data B from the image capturing coordinate system to the montage image coordinate system and storing it in the second buffer;
And Step 8 for obtaining the shift amount of the two images subjected to matching images stored in the first buffer and the second buffer,
Step 9 of shifting the image of the second buffer by the calculated amount of shift and storing it in the first buffer, and again storing empty data in the second buffer;
Steps 10 to 9 are repeated, and a correction map is created using correction data obtained by coordinate-converting all the obtained shift amounts from the montage coordinate system to the stage coordinate system;
Step 11 for obtaining coordinates for moving the stage using the correction map when performing montage
A stage motion correction method using image montage processing, comprising:   2. The stage motion correction method by image montage processing according to claim 1, wherein the stage is moved so as to have the same vector when reaching the target position.