KR101543417B1 - Improvement Method of Image Alignment Accuracy in Electron-Tomography - Google Patents

Abstract

여기서,

는

번째 표시자의 위치를 나타내며,

는 기울임 각,

는 기울임 축이 x-y 평면과 이루는 편위 (declination)이며,

는 z 방향의 투영행렬을 나타내고,

는 투영 평면상의 회전각,

는

번째 이미지의 축척을 나타내며,

는

번째 이미지에서

번째 표시자의 2차원 위치를 나타내고,

는

번째 이미지의 이차원 평행이동량을 나타낸다.A method for improving image alignment accuracy in electronic tomography is disclosed.
The method for improving the image alignment accuracy includes an EM projection image acquiring step, an image aligning step of aligning the acquired image, a reconstruction step of reconstructing the aligned image into a 3D tomogram (image result), and an electronic In the tomogram, in the image alignment step, the image alignment step is performed by the following formula 1 using a position reference indicator.
[Formula 1]

here,

The

Represents the position of the second marker,

Lt; / RTI >

Is the declination that the tilt axis makes with the xy plane,

Represents a projection matrix in the z direction,

The rotation angle on the projection plane,

The

Represents the scale of the second image,

The

In the second image

Dimensional position of the second indicator,

The

Dimensional image of the second image.

Description

[0001] The present invention relates to a method of improving image alignment in electronic tomography,

The present invention relates to a method for improving the accuracy of image alignment in electronic tomography, which can efficiently perform image alignment, which is an important process in electronic tomography.

Electron-tomography is a three-dimensional structural analysis of a material, with typical resolutions between atomic resolution and X-ray tomography (μm).

With the development of electron-tomography technology, the resolution, which was about 1 nm in 2004, has been improved to 2.4 Å in recent 3-dimensional structure studies of 10 nm nano-gold particles. Electronic tomography is completed through the following four steps. First, acquisition of a 2D EM projection image in which the sample is inclined continuously within a certain range, second, alignment of the obtained image, third, reconstruction with a 3D tomogram, and fourth, visualization process.

One of the most important processes of electron-tomography is to align the TEM image. Due to the constraints of computer computation time and memory, independent 2D 2-D operations are used consecutively for 3-D reconstruction, which can not produce good results if all images are not correctly aligned.

Much research has been done on image alignment, and fiducial markers, such as nanocomposite particles, are used where possible for effective alignment. Luther et al. (1988) have used IMOD S / W (Kremer et al., 1996) as a convenient way to apply various parameters in various ways.

However, there is a drawback that the position reference indicator must be present in all images. Biological samples composed of thin or light carbon elements such as carbon can be used to observe the markers at high angles, however, because thick specimens or heavy element samples do not distinguish specific markers from elevation angles. Jing and Sachs (1991) and Penczek et al. (1995) have suggested various methods of transforming least squares.

Diez et al. (2006) used the Newton method and the Conjugate Gradient method together for efficient solution of the nonlinear least squares method.

Meanwhile, the method for enhancing the image alignment accuracy in the electronic tomography according to the present invention can overcome the limitation of several methods used for aligning images using the position reference indicator. In the modified model, The Levenberg-Marquardt method (Levenberg, 1944; Marquardt, 1963), which is widely used in the field of geophysics for solving mathematical problems, presents a nonlinear inversion equation that can be considered when a sufficient number of position- Were used.

In addition, an effective initial value selection method to improve convergence stability and efficiency is proposed.

Finally, we try to verify the superiority of the method proposed by the present invention through a tomography process for a geopolymer sample.

[1] De Rosier DJ, Kluge: Reconstruction of three dimensional structures from electron micrographs. Nature 217: 130-134, 1968. [2] Diez D C, Seybert A, and Frangakis A: Tilt-series and electron microscope alignment for the non-perpendicularity of beam and tilt-axis. Ultramicroscopy 154 (2): 195-205, 2006. [3] Jing Z, Sachs F: Alignment of tomographic projections using an incomplete set of fiducial markers. Ultramicroscopy 35 (1): 37-43, 1991. [4] Kremer JR, Mastronarde DN, McIntosh JR: Computer Visualization of Three-Dimensional Image Data Using IMOD. J. Struct. Biology 116 (1): 71-76,1996. [5] Levenberg K: A method for the solution of certain problems in least squares. Quart. Appl. Math. 2: 164-168, 1944. [6] Luther PK, Lawrence MC, and Crowther RA: A method for monitoring the collision of plastic sections as a function of electron dose. Ultramicroscopy 24 (1): 7-18, 1988. [7] Marquardt D: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11: 431-441, 1963. [8] Midgley PA, Dunin-Borkowski RE: Electron tomography and holography in materials science. Nature Mater 8: 271-280, 2009. [9] Penczek P, Marko M, Buttle K, Frank J: Double-tilt electron tomography. Ultramicroscopy 60 (3): 393-410, 1995, [10] Scott MC, Chen C-C, Mecklenburg M, Zhu C, Xu R, Ercius P, Dahmen U, Regan BC, Miao J: Electron tomography at 2.4-angstrom resolution. Nature 483: 444-447, 2012. [11] Ziesei, Jong KP, Koster AJ: Electron tomography: A tool for 3D structural probing of heterogeneous catalysts at the nanometer scale. Applied Catalysis A-General 260: 71-74, 2004.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method for enhancing the accuracy of image alignment in electronic tomography that can more effectively perform image alignment in electronic tomography.

Here, the new model expression and minimization formula necessary for the image sorting proposed in the present invention have an advantage that all the selected markers in an arbitrary image can be used, and the Levenberg-Marquardt method stably obtains the solution of the minimization formula, Can be greatly improved.

In addition, it is very useful even when it is difficult to select all of the markers on the whole image because of thick samples or a lot of noise.

According to another aspect of the present invention, there is provided a method for improving the accuracy of image alignment in electronic tomography, comprising the steps of: acquiring an EM projection image; arranging an image to align the acquired image; And a reconstruction step of reconstructing the image using the position reference indicator and the visualization step in the order of the image sorting step, the image sorting step is performed by the following formula 1 using the position reference indicator in the image sorting step .

[Formula 1]

는 i번째 표시자의 위치를 나타내며,

는 기울임 각,

는 기울임 축이 x-y 평면과 이루는 편위 (declination)이며,

는 z 방향의 투영행렬을 나타내고,

는 투영 평면상의 회전각,

는 j번째 이미지의 축척을 나타내며,

는 j번째 이미지에서 i번째 표시자의 2차원 위치를 나타내고,

는 j번째 이미지의 이차원 평행이동량을 나타낸다.
here,

Indicates the position of the i-th marker,

Lt; / RTI >

Is the declination that the tilt axis makes with the xy plane,

Represents a projection matrix in the z direction,

The rotation angle on the projection plane,

Represents the scale of the j-th image,

Represents the two-dimensional position of the i-th indicator in the j-th image,

Represents the two-dimensional translation of the jth image.

와

의 차이를 최소화시키는 미지변수인 축척

, 평면상에서의 회전각

, 기울임 각

, 상기 위치기준 표시자의 3차원 위치

, 평행이동량

를 산출하기 위하여, 아래에 기재된 최소 자승식으로 표현되는 RMS(root mean squares) 오차인 L을 최소화시키는 것을 특징으로 한다.The image aligning step using Equation (1) may include:

Wow

Scale, which is an unknown variable that minimizes the difference

, The rotational angle on the plane

, Tilt angle

, A three-dimensional position of the position reference indicator

, Parallel movement amount

The root mean squares (RMS) error represented by the minimum square matrix described below is minimized.

[Formula 2]

Where N _p is the total number of selected marker positions to be used in the least squares method.

And the Levenberg-Marquardt (LM) method, which is a nonlinear least squares method, is applied in the calculation process for minimizing the L. [

을 기 설정된 값으로 설정한 후, 나머지 변수들을 추정하는 역산단계 및 상기 기울임 각

을 도출하는 기울임 각 추정 단계를 포함하는 것을 특징으로 한다.
The calculation process includes the steps of:

To a preset value, and then estimates remaining variables,

And a tilt angle estimating step of deriving the tilt angle.

The method of improving the image alignment accuracy in electronic tomography according to the present invention is advantageous in that the new model equation and the minimization formula can use all the selected markers in an arbitrary image and the Levenberg-Marquardt method can stabilize the solution of the minimization formula It provides the advantage of saving.

In addition, the calculation time of the basic L-M method can be greatly improved through the initial value selection process.

In addition, it provides an advantage that it can be very usefully used when it is difficult to select all of the markers on the entire image because of thick samples or a lot of noise.

FIG. 1 is an exemplary diagram showing the effect of a local minimum value in minimizing the initial value.
FIG. 2 is a flow chart for explaining an image alignment process using the Lemann-Marquardt (LM) method.
3 is an exemplary view showing a TEM image of a geopolymer.
FIG. 4 is an exemplary view showing the position and alignment process of the indicator in the IMOD S / W.
FIG. 5 is an exemplary view showing the overlapping of the respective sorted images to compare the result using only six indicators and the result using all 30 indicators. FIG.
FIG. 6 is a diagram showing coordinates of points arranged in the upper part of FIG.
Fig. 7 (a) shows a tomographic cross section using six indicators, and Fig. 7 (b) shows an example of a tomographic section using thirty indicators.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a detailed description of preferred embodiments of the present invention will be given with reference to the accompanying drawings. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Embodiments in accordance with the concepts of the present invention can make various changes and have various forms, so that specific embodiments are illustrated in the drawings and described in detail in this specification or application. It is to be understood, however, that it is not intended to limit the embodiments according to the concepts of the present invention to the particular forms of disclosure, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises ",or" having ", or the like, specify that there is a stated feature, number, step, operation, , Steps, operations, components, parts, or combinations thereof, as a matter of principle.

Hereinafter, an image alignment accuracy enhancement method and a recording medium in the electronic tomography according to an embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 is an exemplary diagram showing the effect of a local minimum value in minimizing the initial value.

FIG. 2 is a flow chart for explaining an image alignment process using the L-M (Levenberg-Marquardt) method.

3 is an exemplary view showing a TEM image of a geopolymer.

FIG. 4 is an exemplary view showing the position and alignment process of the indicator in the IMOD S / W.

FIG. 5 is an exemplary view showing the overlapping of the respective sorted images to compare the result using only six indicators and the result using all 30 indicators. FIG.

FIG. 6 is a diagram showing coordinates of points arranged in the upper part of FIG.

Fig. 7 (a) shows a tomographic cross section using six indicators, and Fig. 7 (b) shows an example of a tomographic section using thirty indicators.

First, prior to describing the technique of improving the accuracy of image alignment in electronic tomography according to the present invention, a general image alignment technique will be described.

Generally, image alignment in electronic tomography uses an indicator. In order to align an image using an indicator, a mathematical model equation is first required to confirm how the actual position of the marker on the three-dimensional image is projected on each image.

There are many mathematical model expressions, and Luther et al. (1988) and Diez et al. (2006) are as follows.

First, Luther et al. (1988) presented the following model equation

[Reference expression 1]

는

개의 표시자 중에서 j번째 표시자의 i번째 이미지에서의 2차원 좌표를 나타낸다. y_j는 j번째 표시자의 3D 상에서의 실제 위치좌표를, M은 3x3 대각 행렬로 각 원소

,

,

는 각각 x, y, z 방향으로의 축척계수 (scale factor)를 나타낸다.

는 시료의 기울어진 정도를 정의하는 3x3 회전 행렬로 여기서는 y축을 중심으로 한 기울임만을 고려한다.

는 3차원 시료를 2차원의 투영 이미지로 만드는 투영 행렬 (projection matrix)이며,

는

번째 이미지의 2차원 회전 행렬이다.

는

번째 이미지의 2차원 평행이동 벡터이다.
here,

The

Dimensional coordinate in the i-th image of the j-th marker among the three markers. y _j is the actual position coordinate on the 3D of the jth indicator, M is the 3x3 diagonal matrix,

,

,

Represents a scale factor in the x, y, and z directions, respectively.

Is a 3x3 rotation matrix that defines the degree of tilting of the sample, where only tilts about the y axis are considered.

Is a projection matrix that transforms a three-dimensional sample into a two-dimensional projection image,

The

Lt; th > image.

The

Lt; th > image.

Second, Diez et al. (2006) presented the reference equation 2.

[Reference expression 2]

는 i번째 표시자의 위치를 나타내며,

는 기울임 각,

는 기울임 축이 x-y 평면과 이루는 편위 (declination)이며,

는 z 방향의 투영행렬을 나타내고,

는 투영 평면상의 회전각, s^(j)는 j번째 이미지의 축척을 나타낸다.

는 j번째 이미지에서 i번째 표시자의 2차원 위치를 나타낸다.
here

Indicates the position of the i-th marker,

Lt; / RTI >

Is the declination that the tilt axis makes with the xy plane,

Represents a projection matrix in the z direction,

Is the rotation angle on the projection plane, s ^(j) is the scale of the jth image.

Represents the two-dimensional position of the i-th indicator in the j-th image.

The reference equations 1 and 2 commonly consist of a tilt angle, a scale, a two-dimensional projection, and a two-dimensional rotation around the y-axis, but there are differences in some points.

In reference equation 1, three independent accumulation factors are given in the x, y and z directions, whereas in reference equation 2, only the next projected scale is applied.

In particular, in Ref. Eq. 2, a deflection term is newly added assuming that the tilt axis is not perpendicular to the z axis due to equipment problems or experimental errors.

In addition, in the reference equation 2, the translational term disappears, which is assumed to be a mistake in the notation or omitted in consideration of the electron microscope image acquisition.

First, using the independent scale in the x, y, and z directions in the reference equation 1 increases the flexibility to reduce various errors. However, this is different from the actual experimental environment, and it is realistic to use only one scale that can be projected on a two-dimensional plane and then correct the magnification error in each image (Jing and Sachs, 1991; Penczek et al., 1995).

On the other hand, there is a useful aspect to measure the error that may occur in the actual experiment in the case of the deviation term added in Reference Equation 2.

However, this value can not be used in 2-dimensional tomography operation. In the case of a goniometer for tomography, this term is designed to minimize this deviation, so this term can be omitted. Also, the missing translation term in the reference equation 2 must be included in the normal tomographic image alignment process.

Accordingly, the present invention solves the above-mentioned problems of the reference expressions 1 and 2, and at the same time, suggests a method of improving the accuracy in image alignment.

As shown in FIG. 1, a method for improving the accuracy of image alignment in electronic tomography (S100) includes an EM projection image acquiring step, an image aligning step for aligning the acquired images, a 3D tomogram , And a visualization step. In the electronic tomography, the image alignment step is performed using the position reference indicator using the following equation (1).

[Formula 1]

는 i번째 표시자의 위치를 나타내며,

는 기울임 각,

는 기울임 축이 x-y 평면과 이루는 편위 (declination)이며,

는 z 방향의 투영행렬을 나타내고,

는 투영 평면상의 회전각,

는 j번째 이미지의 축척을 나타내며,

는 j번째 이미지에서 i번째 표시자의 2차원 위치를 나타내고,

는 j번째 이미지의 이차원 평행이동량을 나타낸다.
here,

Indicates the position of the i-th marker,

Lt; / RTI >

Is the declination that the tilt axis makes with the xy plane,

Represents a projection matrix in the z direction,

The rotation angle on the projection plane,

Represents the scale of the j-th image,

Represents the two-dimensional position of the i-th indicator in the j-th image,

Represents the two-dimensional translation of the jth image.

와 식 1의 이론모델 값

의 차이를 최소화시키는 미지변수인 축척

, 평면상에서의 회전각

, 기울임 각

, 표시자의 3차원 위치

, 평행이동량

를 구하는 문제로 귀결된다. 즉 아래의 RMS (root mean squares) 오차 L인 식 2를 최소화하는 해를 구해야 한다.Here, the image alignment step using equation (1)

And the theoretical model values of Equation 1

Scale, which is an unknown variable that minimizes the difference

, The rotational angle on the plane

, Tilt angle

, The three-dimensional position of the marker

, Parallel movement amount

. In other words, we need to find a solution that minimizes the root mean squares (RMS) error L expression 2 below.

[Formula 2]

Where N _p is the total number of selected marker positions to be used in least squares.

Equation 2 is a typical nonlinear inversion problem having a complex relationship between unknowns. In the above-mentioned prior studies, various methods have been used to solve the nonlinear least squares method, and the stability of convergence has been the biggest problem in each method.

See, for example, Luther et al. (1988) used various conditions to prevent all variables from affecting each other as much as possible without choosing to directly minimize L above.

The most important of these is that the markers used in the calculations must be present in all images. This condition makes the problem very simple and increases the stability of convergence.

However, in practice, the thicker the specimen or the greater the experimental noise, the more difficult it is to distinguish the indicator from the electron microscope image, and for this reason it is not easy to satisfy this condition except for the thin bio sample.

을 직접 최소화할 경우 각 이미지에서 구분이 가능한 표시자들 만으로 이루어진

을 만들 수 있어 위에서 언급한 Luther et al. (1988)의 제약이 없다.

을 최소화하기 위해 비선형 최소자승방법을 사용하여 풀어야 한다.In Equation 2,

If you minimize it directly, you will see only the indicators that can be distinguished from each image.

Luther et al. (1988).

The least squares method should be used to solve the problem.

Accordingly, the present invention uses the Levenberg-Marquardt (L-M) method, which is a nonlinear least squares method widely used in nonlinear inversion in various fields such as geophysical fields.

For reference, the LM method is a combination of the Gradient Decent method, which approaches convergence is slow but approaches the exact solution, and the Newton method, which has a fast convergence speed but a risk of divergence. It is an efficient method.

을 최소화하기 위해 L-M 방법을 사용하는데 있어 가장 문제가 되는 것은 변수들 사이의 의존성 (dependency)과 초기치 문제이다. In Equation 2,

The most problematic of using the LM method to minimize the dependency and initial value problems between variables.

In fact, since each parameter is closely related to each other in equation 1 above, there are very many very small values. In this case, selection of the initial value greatly affects the stability of convergence. Figure 1 schematically shows the effect of local minimum values in the minimization problem.

에서 가진다. Referring to FIG. 1, there are three minimum points,

.

를 초기치로 선택한 경우 참 최소값인

로 수렴한다.If a certain point is selected as an initial value in FIG. 1, the direction of convergence proceeds in the direction of decreasing the RMS value,

Is selected as the initial value,

.

혹은

로 선택한 경우 정확한 최소값인

로 수렴하지 못하고

나

같은 국부적인 극소치로 접근하게 된다. However,

or

If you select the correct minimum value

Not converging to

I

It is approached with the same local minimum.

That is, the selection of the initial value is very important for the complex nonlinear least squares method. In particular, if the number of unknown variables is large and the dependency between the variables is severe, convergence may not be achieved properly.

, 표시자의 총 종류를

, 전체 이미지에서 선택한 표시자 좌표의 총 수를

라 하자. 주어진 측정치는

개의

좌표로

개이며, 미지수는 각 이미지당 축척, 회전각, 기울임각, x-y 평행이동 등 변수 5개이므로

, 각 표시자의

좌표의 변수가

으로 총 미지수의 수는

이 된다. In the above equation, the total number of images

, The total type of indicator

, The total number of marker coordinates selected in the

Let's say. Given the measurements

doggy

By coordinates

, And the number of unknowns is 5 for each image such as scale, rotation angle, tilt angle, and xy translation.

, Of each marker

The variables of the coordinates are

The total number of unknowns is

.

For example, if there are 100 images and 30 indicator types, the number of unknown variables is about 590, which is a very large inversion problem.

That is, in Equation (1), the italic angle and various other variables are closely related to each other, so direct minimization of the entire unknown is highly unlikely to converge.

을 기 설정된 값으로 설정한 후, 나머지 변수들을 추정하는 역산단계(Main stage)(S120) 및 상기 기울임 각

을 도출하는 기울임 각 추정 단계(Tilt estimate)(S130)를 이용한다.
In order to solve such a problem, in the present invention,

A main stage (S120) for setting the predetermined value to a predetermined value and then estimating remaining variables,

A tilt estimate step (S130) is used.

In addition, the present invention further includes an initial value estimation step (Initial stage) to increase the computation efficiency of the inverse calculation step (S120).

Here, the initial value estimating step is performed before the inverse calculation step, and the inverse calculation is performed by minimizing the number of unknowns by reducing the number of variables with a high dependency.

Through the initial value estimation step, the number of local minimum values decreases, and a stable solution can be obtained.

Therefore, the values of the unknown variables obtained in the initial value estimation step are reused as initial values in the inversion of the unknown variable, thereby improving the stability and speed of convergence.

Hereinafter, an experimental example of a method for improving the accuracy of image alignment in the electronic tomography according to the present invention will be described.

● Examination materials

The image used in the experiment in the present invention is a TEM image of a geopolymer, which is to analyze the characteristics of pores on the geopolymer using the method of improving the accuracy of image alignment of the present invention.

 TEM observation was obtained from a Tecnai G2 Spirit (FEI, USA) with a 120 kV accelerating voltage and a magnification of 42,000 times and a tilt angle of -55 to 53 at a 1 degree interval with a total of 107 images of 2048 x 2048 pixels .

The indicator used here is a 10 nm diameter nano gold particle.

The image shown in Fig. 3 is an image at an oblique angle of 0 degrees, a white circle on the image indicates the position of the selected indicator, and the number next to it indicates the number of the indicator.

Referring to FIG. 3, a generally rounded white portion is represented by pores present in the geopolymer, and a black portion is affected by diffraction by nano gold particles and crystalline material, and most of the sample is amorphous.

For reference, in order to align the test data in this experiment, as the diffraction effect by the crystalline material shown in FIG. 3 is tilted, the change of light and shade is frequent and acts as a noise in the tomography result, This makes it difficult to sort the images.

Attempts were made to select locations in all TEM italic images for a total of 30 indicators, which were not selected where distinction was difficult.

As a result of selecting the markers as described above, finally, about 2900 positions were selected. In this case, the markers common to all the images are numbered 6, 7, 12, 19, 20, .

● Experimental results

1. Results of IMOD S / W

First, we tried to arrange images using IMOD S / W, which is widely used in electronic tomography. In IMOD S / W, it is assumed that only the indicators existing in the entire image can be used, assuming the theory of Luther (1998), and only 6 indicators (2, 6, 7, 12, 19, 206) Respectively.

FIG. 4 is a diagram showing the position and alignment process of the markers in the IMOD S / W. As a result of the alignment, the RMS error is 0.839, and the calculation time is about 8 seconds.

2. Using the Levenberg-Marquardt method

First, only six markers were selected as in the case of IMOD S / W, and the proposed LM method was applied. Since the total number of images is 107 and the type of indicator is 6, the total measurement value is 107 * 6 * 2 = 1284 and the total number of unknowns is 107 * 5 + 6 * 3 = 553. First, in case of the basic LM method, the information of the tilt angle is input directly to the unknown variable, and the number of unknowns is reduced to 533-107 = 426, and then the inversion is performed. (See Fig. 2).

The initial value determination process is performed before the inverse calculation step (S120). To this end, some variables are fixed in advance to reduce the number of unknowns.

First, the tilted angle 0 degree image assumes that the original sample is projected only without performing any transformation such as tilting, rotating, or translating, and then fixes the selected coordinate value in the image obtained at 0 degree in advance. Then, for each image, the magnification difference is very small, so all scales are fixed at 1.0.

Through these two processes, the number of total unknowns in the LM method used in the initial value decision process is significantly reduced. In this example, the number of total unknowns is reduced to 426-119 = 307 by 119. In this state, the value of the unknown variable is estimated by applying the LM method, and the general LM process is performed using the initial value.

Table 1 shows the results of image alignment using the L-M method.

As shown in Table 1, differences in the presence or absence of the initial value estimation step (S110) are compared.

The results show that the RMS error (L) is 0.653, which is better than the 0.839 of IMOD S / W, but the calculation time is slightly longer.

In the case of the improved LM method, the computation time is somewhat less than that of the general LM method, but the number of unknown variables is small. In the case of the LM method, the RMS error is smaller than that of the IMOD because of the algorithm characteristics and the convergence condition, and the result of the image alignment is almost the same.

Next, let's take a look at the results of inversion results for all selected points to verify the effectiveness of the inversion method.

We select all 30 types of marker and apply the LM method using all the coordinates of all 2900 selected markers in the whole image.

Table 2 is a table showing inversion results using the L-M method.

[Table 2]

As a result, the final RMS error was slightly higher than that of the six indicators shown in Table 1, and the calculation time was considerably high.

Also, compared with the conventional LM method, the LM method using the initial value estimation step (S110) was 2.5 times faster than the convergence speed.

3. Comparison of tomography results

We compare the image alignment results of the case where only 6 markers exist in all images and all 30 markers are used as in IMOD S / W.

One way to look at the accuracy of image alignment is to superimpose each of the sorted images and observe changes in prominent markers.

If alignment is successful, the movement of the markers on the overlapping image moves parallel to the x-axis (ie, maintains a constant y value).

However, it is difficult to use this method because the markers in the TEM images used in the test (Figure 3) are not clearly distinguished,

To solve this problem, we used a method of artificially adding strong contrasting points to the coordinates of selected markers in all images and then superimposing the aligned images.

FIG. 5 is an exemplary diagram showing the overlapping of the respective sorted images to compare the result using only six indicators and the result using all 30 indicators. FIG.

Referring to FIG. 5, the case of using 30 indicators of (b) is somewhat better than the case of using 6 indicators of (a), but it does not show a clear difference.

For a more detailed comparison, the coordinates of the points arranged in the upper part of FIG. 5 are plotted with symbols in FIG.

For convenience of comparison, the y-axis change is emphasized by about three times.

In FIG. 6, the change of the indicator 18, 28, 29 shows a good result when 30 indicators are used, and in the case of the indicator 20, it is better when only 6 indicators are used. This is presumably because the indicator 20 is a marker used directly when performing alignment with only six indicators.

To see the effect of these two image alignment results on tomography, we performed tomography using filtered back-projection (FBP) method on each aligned image. In order to compare the results, the x-y cross section was extracted from each final tomogram obtained from the tomography results and shown in Figs. 7 (a) and 7 (b).

7 (a) and 7 (b), the two cases show generally similar results, but focusing on the results inside the circle shown in the figure, the tomographic cross section using six markers ) (Fig. 7 (b)) shows a more distinct and clear image with the use of thirty markers.

In other words, a very precise image alignment is required to obtain a higher resolution tomography result, and the improved L-M method described above can efficiently obtain sophisticated image alignment.

Therefore, the present invention can provide an advantage that it can be very usefully used when it is difficult to discriminate an indicator, such as when a tomogram is applied to a thick sample or a noise-rich TEM image.

That is, the method of improving the accuracy of image alignment in the electronic tomography according to the present invention is advantageous in that the new model expression and minimization formula can use all the selected markers in an arbitrary image, and the Levenberg-Marquardt method can solve the minimization formula And the like.

In addition, the calculation time of the basic L-M method can be greatly improved through the initial value selection process.

In addition, it provides an advantage that it can be very usefully used when it is difficult to select all of the markers on the entire image because of thick samples or a lot of noise.

The previous description of the disclosed exemplary embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these exemplary embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Accordingly, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

S100: How to improve the accuracy of image alignment
S110: Initial value estimation step
S120:
S130: Tilt angle estimation step

Claims (5)

A method for improving image alignment accuracy using a position reference indicator,
EM projection image acquisition step;
An image sorting step of sorting the acquired images;
A reconstruction step of reconstructing the aligned image into a three-dimensional tomogram (image result); And
And a visualization step of visually displaying the reconstructed image,
Wherein in the image alignment step, the image alignment step is performed through Equation 1 below.
[Formula 1]

(

Indicates the position of the i-th marker,

Lt; / RTI >

Is the declination that the tilt axis makes with the xy plane, P _z is the projection matrix in the z direction,

Is the rotation angle on the projection plane, s ^(j) is the scale of the jth image,

Represents the two-dimensional position of the i-th indicator in the j-th image,

Is the two-dimensional translation of the j-th image)
The method according to claim 1,
The image aligning step using Equation (1)
The coordinates of the position reference marker in each image

Wow

Scale, which is an unknown variable that minimizes the difference

, The rotational angle on the plane

, Tilt angle

, A three-dimensional position of the position reference indicator

, Parallel movement amount

The root mean squares (RMS) error represented by Equation (2), which is the least squares equation described below, is minimized in order to calculate the image alignment accuracy.
[Formula 2]

(Where N _p is the total number of selected marker positions to be used in least squares)
3. The method of claim 2,
And a Levenberg-Marquardt (LM) method, which is a nonlinear least squares method, is applied in the calculation process for minimizing the L.
The method of claim 3,
Wherein,
The tilt angle

To a preset value, and then estimates remaining variables,

And a tilt angle estimation step of deriving a tilt angle estimation step.
delete

Images