KR100839871B1 - Active vision system for variable view imaging - Google Patents

Abstract

An active optical system for a variable view field and a deformable mirror surface shape control method for the same are provided to obtain various visual information by changing the direction of the visual angle, the position of a view field, and the position of a focal plane according to situations. An active optical system for a variable view field is composed of two wedge prisms(11,12) positioned on the same axis and independently rotated to decide the direction of the sight in a cone, a scanning mirror(13) fixing the direction of the sight changed by two wedge prisms, to the surface of an object(OB), and an adaptive optical system correcting aberrations generated by two wedge prisms and the scanning mirror. In the adaptive optical system, a deformable mirror(14) corrects aberrations by changing the shape of the surface and reflects the sight reflected by the scanning mirror, to a camera(C). A first lens(15) is positioned between the scanning mirror and the deformable mirror. A second lens(16) is interposed between the deformable mirror and the camera. A third lens(17) is positioned between the second lens and the camera to regulate an effective diameter of the deformable mirror by using an iris(19). A fourth lens(18) forms an image passing through the third lens, on the camera.

Description

Variable Vision Active Optics and Deformation Mirror Surface Shape Control Method {Active Vision System for Variable View Imaging}

1 is a block diagram of a variable view active optical system according to an embodiment of the present invention;

2 is a view defining a rotation angle of one wedge prism,

3 is a view defining an inclination angle and a bias angle of one wedge prism;

FIG. 4 shows the relationship between the rotation angles θ _r1 , θ _r2 and the pan angle θ _p and the tilt angle θ _t of the two wedge prisms with the two wedge frames positioned on the same axis. drawing,

FIG. 5A illustrates a process of adjusting a gaze direction using two wedge prisms, and FIG. 5B illustrates a process of adjusting a position of a field of view using a scanning mirror;

6 is a structural diagram of a deformation mirror according to the present invention,

7 is an operation flowchart illustrating a method for correcting aberration for one Zernike coefficient according to the present invention;

8 is a configuration diagram of a variable view active optical system according to the present invention finally designed;

9 is a diagram showing a configuration example of two wedge prisms;

10 is a table summarizing the maximum value and the minimum value error of the wavefront before correcting the wavefront aberration, and the maximum value and the minimum value error of the wavefront after correcting the wavefront aberration using the strain mirror.

11 shows an example of the surface shape of a deformation mirror;

12 is a diagram illustrating an image before and after aberration correction acquired while changing a moving distance δZ;

13 and 14 illustrate an image obtained before correcting aberration, an image obtained after correcting aberration, in the configuration example of the two wedge prisms shown in FIG. 9;

15 shows the variable line of sight capability of the system according to the invention.

     <Brief description of symbols for the main parts of the drawings>

11, 12: Wedge Prism 13: Scanning Mirror

14 deformed mirror 15 first lens

16: second lens 17: third lens

18: fourth lens 19: aperture

OB: Object C: Camera

The present invention relates to a variable field of view active optical system, and more particularly, to a field of view variable optical field of view that enables the micro-objects at any position in the operating area to be viewed in a desired direction. The invention also relates to a method of controlling the surface shape of a deformation mirror for implementing a variable field of view active optical system.

Recent advances in MEMS technology and biotechnology have led to the need for automated microassembly and micromanipulation, with faster throughput and higher yield dictating competitiveness. In order to meet this need, microassembly and micromanipulation using visual sensors have recently been attempted.

However, there is a lack of three-dimensional visual information of the micro-objects observed by the system, which makes it difficult to finely assemble and manipulate the micro-objects. Here, occlusion means that an important part of an object is covered by another object, and low resolution means that the visual sensor cannot measure the position and posture of the object.

As described above, since there is a lack of three-dimensional visual information in the related art, there is a problem in that fine assembly and fine manipulation cannot be performed on the visual sensor.

In the macro world, however, research on an active visual system to solve the lack of visual information is already in progress. Active vision systems in this macro world adapt to their surroundings and adjust camera parameters such as position, posture, focal length, and zoom to accomplish a given task.

However, it is difficult to implement such an active visual system in the micro world due to some fundamental problems of the optical microscope, such as the complexity of the micro world's optical microscope, small angle of view, and shallow depth of focus.

In an attempt to develop an active vision system in the optical microscope of the microworld, the following papers have been proposed.

X. Tao, H.S. In Cho, and Y. Cho's paper "Microassembly of peg and hole using active zooming, SPIE International Symposium on Optomechatronic Technologies (ISOT 2005), Sapporo, Japan, 2005", a wide field of view in the initial state and high resolution in the later state An active zooming system for providing is disclosed.

Paper by B. Potsaid, Y. Bellouard, and J. Wen "Adaptive Scanning Optical Microscope (ASOM): A multidisciplinary optical microscope design for large field of view and high resolution imaging, Opt. Express 13, 6504-6518, 2005" Discloses an Active Scanning Optical Microscope. With this active scanning optical microscope, you can not only capture images in a narrow area and synthesize images with wide angle of view, but also track moving objects with high resolution.

However, all of these systems have limitations in technology that cannot change the direction of the gaze to overcome blindness or improve resolution.

An object of the present invention, which is devised to solve the problems of the prior art, is to provide a variable field of view active optical system that provides a variable field of view without moving the specimen, overcomes obstruction, and corrects aberration to improve resolution. .

Another object of the present invention is to provide a method for controlling the surface shape of a deformation mirror for implementing a variable field of view active optical system.

The variable field active optical system according to the present invention for achieving the above object comprises two wedge prisms positioned on the same axis and independently rotated to determine the direction of the eye within the cone;

And a scanning mirror that fixes the eye direction changed by the two wedge prisms to the surface of the object.

In addition, the method of controlling the shape of the deformed mirror surface according to the present invention includes two wedge prisms positioned on the same axis and independently rotating to determine a gaze direction in a cone, and a gaze direction changed by the two wedge prisms to determine the direction of the object. In a variable field active optical system including a scanning mirror fixed to a surface, and a deformation mirror for correcting aberrations generated by the two wedge prisms and the scanning mirror, the method of controlling the surface shape of the deformation mirror,

Measuring wavefront aberration generated by the wedge prism and the scanning mirror, calculating a control signal for modifying the surface shape of the deformation mirror according to the wavefront aberration, and outputting the calculated control signal to the deformation mirror The shape of the surface of the deformation mirror is the same as that of the wavefront aberration, and the amplitude is modified so that the amplitude is half the amplitude of the wavefront aberration.

Hereinafter, with reference to the accompanying drawings will be described in detail an embodiment of a variable field of view active optical system and a method for controlling the shape of the deformation mirror surface according to the present invention.

1 is a block diagram of a variable view active optical system according to an exemplary embodiment of the present invention.

The variable field of view active optical system of the present invention varies the line of sight and the field of view to provide a camera with various views of the surface of the micro-object without movement of the micro-object.

This variable field of view active optical system has two wedge prisms 11 and 12 positioned on the same axis and independently rotated to determine a viewing direction in a cone, and the two wedge prisms 11 and 12 change the viewing direction. And a scanning mirror 13 which secures the field of view to the surface of the object. Since the two wedge prisms 11 and 12 and the scanning mirror 13 allow the object OB to be viewed from different eye directions, the two wedge prisms 11 and 12 and the scanning mirror 13 The combination of can provide a variable field of view without moving the fine object.

However, since an aberration occurs in the optical system consisting of these two wedge prisms 11 and 12 and the scanning mirror 13, the present invention further includes an adaptive optical system for correcting this aberration.

This adaptive optical system is provided between the deformation mirror 14 and the scanning mirror 13 and the deformation mirror 14, which are deformed surface shape and reflect the reflected angle through the scanning mirror 13 toward the camera C. The first lens 15, the second mirror 16 positioned between the modified mirror 14 and the camera C, and the second modified mirror 14 located between the second lens 16 and the camera C 14. The third lens 17 to adjust the effective diameter of the aperture using the aperture 19, and the fourth lens 18 to form an image passing through the third lens 17 to the camera (C). An image plane is formed between the second lens 16 and the third lens 17.

The following describes each component in detail.

2 is a diagram defining the rotation angle of one wedge prism. Θ _ri in the figure is the angle of the wedge prism. Here, i is an index of the prism, the first wedge prism denotes i as 1, and the second wedge prism denotes i as 2.

3 is a view defining the inclination angle and the bias angle of one wedge prism. Θ _w is the angle of inclination and θ _d is the angle of deviation in the figure. The relationship between the inclination angle and the bias angle is shown in Equation 1 below.

Where n is the refractive index of the wedge prism.

FIG. 4 shows the relationship between the rotation angles θ _r1 , θ _r2 and the pan angle θ _p and the tilt angle θ _t of the two wedge prisms with the two wedge frames positioned on the same axis. Drawing. If two wedge frames are placed on the same axis and each rotates independently, the light beam rotates in any direction within the cone. The relationship between the rotation angles θ _r1 and θ _r2 of the two wedge frames, the fan angle θ _p and the tilt angle θ _t of the light beam is expressed by Equation 2 below.

Relations to thick prisms can be used for preliminary design, and in actual design, more accurate results can be obtained by using a lens design program such as ZEMAX.

FIG. 5A illustrates a process of adjusting the gaze direction using two wedge prisms, and FIG. 5B illustrates a process of adjusting a position of a field of view using a scanning mirror.

As shown in FIG. 5A, when the two wedge prisms are rotated to adjust the line of sight, the field of view is also changed along with the line of sight, and a scanning mirror is used to fix the field of view.

6 is a structural diagram of a deformation mirror according to the present invention.

When two wedge prisms are used for converging rays, aberrations occur, in which the aberration mirror is used to correct the aberrations. That is, the aberration is corrected by modifying the surface shape of the deformation mirror so that its shape is equal to the wavefront aberration and its amplitude is half the amplitude of the wavefront aberration so that the aberration disappears after the wavefront is reflected from the deformation mirror.

At this time, the surface shape of the deformation mirror can be expressed using the Zernike polynomial, and the change in the surface shape of the deformation mirror with respect to the control signal of the mirror driver can be expressed using an influence function as shown in Equation (3).

Where s is the column vector (n × 1) of the Zernike coefficients for the mirror shape, and x is the column vector (m × 1) of the control signal for the mirror driver. A is an n × m influence matrix, where element a _ij is the slope of the linear function relation between the i th Zernike coefficient s _i and the control signal x _j for the j th channel of the mirror driver. This slope is determined by experiment.

By modifying Equation 3, the control signal of the mirror driver can be calculated when the desired shape of the mirror surface is given, which is expressed by Equation 4 below.

Here, A ⁺ is a pseudo inverse of the influence function matrix, which can be obtained using the singular value decomposition method.

Therefore, when the wavefront aberration information of the optical system is obtained, the surface shape information of the deformation mirror to be deformed can be obtained using the wavefront aberration information. Using the surface shape information of the deformed mirror, the control signal of the mirror driver can be calculated. When the control signal of the mirror driver is calculated in this way, the mirror driver outputs the control signal to the deformation mirror to deform the surface shape of the deformation mirror.

That is, in order to correct the wave front aberration using the deformation mirror, the wave front aberration of the optical system should be measured first.

As a method of measuring the wave front aberration of the optical system, a method using a Hartmann or a Schon-Heartmann sensor is mainly used. However, these sensors require a stable point light source, whereas most of the micro-objects cannot provide sufficient light source because they do not have enough reflectance, so using these sensors to accurately measure the wavefront of the micro-objects is very It is difficult.

The present invention proposes a method for measuring wavefront aberration using a focus scale obtained from an image. Focus A frequency selective weighted median (FSWM) filter algorithm is used to find the focus scale.

By the orthogonality of the Zernike polynomial, the surface shape of the deformation mirror is controlled using the Zernike mode instead of controlling the individual mirror drivers.

7 is an operation flowchart showing a method for correcting aberration for one Zernike coefficient according to the present invention.

를 초기화하고(S71), 아래의 수학식 5를 이용하여

를 계산한다(S72).First, the step size of the m iteration for the i th Zernike coefficient

To initialize (S71), using Equation 5 below

Calculate (S72).

Where A ⁺ is the pseudo inverse of the influence matrix.

와

를 거울 구동기에 가하고(S73), 그에 따른 영상으로부터 초점척도

와

를 계산한다(S74).Next, a control signal defined as in Equation 6

Wow

Is applied to the mirror driver (S73), the focus scale from the image accordingly

Wow

Calculate (S74).

,

,

Next, the control signal is updated as shown in Equation (7).

는 제어신호

가 변형 거울에 가해졌을 때의 초점척도를 의미한다.μ is a constant,

Is the control signal

The scale of focus when is applied to the deformation mirror.

또는

이면(S76) 스텝 사이즈

를 수학식 8과 같이 줄인다(S77).next,

or

Back side (S76) step size

Reduce as shown in Equation (8) (S77).

Where ω is the reduction factor.

Next, if the difference between I ^m and I ^{m -1} is greater than the threshold value I _c (S78), the process returns to step S72 and repeats the above process, and the difference between I ^m and I ^{m -1} is the threshold value ( If it is not larger than I _c ) (S78), the process ends.

The process of FIG. 7 is performed for all Zernike coefficients to correct aberrations for all Zernike coefficients.

The system design process of the present invention will be described. First, the active visual system to be designed is divided into an imaging system and a steering system, and the layout and elements of the system are determined according to the design specification. In the present invention, a post objective scanning system is used without designing a scanning lens separately. Post-objective scanning has a problem of causing top surface curvature, but the top surface curvature caused by this post objective scanning can be corrected by a deformation mirror. Next, determine the position of each optical element using the ZEMAX optimization tool. This reduces the aberration to within 3.5λ at wavelength λ = 0.660μm.

8 is a schematic diagram of a variable view active optical system according to the present invention finally designed. The lens of the present invention uses achromatic aberration standard lens, and the bias angle obtained by using the wedge prism is 10 degrees. In this case, the maximum deviation angle obtained from the two wedge prisms is 20 degrees. Increasing this bias allows a wider range of field of view. In simulation experiments, the surface type of the deformation mirror is chosen as the Zernike standard phase. In order to find the optimum shape, the mirror shape such as the aberration coefficient is set as an initial value and optimization is performed. In this way, the time required for the optimization process can be reduced.

FIG. 9 is a diagram illustrating a configuration example of two wedge prisms, in which the rotation angles θ _r1 and θ _{r2 of} the first wedge prism and the second wedge prism coincide with each other, and may be 180 degrees different. The observation point and direction of the object vary according to the difference in the rotation angles of the two wedge prisms, and the observation point and direction of the object also vary according to the rotation angle θ _s of the scanning mirror at that time.

FIG. 10 is a table summarizing the maximum value and the minimum value error of the wavefront before correcting the wavefront aberration, and the maximum value and the minimum value error of the wavefront after the wavefront aberration is corrected using the strain mirror. Referring to FIG. 10, the aberration correction capability of the deformation mirror may be confirmed.

11 is a view showing an example of the surface shape of the deformation mirror, it can be seen that the deformation mirror is deformed into various shapes according to the rotation angle of the two wedge prism and the rotation angle of the scanning mirror.

The deformable mirrors of this system have the ability to dynamically focus moving objects. To test the dynamic focusing ability of the deformation mirror, an aberration correction algorithm is applied while moving the object by δZ from the focal plane.

FIG. 12 is a diagram illustrating an image before and after aberration correction acquired while changing the moving distance δZ. As the moving distance δZ increases, the time required for aberration correction increases, and the value of the final focus scale decreases. If the moving distance δZ is 8 mm or more, the aberration is not corrected because it is out of the aberration correction range of the deformation mirror.

When the optical system is constructed using the wedge prism, other aberrations such as coma and astigmatism occur. These aberrations are also corrected by the deformation mirror. 13 and 14 show images obtained before correcting aberration and images obtained after correcting aberration, respectively, in the six configuration examples shown in FIG. 9 to confirm the correction capability by the deformation mirror. 13 and 14, it can be seen that the image quality is improved by using the deformation mirror.

Figure 15 shows the variable line of sight capability of the system according to the present invention, where (a) only observes the front of the chip, while (b) and (c) modify the wedge prism and scanning mirror to It can be seen that the bottom side can also be observed.

Although the technical spirit of the present invention has been described above with the accompanying drawings, it is intended to exemplarily describe the best embodiment of the present invention, but not to limit the present invention. In addition, it is obvious that any person skilled in the art may make various modifications and imitations without departing from the scope of the technical idea of the present invention.

As described above, according to the present invention, various visual information may be provided by changing the visual direction, the visual field position, the position of the focal plane, etc. according to the situation. This invention can be used to observe moving three-dimensional micro-objects, biological observation, surface inspection, circuit board inspection, confocal microscope and the like.

Claims (9)

delete delete Two wedge prisms positioned on the same axis and independently rotating to determine a gaze direction in the cone, a scanning mirror for fixing the gaze direction changed by the two wedge prisms to the surface of the object, and the two wedge prisms; An adaptive optics for correcting aberrations generated by the scanning mirror, The adaptive optical system, A deformation mirror that deforms a surface shape to correct the aberration and reflects a view reflected through the scanning mirror toward the camera; A first lens positioned between the scanning mirror and the deformation mirror; A second lens positioned between the deformation mirror and the camera; A third lens positioned between the second lens and the camera to adjust an effective diameter of the deformable mirror using an aperture; And a fourth lens, which forms an image passing through the third lens on the camera. 4. The apparatus of claim 3, further comprising mirror driving means for obtaining a wave front aberration of the optical system and modifying the surface shape of the deformable mirror to have the same shape as the wave front aberration, but the amplitude of which is half the amplitude of the wave front aberration. A variable field of view active optical system. 5. The variable field active optical system of claim 4, wherein the wavefront aberration is calculated using a focus measure of an image obtained by the optical system. Two wedge prisms positioned on the same axis and independently rotating to determine a gaze direction in the cone, a scanning mirror for fixing the gaze direction changed by the two wedge prisms to the surface of the object, and the two wedge prisms; In a variable field of view active optical system including a distortion mirror for correcting the aberration generated by the scanning mirror, in the method of controlling the surface shape of the deformation mirror, Measure the wave front aberration generated by the wedge prism and the scanning mirror, Calculating a control signal for modifying the surface shape of the deformation mirror according to the wavefront aberration, Outputting the calculated control signal to the deformation mirror, And deforming the surface shape of the deformable mirror so as to have the same shape as the wavefront aberration, the amplitude of which is half the amplitude of the wavefront aberration. 7. The method of claim 6, wherein the wavefront aberration is measured using a focal scale of an image obtained from the optical system. 8. The method of claim 7, wherein the focus scale is obtained using a frequency selective weighted median filter algorithm. The aberration correction procedure is repeated for all Zernike coefficients. The aberration correction procedure for any i th Zernike coefficient A first step of initializing the m-th iteration step size for the i-th Zernike coefficient and calculating a control signal variation using the pseudo inverse of the iteration step size and the influence function matrix; a second step of deforming the surface shape of the deformation mirror by calculating the control signal of the m-th iteration process by using the control signal obtained in the m-1 iteration process and the control signal variation obtained in the first step; A third step of calculating a focus scale from an image acquired while a control signal of an mth iteration process is applied to the deformation mirror; A fourth step of updating a control signal of an mth iteration process using the focus scale; A fifth step of reducing the m-th iteration step size according to the value of the focal scale; If the difference between the focal scale obtained in the m-th iteration process and the focal scale obtained in the m-th iteration process is larger than the threshold value, the second to fifth steps are repeated, and the difference value is greater than the threshold value. And a sixth step of terminating if not.

Images