JPH11127382A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To have a simple configuration and also to perform observation in real time by picking up an image that is acquired by an optical double focus system and recovering the entire image that is picked up by using a space frequency filter which ideally recovers an image on one focal plane of the optical double focus system. SOLUTION: An image input device 4 is provided with a space frequency filter that has characteristic M=(H1-H2)/H which ideally recovers the image of an object plane that is in focus when respective space frequency characteristics of an image formed only by light that is in focus, an image formed only by light that is out of focus and an image formed by both light on an object plane that is in focus are H1, H2 and H to the image on the object plane which is in focus about one light of one light that constitutes two focal points of a double focal lens 2 and is out of focus about the other light. By this, it is possible to acquire a clear image that is ideally recovered from the deterioration of double images due to the lens 2.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a signal processing apparatus for generating an observation image of a three-dimensional object having a height in an optical axis direction.

[0002]

2. Description of the Related Art Generally, in order to optically capture an image having excellent resolution and high brightness, an image forming optical system using an optical element having a large aperture is required. However,
Since the depth of focus of an imaging optical element represented by a lens is reduced as the aperture becomes larger, the depth of focus remains small. A biological specimen,
When observing a three-dimensional object having a height in the optical axis direction such as an LCD panel inspection, a blurred image is mixed with the observed image and the resolution is remarkably deteriorated, which hinders an inspection process based on the observed image. There was a fear.

Therefore, conventionally, it is conceivable to increase the depth of focus in order to make it possible to observe such an image of a three-dimensional object all at once. There is a type in which an image having a large depth of focus is reproduced without impairing resolution or brightness by adding different observation images and performing appropriate image processing on the obtained image.

That is, based on such a concept, as disclosed in Japanese Patent Application Laid-Open No. 8-317273, a sample image is sampled by a television camera while a stage is driven so that a sample is brought into focus. By taking an image, storing the sample image obtained by the television camera in a memory, further adding the sample image and the stored image at an addition timing with respect to the stored image, and restoring the added image, a plurality of images having different focus positions are obtained. , An image having a deep depth of focus obtained by adding the above-mentioned images, or JP-A-8-2940.
As disclosed in Japanese Patent Publication No. 35, a focus position detection based on contrast information is performed for a portion where the contrast at the time of focusing is high and a luminance change is drastic, and a focus position detection based on a luminance process is performed for a remaining portion. There is an image obtained by synthesizing images obtained from these to obtain a final focused image.

[0005]

However, all of these conventional devices take in a plurality of images having different focus positions and synthesize these images.
It is not possible to observe in real time, and because the sample stage or the optical part of the microscope is mechanically moved to capture multiple images, the structure of the structure for accurately driving them is complicated. A mechanical drive mechanism is required, which increases the size of the apparatus and increases the price. Further, direct movement of these stages or the optical system portion of the microscope also serves as a vibration source that vibrates the observation target. This means that in the case of a working microscope that allows the work to be performed on the observation target using a manipulator or tweezers, the observation target moves due to vibration and accidentally touches the manipulator or tweezers, and the work must be redone. For example, there is a possibility that work efficiency may be reduced, such as disappearance. The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide an image processing apparatus having a simple configuration and capable of real-time observation.

[0006]

According to the first aspect of the present invention,
A bifocal optical system, an imaging unit that captures an image obtained by the bifocal optical system, and an image of one of the focal planes of the bifocal optical system out of the images captured by the imaging unit. And an image processing means for restoring the entire image captured by the imaging means using the spatial frequency filter.

According to a second aspect of the present invention, in the first aspect, the spatial frequency filter is a bifocal optical system.
For an image on an object plane where one of the light rays that make up one focus is in focus and the other light ray is out of focus, the spatial frequency characteristic of the image with only the focused light is H1, When the spatial frequency characteristic of the image due to only the out-of-focus light beam is H2 and the spatial frequency characteristic of the image due to both light beams at the in-focus object plane is H (= H1 + H2), M = (H1 -H2) / H which ideally restores the image of the object surface
Has the following spatial frequency characteristics.

According to a third aspect of the present invention, in the first aspect, the bifocal optical system comprises at least one lens made of a birefringent crystal. As a result,
According to the present invention, the deterioration component of the double image can be removed from the image captured through the double focus optical system by the spatial frequency filter, and the depth of focus can be doubled while maintaining the resolution and working distance. Also, real-time observation becomes possible by recovering the deterioration by the direct recovery processing by the spatial frequency filter.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows a schematic configuration of an image processing apparatus to which the present invention is applied. In the figure, 1 is a microscope, and this microscope 1 has at least one birefringent crystal.
It has a bifocal lens 2 which is a bifocal optical system using two or more quartz crystals as a glass material, and a CCD camera 3 as an imaging means for imaging an image formed by the bifocal lens 2.

Here, the bifocal lens 2 has a focal depth d1 of a light beam which forms two focal points F1 and F2 as shown in FIG.
, D2 are designed to be continuous. Further, a lens using such a birefringent crystal as a glass material is described in Hisao Kikuta and Koichi Iwata, "Racing of a birefringent lens", Optical Technology Contact, Vol. 31. No5, p247-259 (199
As described in 3), the light is focused at two positions due to the difference in the polarization direction of the incident light with respect to the optical axis of the birefringent crystal.

The bifocal lens 2 is not limited to a lens using a birefringent crystal, but is formed of an aspherical lens or a diffractive lens as shown in FIG.
, D2 are continuous, and both images are obtained at the same time.

An image processing device 4 is connected to the CCD camera 3. The image processing device 4 performs a recovery process by spatial frequency filtering. One of the light beams constituting the two focal points F1 and F2 of the bifocal lens 2 is focused, and the other light beam is focused. The image (position (b) or (d) shown in FIG. 2) on the object plane where the image is not focused is compared with the image (ray 1 at the position (b) shown in FIG. 2) using only the focused light. The spatial frequency characteristic of the image or the image by the light ray 2 at the position (d) is H1, and the image by the unfocused light beam alone (the image of the light ray 2 at the position (b) shown in FIG. 2 or the light ray 1 at the position (d)). Assuming that the spatial frequency characteristic of the image due to light is H2 and the spatial frequency characteristic of the image by the light rays 1 and 2 on the focused object surface is H (= H1 + H2), Ideal image As a spatial frequency filter to recover, M = (H1 -H2) / H is set. Here, H1 is the characteristic of the image that is originally in focus, and H2 is the characteristic of the image that is out of focus. H1−H2 is the characteristic of the image that is out of focus. This shows the characteristics of an image close to the ideal image recovered from the deterioration of the double image due to the double focus lens 2 in which is drawn.

A monitor 5 is connected to the image processing device 4. The monitor 5 displays an image restored by the image processing device 4. Next, the operation of the embodiment configured as described above will be described.

Now, a sample of a three-dimensional object (not shown) is arranged in the range of the depth of focus d 1 and d 2 of the bifocal lens 2, and the sample is imaged by the CCD camera 3 through the bifocal lens 2. Then, an image picked up by the CCD camera 3 is input to the image processing device 4. The spatial frequency characteristic (MTF) of the image before input to the image processing device 4 is input.
Are given to the respective object planes at the positions (a) to (e) shown in FIG. 2 as shown in FIGS.

Here, the NA of light 1 and light 2 shown in FIG.
Are substantially equal, so that the spatial frequency characteristic shown in FIG. 3B and the spatial frequency characteristic shown in FIG. 3D are almost equal, and the spatial frequency characteristic shown in FIG.
The spatial frequency characteristics shown in FIG.

An image having such a spatial frequency characteristic is processed by an image processing device 4 having a spatial frequency characteristic of M = (H 1 −H 2) / H. In this case, since the position shown in FIG. 2B is dominant in the image formed by the bifocal lens 2, focusing on this position (b), this position (b) FIG. 4A shows the spatial frequency characteristic H1 of the image formed by the focused light beam 1 with respect to this object surface. The spatial frequency characteristic H2 of the image due to the unmatched ray 2 is
The result is as shown in FIG.

As a result, the spatial frequency characteristic of the object plane at the position (b) shown in FIG. H that is the sum of the spatial frequency characteristics H1 and H2 of
(= H1 + H2).

Then, an image having such a spatial frequency characteristic H is inputted to the image processing device 4 and M = M shown in FIG.
When the restoration processing is performed by the spatial frequency filter having the characteristic of (H1 -H2) / H, the characteristic of the processed image is as follows: H × M = (H1 + H2) × {(H1−H2) / H} = H
1 -H2.

As a result, H1 is the characteristic of the image that is originally in focus, and H2 is the characteristic of the image that is not in focus. , An out-of-focus image is drawn, and a clear image ideally recovered from the deterioration of the double image due to the double focus lens 2 is obtained. This image is displayed on the monitor 5.

When such a recovery processing by the spatial frequency filter is executed for each object plane at each of the positions (a) to (e) shown in FIG.
The characteristics as shown in (e) were obtained, and the images of the respective object surfaces were recovered in the direction of becoming clear.

By the way, the image picked up by the CCD camera 3 through the bifocal lens 2 using a sample obtained by mounting a three-dimensional object IC chip 7 on a planar substrate 6 as shown in FIG. Comparing the state before and after the restoration processing by the spatial frequency filter, before the restoration processing, the one in which the degraded component of the double image by the bifocal lens 2 remains as shown in FIG. Is
As shown in FIG. 8B, an ideally restored clear display image from which these degraded components of the double image were removed was obtained.

Therefore, according to such a configuration, an image picked up by the CCD camera 3 through the bifocal lens 2 which is a birefringent crystal is input to the image processing device 4, and the image processing device 4 Since the entire image captured by the CCD camera 3 is recovered using a spatial frequency filter that ideally recovers an image on one focal plane of the bifocal lens 2, the configuration can be simplified, and The degraded components of the double image can be removed from the image taken through the bifocal lens 2, and the NA of the objective lens, that is, the resolution and the working distance are maintained while maintaining the NA of the objective lens as compared with a conventional single focus microscope. The depth can be doubled, and a sharply restored image can be obtained even for a plurality of images having different focal positions.

Further, since the deterioration is recovered by the direct recovery processing using the spatial frequency filter without using means such as image switching, real-time observation becomes possible and efficient observation work can be realized. . Note that the characteristics H1 and H2 of the spatial frequency filter of the image processing device 4 with respect to the object plane at the position (b) shown in FIG.
Instead of

[0024]

(Equation 1)

[0025]

According to the present invention, the structure can be simplified, and
Degradation components of the double image can be removed from the image captured through the bifocal optical system. Compared with a conventional single-focus microscope, the NA of the objective lens, that is, the resolution and working distance are maintained while maintaining the focus. Because you can double the depth,
Even for a plurality of images having different focal positions, a sharply restored image can be obtained.

Further, since the deterioration is recovered by a direct recovery process using a spatial frequency filter without using means such as image switching, real-time observation becomes possible, and efficient observation work can be realized. .

[Brief description of the drawings]

FIG. 1 is a diagram showing a schematic configuration of an embodiment of the present invention.

FIG. 2 is a diagram illustrating one embodiment.

FIG. 3 is a diagram showing spatial frequency characteristics of a bifocal lens used in one embodiment.

FIG. 4 is a diagram illustrating spatial frequency characteristics of an image obtained by different light sources according to the embodiment;

FIG. 5 is a diagram showing spatial frequency characteristics of a spatial frequency filter used in one embodiment.

FIG. 6 is a diagram showing spatial frequency characteristics after a recovery process according to the embodiment;

FIG. 7 illustrates a specific example of a sample of one embodiment.

FIG. 8 is a diagram showing an example of image display before and after a recovery process for a specific sample according to the embodiment;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Microscope, 2 ... Bifocal lens, 3 ... CCD camera, 4 ... Image processing apparatus, 5 ... Monitor, 6 ... Substrate, 7 ... IC chip.

Continued on the front page (51) Int.Cl. ⁶ Identification symbol FI // G02B 3/10 G06F 15/68 400A

Claims (3)

[Claims] 1. A bifocal optical system, an imaging unit for imaging an image obtained by the bifocal optical system, and one of the focal planes of the bifocal optical system among images captured by the imaging unit An image processing device, comprising: a spatial frequency filter that ideally performs a restoration process on the image of (a), and an image processing unit that restores the entire image captured by the imaging unit using the spatial frequency filter. . 2. A spatial frequency filter is used for an image on an object plane where one of the two light beams constituting the focal point of the bifocal optical system is focused and the other light beam is not focused. H1 is the spatial frequency characteristic of the image due to only the focused light beam, H2 is the spatial frequency characteristic of the image due to the unfocused light beam, and H2 is the spatial frequency characteristic of the image due to both light beams at the focused object plane. Assuming that the frequency characteristic is H (= H1 + H2), M = (H1 -H) that ideally restores the focused image of the object surface.
2) The image processing apparatus according to claim 1, wherein the image processing apparatus has a spatial frequency characteristic of / H. 3. The image processing apparatus according to claim 1, wherein the bifocal optical system comprises at least one lens made of a birefringent crystal.