KR20200089421A - Digital microscope and operating method thereof - Google Patents

Abstract

The present invention relates to a digital microscope and an operation method thereof, which can reduce manufacturing cost, are advantageous for miniaturization, provide high-resolution images, and can be used in various fields. The digital microscope according to the present invention comprises: an objective lens disposed on one side of a sample; an image sensor disposed on one side of the objective lens and acquiring an image of the sample magnified through the objective lens; a variable means for varying the distance between the sample and the objective lens; and an image processing unit which receives, from the image sensor, a first image obtained when the distance between the objective lens and the sample is a first distance and a second image acquired when the distance is a second distance, and generates a fused image by fusing the first image and the second image.

Description

Digital microscope and its operating method

The present invention relates to a microscope, and more particularly, to a digital microscope equipped with an image sensor and a method for operating the same.

In general, a microscope is used to greatly observe an object to be observed, and is widely used throughout industries such as biological fields, medical fields, scientific fields, learning teaching materials, and semiconductor inspection equipment.

Due to its structure, a conventional optical microscope has a disadvantage in that it is difficult for several people to simultaneously view the object to be observed, and it is difficult to record and store the object in the form of an image.

In order to improve the disadvantages of optical microscopes, digital microscopes equipped with image sensors have been developed. The digital microscope is bulky and complex due to the addition of various accessories to increase the magnification and increase the image quality in order to meet its use, so the manufacturing cost is high and the constraints of the use environment are large. There was a problem that the usability was greatly reduced due to limitations.

The technical problem to be achieved by the present invention is to provide a digital microscope and a method for operating the same, which can reduce manufacturing cost, are advantageous for miniaturization, can provide a high resolution image, and can be used in various fields.

The problem to be solved of the present invention is not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The digital microscope according to the present invention for solving the above technical problem, an objective lens disposed on one side of the sample; An image sensor disposed on one side of the objective lens and acquiring an image of the sample magnified through the objective lens; Variable means for varying the distance between the sample and the objective lens; And a first image obtained when the distance between the objective lens and the sample is a first interval, and a second image obtained when the interval is a second interval, and the first image and the first It characterized in that it comprises an image processing unit for generating a fused image by fusing two images.

The image processor may fuse the first image and the second image in the frequency domain.

The first image may be an image focused at the center of the image sensor, and the second image may be an image focused outside the center of the image sensor. Here, the second interval may be smaller than the first interval.

The image processor acquires first and second frequency domain images through DWT (Discrete Wavelet Transform) from the first and second images, respectively, and fuses the first and second frequency domain images to fuse the frequency domain images. And obtaining the fusion image from the frequency domain fusion image through IDWT (Inverse Discrete Wavelet Transform).

The image processing unit acquires first pyramid level images and second pyramid level images through pyramid expansion from the first and second frequency domain images, respectively, and the first pyramid level images and the first Two pyramid level images can be fused for each level to obtain fusion pyramid level images, and the frequency domain fusion image can be obtained from the fusion pyramid level images through pyramid compression.

The digital microscope may include a light source irradiating light to the sample; And a focusing lens for focusing light from the light source.

In the operation method of a digital microscope according to the present invention for solving the above technical problem, the digital microscope is disposed on one side of the sample, and is disposed on one side of the objective lens, and magnified through the objective lens And an image sensor for acquiring an image of the sample, the operation method comprising: acquiring a first image of the sample with the image sensor when an interval between the objective lens and the sample is a first interval; Acquiring a second image of the sample with the image sensor when the distance between the objective lens and the sample is a second interval; And generating a fused image by fusing the first image and the second image.

In the generating of the fused image, the first image and the second image may be fused in a frequency domain.

The first image may be an image focused at the center of the image sensor, and the second image may be an image focused outside the center of the image sensor. Here, the second interval may be smaller than the first interval.

The generating of the fused image may include obtaining first and second frequency domain images from the first and second images through DWT (Discrete Wavelet Transform), respectively; Fusing the first and second frequency domain images to obtain a frequency domain fused image; And obtaining the fusion image from the frequency domain fusion image through an Inverse Discrete Wavelet Transform (IDWT).

The obtaining of the frequency domain fusion image may include: obtaining first pyramid level images and second pyramid level images from the first and second frequency domain images through pyramid expansion, respectively; Fusing the first pyramid level images and the second pyramid level images for each level to obtain fusion pyramid level images; And acquiring the frequency domain fusion image through pyramid compression from the fusion pyramid level images.

The digital microscope may further include a light source that irradiates light to the sample and a focusing lens that focuses light from the light source.

The digital microscope according to the present invention and its operation method have advantages such as reduced manufacturing cost, advantageous for miniaturization, high resolution image, and various fields.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 shows a schematic configuration of a digital microscope according to an embodiment of the present invention.
2 is a flowchart illustrating a method of operating a digital microscope according to an embodiment of the present invention.
FIG. 3 shows that the first image and the second image are obtained while the distance between the objective lens 20 and the sample S is set to the first interval and the second interval, respectively.
FIG. 4 specifically illustrates the process of generating a fused image by fusing step 230 of FIG. 2, that is, a first image focused on the center of the image sensor 30 and a second image focused on the outside of the center of the image sensor 30. This is a flow chart.
FIG. 5 schematically shows a process in which a fused image is generated from a first image and a second image according to the flowchart of FIG. 4.
FIG. 6 is a diagram illustrating a partial enlarged comparison of a first image, a second image, and a fused image obtained according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the following description and the accompanying drawings, elements that are substantially the same are denoted by the same reference numerals, and redundant description will be omitted. In addition, in the description of the present invention, when it is determined that detailed descriptions of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, detailed descriptions thereof will be omitted.

1 shows a schematic configuration of a digital microscope according to an embodiment of the present invention.

The digital microscope according to the present embodiment is arranged on one side of the stage 10 for placing the sample S and the objective lens 20 and the objective lens 20 which are arranged on one side of the sample S to enlarge the image. The image sensor 30 that acquires the image of the sample S magnified through the objective lens 20 is disposed on the other side of the sample S, and a light source irradiating light to the sample S (for example, an LED light source) ) 40, a focusing lens 50 that focuses light from the light source 40 to the sample S, and an image processing unit 60 that processes an image input from the image sensor 30.

Referring to FIG. 1, light emitted from the light source 40 is focused on the sample S by the focusing lens 50, light passing through the sample S enters the objective lens 20, and the objective lens ( 20) Light passing through is sensed by the image sensor 30. Accordingly, the image sensor 30 acquires an image of the sample S magnified through the objective lens 20.

In this embodiment, the light source 40 and the focusing lens 50 are arranged on the same axis as the sample S, the objective lens 20 and the image sensor 30 so that the sample S faces the objective lens 20. However, the structure may be a structure in which a light splitter is disposed between the objective lens 20 and the image sensor 30 and a light source 40 and a focusing lens 50 are disposed on the side. According to this structure, the light emitted from the light source 40 is irradiated to the sample S through the objective lens 20 by changing the traveling path by the optical splitter, and the light reflected from the sample S is the objective lens 20 Light incident through the objective lens 20 is sensed by the image sensor 30. In this structure, the stage 10 can be omitted.

One of the features of the present invention is that, in the case of a conventional microscope optical system, a tube or tube lens that should be provided between the objective lens and the image sensor is omitted. In the present invention, the objective lens 20 and the image sensor 30 can be disposed close by omitting the tube or tube lens. As the objective lens 20 and the image sensor 30 are disposed close to each other without a tube or tube lens, light attenuation is reduced due to the reduction in optical components, the structure is simple, and the size of the microscope is reduced. In addition, since the image sensor 30 is close to the objective lens 20, it is possible to use a high-quality image sensor having a small pixel size, reduce the size of the image sensor, and provide a wider field of view. Can. In addition, the microscope according to the present invention can use a relatively inexpensive commercial finite type objective lens. Due to the reduction of optical components and the use of inexpensive objective lenses, the manufacturing cost of the microscope can be significantly reduced.

The focal plane in the image sensor 30 of the light passing through the objective lens 20 is flat because the objective lens 20 and the image sensor 30 are disposed close by omitting the tube or tube lens. No, it becomes closer to the sphere. This results in non-uniform focusing of the image within a given field of view (FOV), so that the focused part is clear, but the off-focus part is less sharp.

Another of the features of the present invention is to obtain a clear image as a whole even though the focal plane of the light passing through the objective lens 20 is not a plane, by adjusting the distance between the objective lens 20 and the sample S It is to obtain two or more images focused on different parts of the sensor 30, and fuse the images to obtain a final image as a whole. Hereinafter, an additional configuration and operation method of a digital microscope according to an embodiment of the present invention will be described.

The digital microscope according to an embodiment of the present invention includes variable means (not shown) for varying the distance between the sample S and the objective lens 20. The variable means is implemented in such a way that the stage 10 is moved in the axial direction while the objective lens 20 and the image sensor 30 are fixed, or the sample S or the stage 10 is fixed. It may be implemented in such a way that the objective lens 20 and the image sensor 30 are moved in the axial direction. The variable means may be implemented to operate manually, or may be implemented to operate automatically.

2 is a flowchart illustrating a method of operating a digital microscope according to an embodiment of the present invention.

The distance between the objective lens 20 and the sample S is set to a first interval, and a first image of the sample S is acquired by the image sensor 30 (step 210), and the objective lens 20 and the sample S ) Is set as the second interval, and the first image of the sample S is acquired by the image sensor 30 (step 220). The first and second images acquired by the image sensor 30 are input to the image processing unit 60.

FIG. 3 shows, for example, when the variable means is implemented by moving the stage 10, the distance between the objective lens 20 and the sample S is set to the first interval and the second interval, respectively, and the first image and It shows that a 2nd image is obtained.

Referring to the left setting of FIG. 3, between the objective lens 20 and the sample S, the focal plane fp1 of light passing through the objective lens 20 passes through the center O of the image sensor plane 30 ′. The interval is set to the first interval d1, so that the first image focused on the center O of the image sensor 30 is obtained. Therefore, in the first image, the center O portion is clear, but the center outer portion becomes a less sharp image.

Referring to the right setting of FIG. 3, the objective lens 20 so that the focal plane fp2 of light passing through the objective lens 20 passes an outer point away from the center O of the image sensor plane 30 ′ by R. ) And the sample S are set to a second distance d2, so that a second image focused on the outside R of the center of the image sensor 30 is obtained. Here, the second interval d2 is smaller than the first interval d1. Therefore, in the second image, the center outer (R) portion is clear, but the center (O) portion is a less sharp image.

Referring back to FIG. 2, the image processing unit 60 fuses a first image focused at the center of the image sensor 30 and a second image focused at the center of the image sensor 30 in the frequency domain as described above, Both the center and the center of the center generate a clear fusion image (step 230). The fused image is displayed through a display device (not shown) connected to the image processing unit 60. If necessary (for example, to focus on a desired point at the center or outside), the first image and the second image are displayed (not shown) in the process of adjusting the distance between the sample S and the objective lens 20. ).

In this embodiment, two images, that is, a first image focused at the center of the image sensor 30 and a second image focused at a point outside the center of the image sensor 30 are acquired, and the two images are fused. As described above, three or more images may be acquired and fused. For example, the first image focused on the center of the image sensor 30, the second image focused on the first point outside the center of the image sensor 30, and the second image focused on the second point outside the center of the image sensor 30 A third image may be obtained, and the first to third images may be fused to obtain a fused image.

FIG. 4 specifically illustrates a process of generating a fused image by fusion of step 230 of FIG. 2, that is, a first image focused on the center of the image sensor 30 and a second image focused on the outside of the center of the image sensor 30. 5 is a schematic diagram illustrating a process in which a fused image is generated from a first image and a second image according to the flowchart of FIG. 4.

The image processing unit 60 acquires the first frequency domain image 530 through the discrete wavelet transform (DWT) from the first image 510 focused at the center (step 231), and focuses outside the center The second frequency domain image 540 is acquired through the DWT from the second image 520 (step 232). Wavelet transform is effective for representing isolated sharp objects such as microscale structures.

Next, the image processor 60 acquires a plurality of first pyramid level images 550 from the first frequency domain image 530 through pyramid expansion (step 233), and the second frequency domain image From 540, a plurality of second pyramid level images 560 are obtained through pyramid expansion (step 234). Through pyramid expansion, for example, five levels of pyramid level images can be obtained, and the total number of levels can be determined according to the size of the original image. Pyramid expansion is to resize the image using a Gaussian blur. In pyramid expansion, a Laplacian pyramid that utilizes global and local information in the image can be used.

Next, the image processor 60 fuses the first pyramid level images 550 and the second pyramid level images 560 in the wavelet region for each level, thereby obtaining the fusion pyramid level images 570 (step 235). ).

Next, the image processing unit 60 obtains a frequency domain fusion image 580 through pyramid compression from the fusion pyramid level images 570 (step 236). Pyramid compression corresponds to the inverse transformation of pyramid expansion in step 233.

Next, the image processing unit 60 obtains the fusion image 590 from the frequency domain fusion image 580 through an inverse discrete wavelet transform (IDWT) (step 237). The fused image 590 becomes a clear image in both the central region and the outer region.

FIG. 6 is a diagram illustrating a partial enlarged comparison of a first image, a second image, and a fused image obtained according to an embodiment of the present invention.

Referring to FIG. 6, the first image 510 focused on the center has a clear central area, but the outer area of the center is blurry, and the second image 520 focused on the outer center of the center has a clear outer center but the center. While the region is blurry, it can be seen that the fusion image 590 is clear in both the central region and the outer region.

The digital microscope according to the present invention includes millimeters, micrometers to submicrometers, imaging in an incubator, real-time monitoring in industrial sites or inspection of fine patterns, and portable for field inspection (point of care). It can be used in various fields such as a microscope.

Meanwhile, the above-described embodiments of the present invention can be written in a program executable on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (eg, ROM, floppy disk, hard disk, etc.), an optical reading medium (eg, CD-ROM, DVD, etc.).

Embodiments of the present invention may be represented by functional block configurations and various processing steps. These functional blocks can be implemented with various numbers of hardware or/and software configurations that perform specific functions. For example, an embodiment may be implemented in an integrated circuit configuration such as memory, processing, logic, look-up table, etc., capable of executing various functions by control of one or more microprocessors or other control devices. You can hire them. Similar to how the components of the present invention can be implemented in software programming or software components, embodiments include C, C++, including various algorithms implemented in a combination of data structures, processes, routines or other programming components. , Can be implemented in programming or scripting languages such as Java, assembler, etc. Functional aspects can be implemented with algorithms running on one or more processors. In addition, embodiments may employ conventional techniques for electronic environment setting, signal processing, and/or data processing. Terms such as "mechanism", "element", "means", and "configuration" can be used widely and are not limited to mechanical and physical configurations. The term may include the meaning of a series of software routines in connection with a processor or the like.

The specific implementations described in the embodiment are one embodiment, and do not limit the scope of the embodiment in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings are illustrative examples of functional connections and/or physical or circuit connections, which can be replaced or additional various functional connections in physical devices, physical It can be represented as a connection, or circuit connections. In addition, unless specifically mentioned, such as "essential", "important", etc., it may not be a necessary component for application of the present invention.

So far, the present invention has been focused on the preferred embodiments. Those skilled in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in terms of explanation, not limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent range should be interpreted as being included in the present invention.

Claims (14)

An objective lens disposed on one side of the sample;
An image sensor disposed on one side of the objective lens and acquiring an image of the sample magnified through the objective lens;
Variable means for varying the distance between the sample and the objective lens; And
From the image sensor, the first image obtained when the distance between the objective lens and the sample is the first interval and the second image obtained when the distance is the second interval are input, and the first image and the second A digital microscope comprising an image processing unit that fuses an image to generate a fused image. According to claim 1,
The image processing unit is a digital microscope, characterized in that the first image and the second image are fused in the frequency domain. According to claim 1,
The first image is an image focused on the center of the image sensor, and the second image is a digital microscope characterized in that the image is focused on the outside of the center of the image sensor. According to claim 3,
The second interval is a digital microscope, characterized in that smaller than the first interval. According to claim 2,
The image processor acquires first and second frequency domain images through DWT (Discrete Wavelet Transform) from the first and second images, respectively, and fuses the first and second frequency domain images to fuse the frequency domain images. And acquiring the fusion image through IDWT (Inverse Discrete Wavelet Transform) from the frequency domain fusion image. The method of claim 5,
The image processing unit acquires first pyramid level images and second pyramid level images through pyramid expansion from the first and second frequency domain images, respectively, and the first pyramid level images and the first 2 A digital microscope characterized by fusing pyramid level images for each level to obtain fusion pyramid level images, and acquiring the frequency domain fusion image through pyramid compression from the fusion pyramid level images. According to claim 1,
A light source irradiating light to the sample; And
And a focusing lens for focusing light from the light source. As a method of operating a digital microscope,
The digital microscope includes an objective lens disposed on one side of the sample, and an image sensor disposed on one side of the objective lens and obtaining an image of the sample magnified through the objective lens,
Acquiring a first image of the sample with the image sensor when the distance between the objective lens and the sample is a first interval;
Acquiring a second image of the sample with the image sensor when the distance between the objective lens and the sample is a second interval; And
And generating a fused image by fusing the first image and the second image. The method of claim 8,
The generating of the fused image is a method of operating a digital microscope, wherein the first image and the second image are fused in a frequency domain. The method of claim 8,
The first image is an image focused on the center of the image sensor, and the second image is an image focused on the outside of the center of the image sensor. The method of claim 10,
The second interval is less than the first interval, the operation method of the digital microscope, characterized in that. The method of claim 9,
The step of generating the fusion image,
Obtaining first and second frequency domain images through DWT (Discrete Wavelet Transform) from the first and second images, respectively;
Fusing the first and second frequency domain images to obtain a frequency domain fused image; And
And acquiring the fusion image from the frequency domain fusion image through an Inverse Discrete Wavelet Transform (IDWT). The method of claim 12,
Acquiring the frequency domain fusion image,
Obtaining first pyramid level images and second pyramid level images through pyramid expansion from the first and second frequency domain images, respectively;
Fusing the first pyramid level images and the second pyramid level images for each level to obtain fusion pyramid level images; And
And acquiring the frequency domain fusion image through pyramid compression from the fusion pyramid level images. The method of claim 8,
The digital microscope, the method of operating a digital microscope, characterized in that it further comprises a light source for irradiating light to the sample and a focusing lens for focusing light from the light source.

Images