KR102401164B1 - Autonomous Operation of Atomic Force Microscope by Using Machine Learning-Based Object Detection Technique - Google Patents

Abstract

An atomic force microscope of the present invention includes a sample stage for receiving a sample; a cantilever disposed on the sample and including a probe; a laser that scans the cantilever with a laser beam; a photo detector receiving the laser beam reflected from the cantilever; a first camera for photographing the sample and the cantilever; a second camera for photographing a laser spot scanned from the cantilever and the laser; and a processor electrically connected to the first camera, the second camera, and the photo detector, and processing data obtained from the first camera, the second camera, and the photo detector.
A method of operating an atomic force microscope according to the present invention includes: detecting positions of a cantilever and a sample through a first camera; adjusting the position of the sample; detecting the positions of the laser and the cantilever through a second camera; aligning the laser; detecting the position of the laser beam through a photodetector; It may include aligning the position of the photo detector.

Description

Atomic Force Microscope and its Operation Method Using Artificial Intelligence Object Recognition Technology {Autonomous Operation of Atomic Force Microscope by Using Machine Learning-Based Object Detection Technique}

The present invention relates to an atomic force microscope and a method of operation thereof. Specifically, the present invention relates to an atomic force microscope automated using artificial intelligence object recognition technology and a method of operating the same.

An atomic force microscope is an instrument that measures the shape of a sample by scanning the surface of a sample with a fine probe attached to the tip of a cantilever. Here, the atomic force microscope uses lasers and photodetectors to visualize the topography. The laser is used to read the movement of the cantilever, and is fired at the tip of the cantilever. The laser reflected from the cantilever is input to the photodetector, and as the cantilever is bent, the input laser signal changes. Scanning the surface of a sample results in an image that we can see with our eyes.

The preparation process for driving the atomic force microscope is divided into three main steps. It consists of three steps: (i) laser-cantilever alignment, (ii) laser-photodetector alignment, and (iii) tip-sample access and scan.

On the other hand, since the probe is damaged every time a sample is measured, it must be replaced periodically. At this time, every time the cantilever is replaced, in most commercial equipment, laser-cantilever alignment, laser-photo detector alignment, and tip-sample access are manually performed by humans, which takes a long time and reduces the user-to-user alignment process. Accuracy occurs, which leads to differences in the final result. There is also a risk of damage to parts such as the cantilever or sample during the manual tip-sample approach.

An object of the present invention is to provide an automated atomic force microscope using artificial intelligence technology and a method of operating the same.

In order to achieve the object of the present invention as described above,

An atomic force microscope of the present invention includes a sample stage for receiving a sample; a cantilever disposed on the sample and including a probe; a laser that scans the cantilever with a laser beam; a photo detector receiving the laser beam reflected from the cantilever; a first camera for photographing the sample and the cantilever; a second camera for photographing a laser spot scanned from the cantilever and the laser; and a processor electrically connected to the first camera, the second camera, and the photo detector, and processing data obtained from the first camera, the second camera, and the photo detector.

A method of operating an atomic force microscope according to the present invention includes: detecting positions of a cantilever and a sample through a first camera; adjusting the position of the sample; detecting the positions of the laser and the cantilever through a second camera; aligning the laser; detecting the position of the laser beam through a photodetector; It may include aligning the position of the photo detector.

Through the atomic force microscope of the present invention and its operation method, it is possible to automatically and accurately align the position of the cantilever and sample, the position of the cantilever and the laser spot, and the position of the laser beam and the photodetector that require very fine and delicate adjustment. have. Conventionally, all of these adjustments are manually performed, which takes a long time, and there is a risk of damage to parts such as a cantilever or a sample due to a difference in positions occurring during the alignment process between users, resulting in poor alignment accuracy. However, when using the atomic force microscope of the present invention, it is possible to automatically and accurately align, reduce the risk of parts being damaged, and furthermore, it is possible to obtain a highly reliable image using the atomic force microscope.

BRIEF DESCRIPTION OF THE DRAWINGS It is a schematic diagram of an atomic force microscope.
2 is a schematic diagram of an atomic force microscope according to an embodiment of the present invention.
3 is a sample photograph of an atomic force microscope according to an embodiment of the present invention.
4 is a flowchart of a method of operating an atomic force microscope according to an embodiment of the present invention.
5 is a flowchart of a method of operating an atomic force microscope according to an embodiment of the present invention.
6 is a photograph for explaining a method of operating an atomic force microscope according to an embodiment of the present invention.
7 is a flowchart of a method of operating an atomic force microscope according to an embodiment of the present invention.
8 is a photograph for explaining a method of operating an atomic force microscope according to an embodiment of the present invention.
9 is a flowchart of a method of operating an atomic force microscope according to an embodiment of the present invention.
10 is a photograph for explaining a method of operating an atomic force microscope according to an embodiment of the present invention.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. The examples and terms used therein are not intended to limit the technology described in this document to a specific embodiment, but should be understood to include various modifications, equivalents, and/or substitutions of the embodiments.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, referring to FIG. 1 , the atomic force microscope receives a sample stage (Sample Stage) for receiving a sample, a cantilever including a probe, a laser for scanning a laser beam, and a reflected laser beam. It may include a receiving photo detector (Photo Detector).

Hereinafter, an atomic force microscope according to an embodiment of the present invention will be described in more detail with reference to FIG. 2 .

Referring to FIG. 2 , an atomic force microscope according to an embodiment of the present invention includes a sample stage 120 for accommodating a sample S, and a motorized sample stage for adjusting the positions of the sample stage 120 . ) (125), a cantilever 130 including a probe (T), a laser 140 for scanning a laser beam on the cantilever 130, a laser position adjusting unit for adjusting the position of the laser 14 (Motorized stage) (145) ), a photo detector 150 receiving the laser beam reflected from the cantilever 130, a photo detector position adjusting unit for adjusting the position of the photo detector 150 (Motorized stage) 155, the first camera 160, the first 2 includes a camera 170 , controllers 180 and 190 , and a processor 100 .

The first camera 160 may be arranged to photograph the sample S and the cantilever 130 . The second camera 170 may be arranged to photograph the laser spot scanned from the cantilever 130 and the laser 140 .

The processor 100 is electrically connected to the first camera 160 , the second camera 170 , and the photo detector 150 , and operates from the first camera 160 , the second camera 170 and the photo detector 150 . The acquired data can be processed. The processor 100 may store various AI object recognition programs to process data obtained from the first camera 160 , the second camera 170 , and the photo detector 150 . For example, the processor 100 may store programming software capable of controlling various configurations of the atomic force microscope. In addition, the processor 100 includes an artificial intelligence object recognition program such as a machine learning program capable of identifying and detecting a target configuration from an image captured by the first camera 160 and the second camera 170 . can be saved. For example, LabVIEW may be used as the programming software, and TensorFlow may be used as the machine learning program. However, the embodiment is not limited thereto, and various types of programming software and machine learning programs for artificial intelligence object recognition technology may be stored in the processor 100 .

The processor 100 may process various data and adjust positions of the sample stage 120 , the laser 140 , and the photo detector 150 through the controllers 180 and 190 . Specifically, the processor 100 is electrically connected to the controllers 180 and 190 , and the controllers 180 and 190 may align the position of the sample stage 120 through the sample stage position adjusting unit 125 , and the laser The position of the laser 140 may be aligned through the position adjusting unit 145 , and the position of the photo detector 150 may be aligned through the photo detector position adjusting unit 155 .

The processor 100 may acquire image data of the cantilever 130 and the sample S by using the first camera 160 . On the other hand, by learning the shape of the cantilever 130 and the sample (S) through the machine learning program stored in the processor 100, the processor 100 cantilever 130 and the cantilever from the image data of the sample (S) It is possible to identify the samples S and generate their location information. The processor 100 may correct the positions of the cantilever 130 and the sample S by using the position information of the cantilever 130 and the sample S and pre-stored position information. That is, the processor 100 compares the position information obtained from the current image data with the position information in which the probe T at the tip of the cantilever 130 is located at an appropriate position on the surface of the sample S, and the current cantilever 130 and When the position of the sample S needs to be corrected, it is possible to generate corrected position information. The processor 100 may transmit the corrected position information to the controller 180 , and the controller 180 may align the position of the sample stage 120 to the corrected position through the sample stage position adjusting unit 125 . .

The processor 100 may acquire image data of a laser spot irradiated from the cantilever 130 and the laser 140 using the second camera 170 . On the other hand, by learning the shape of the cantilever 130 and the laser spot through the machine learning program stored in the processor 100, the processor 100 cantilever 130 and the laser spot from the image data of the cantilever 130 and the laser spot. can be identified and their location information can be generated. The processor 100 may correct the positions of the cantilever 130 and the laser spot by using the position information of the cantilever 130 and the laser spot and pre-stored position information. That is, the processor 100 compares the position information at which the laser spot is located at an appropriate position on the end face of the cantilever 130 with the position information obtained from the current image data, and corrects the position of the current cantilever 130 and the laser spot. If necessary, it is possible to generate corrected position information. The processor 100 may transmit the corrected position information to the controller 190 , and the controller 190 may align the position of the laser 140 to the corrected position through the laser position adjusting unit 145 . That is, the position of the laser 140 may be aligned so that the laser spot may be irradiated to the end surface of the cantilever 130 .

The processor 100 may obtain position data of a laser beam reflected from the cantilever 130 and incident on the photo detector 150 using the photo detector 150 . Corrected position information of the laser beam may be generated by using the position data of the laser beam and the pre-stored position data. That is, the processor 100 compares the position information of the laser beam located at the exact center of the photo detector 150 with the position information of the current laser beam from the photo detector 150, and when the position of the laser beam needs to be corrected, the corrected position The processor 100 may transmit the corrected position information to the controller 190 , and the controller 190 moves the photo detector 150 to a corrected position through the photo detector position adjusting unit 155 . position of the photo detector 150 may be aligned, that is, the position of the photo detector 150 may be aligned so that the laser beam reflected from the cantilever 130 may be incident on the center of the photo detector 150 .

Meanwhile, FIG. 3 is a sample photograph of an atomic force microscope according to an embodiment of the present invention. Referring to FIG. 3 , automation can be implemented by installing a first camera (Camera 1) and a second camera (Camera 2) and applying an object recognition technology using artificial intelligence. . Therefore, through the atomic force microscope of the present invention, it is possible to automatically and accurately align the positioning of the cantilever and the sample, the positioning of the cantilever and the laser spot, and the positioning of the laser beam and the photodetector that require very fine and delicate adjustment. Conventionally, in many cases, such adjustments are manually performed and take a long time, and accuracy is reduced due to differences in positions during the alignment process between users, and there is a risk of damage to parts such as cantilevers or samples. In addition, since most of the conventional automation equipment implements automation that indirectly detects an object based on an electronic signal, there is a possibility of malfunction and possible damage to the cantilever or sample. However, in the present invention, the accuracy is very high because the target configuration itself can be directly recognized and the position can be adjusted using the artificial intelligence object recognition technology. However, when using the atomic force microscope of the present invention, it is possible to automatically and accurately align, reduce the risk of parts being damaged, and furthermore, it is possible to obtain a highly reliable image using the atomic force microscope.

Hereinafter, an operating method of an atomic force microscope according to an embodiment of the present invention will be described with reference to FIGS. 4 to 10 .

Referring to FIG. 4 , the method of operating an atomic force microscope according to an embodiment of the present invention includes the steps of detecting the positions of the cantilever and the sample through a first camera (S100), and adjusting the positions of the sample (S200). , the step of detecting the position of the laser and the cantilever through the second camera (S300), the step of aligning the laser (S400), the step of detecting the position of the laser beam through the photo detector (S500), and the position of the photo detector It may include an aligning step (S600).

Hereinafter, an operating method of an atomic force microscope according to an embodiment of the present invention will be described in more detail with reference to FIGS. 5 and 6 .

Referring to FIG. 5 , the first camera 160 may photograph the sample S and the cantilever 130 through the image sensor, and generate image data of the cantilever 130 and the sample S ( S210 ). . Such image data may be provided to the processor 100 ( S220 ). The processor 100 may acquire image data of the cantilever 130 and the sample S (S230), and generate position information of the cantilever 130 and the sample S (S240). Next, the processor 100 generates corrected position information of the cantilever 130 and the sample S by using the position information of the cantilever 130 and the sample S and the previously stored position information (S250). can The processor 100 may provide the corrected location information to the controller 180 ( S260 ). The controller 180 may adjust the position of the sample stage 120 using the corrected position information ( S270 ). Meanwhile, as shown in FIG. 6 , the processor 100 may perform the above-described operation through a program capable of confirming the positions of the cantilever 130 and the sample S.

Referring to FIG. 7 , the second camera 170 may photograph the laser spot irradiated from the cantilever 130 and the laser 140 , and may generate image data of the cantilever 130 and the laser spot ( S410 ). Such image data may be provided to the processor 100 ( S420 ). The processor 100 may acquire image data of the cantilever 130 and the laser spot ( S430 ), and generate position information of the cantilever 130 and the laser spot ( S440 ). Next, the processor 100 may generate corrected position information of the cantilever 130 and the laser spot by using the position information of the cantilever 130 and the laser spot and the previously stored position information ( S450 ). The processor 100 may provide the corrected location information to the controller 190 ( S460 ). The controller 190 may adjust the position of the laser 140 using the corrected position information ( S470 ). Meanwhile, as shown in FIG. 8 , the processor 100 may perform the above-described operation through a program capable of confirming the positions of the cantilever 130 and the laser spot.

Referring to FIG. 9 , the photo detector 150 may generate position data of a laser beam reflected from the cantilever 130 and incident on the photo detector 150 ( S610 ). Such location data may be provided to the processor 100 (S620). The processor 100 may generate corrected position information of the laser beam by using the position data of the laser beam and the pre-stored position data (S640). The processor 100 may provide the corrected location information to the controller 190 ( S650 ). The controller 190 may adjust the position of the photo detector 150 using the corrected position information ( S670 ). Meanwhile, as shown in FIG. 10 , the processor 100 may perform the above-described operation through a program capable of confirming the position of the laser beam.

So far, the present invention has been looked at with respect to preferred embodiments thereof. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims (13)

a sample stage for receiving a sample;
a cantilever disposed on the sample and including a probe;
a laser that scans the cantilever with a laser beam;
a photo detector receiving the laser beam reflected from the cantilever;
a first camera for photographing the sample and the cantilever;
a second camera for photographing a laser spot scanned from the cantilever and the laser; and
A processor electrically connected to the first camera, the second camera, and the photo detector to process data obtained from the first camera, the second camera, and the photo detector;
The processor learns the shape of the cantilever, the sample, and the laser spot through an artificial intelligence object recognition program,
Obtaining image data of the cantilever and the sample using the first camera, and generating position information of the cantilever and the sample from the image data using the shape learned by the AI object recognition program,
using the position information of the cantilever and the sample and the previously stored position information to generate corrected position information of the cantilever and the sample;
Adjusting the position of the sample stage through a controller using the corrected position information of the cantilever and the sample,
Obtaining image data of the cantilever and the laser spot using the second camera,
generating position information of a cantilever and a laser spot from the image data using the shape learned by the artificial intelligence object recognition program;
using the position information of the cantilever and the laser spot and the previously stored position information to generate corrected position information of the cantilever and the laser spot;
Adjusting the position of the laser through the controller using the corrected position information of the cantilever and the laser spot,
Obtaining position data of a laser beam incident on the photodetector by using the photodetector,
By using the position data and the pre-stored position data, to generate corrected position information of the laser beam,
An atomic force microscope that adjusts the position of the photo detector through a controller using the corrected position information of the laser beam. delete delete delete delete delete delete delete delete delete Learning the shape of the cantilever, the sample, and the laser spot through an artificial intelligence object recognition program;
acquiring image data of the cantilever and the sample by using the first camera;
generating position information of the cantilever and the sample from the image data of the cantilever and the sample by using the shape learned by the artificial intelligence object recognition program;
generating corrected position information of the cantilever and the sample by using the position information of the cantilever and the sample and pre-stored position information;
adjusting the position of the sample stage through a controller using the corrected position information of the cantilever and the sample;
acquiring image data of the cantilever and the laser spot by using a second camera;
generating position information of the cantilever and the laser spot from the image data of the cantilever and the laser spot by using the shape learned by the artificial intelligence object recognition program;
generating corrected position information of the cantilever and the laser spot by using the position information of the cantilever and the laser spot and pre-stored position information;
adjusting the position of the laser through a controller using the corrected position information of the cantilever and the laser spot;
acquiring position data of a laser beam incident on the photo detector by using the photo detector;
generating corrected position information of the laser beam by using the position data of the laser beam and pre-stored position data; and
and adjusting a position of the photo detector through a controller using the corrected position information of the laser beam. delete delete

Images