CN115657264A - Focus following method and device of heat reflection microscopic thermal imaging system and electronic equipment - Google Patents
Focus following method and device of heat reflection microscopic thermal imaging system and electronic equipment Download PDFInfo
- Publication number
- CN115657264A CN115657264A CN202211324332.6A CN202211324332A CN115657264A CN 115657264 A CN115657264 A CN 115657264A CN 202211324332 A CN202211324332 A CN 202211324332A CN 115657264 A CN115657264 A CN 115657264A
- Authority
- CN
- China
- Prior art keywords
- image
- object distance
- initial
- step length
- definition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention provides a focus following method and device of a heat reflection microscopic thermal imaging system and electronic equipment. The method comprises the following steps: an initial image, a first image and a second image are obtained. And respectively calculating the image definition of the initial image, each first image and each second image. And if the first average definition is larger than the second average definition, adjusting the object distance to the initial object distance minus a preset step length. And if the first average definition is less than or equal to the second average definition, adjusting the object distance to the initial object distance plus a preset step length. And taking the adjusted object distance as the initial object distance. According to the invention, the object distance is alternately adjusted according to the preset step length, images are acquired, the average image definition under two adjacent object distances is compared, after the object distance is adjusted according to the preset step length, the object distance is again taken as a reference to continuously and alternately adjust the object distance to acquire the images and adjust the object distance. The method can be circularly executed in the temperature testing process of the heat reflection microscopic thermal imaging system, the object distance is adjusted in real time, the image is continuously kept clear, and continuous real-time focus following is realized.
Description
Technical Field
The invention relates to the technical field of heat reflection microscopic thermal imaging, in particular to a focus following method and device of a heat reflection microscopic thermal imaging system and electronic equipment.
Background
The heat reflection temperature measurement technology is a non-contact temperature measurement technology, and is based on the heat reflection phenomenon. The heat reflection phenomenon is basically characterized in that the reflectivity of an object changes with the temperature of the object. The basic principle of heat reflection temperature measurement is to measure the relative change condition delta R/R of the reflectivity R 0 To derive the temperature change at. By utilizing the heat reflection phenomenon, the image of the tested piece is collected by utilizing the heat reflection micro thermal imaging system, and the heat reflection micro thermal imaging temperature measurement can be realized.
Thermal reflection microscopy thermography generally comprises a three-axis nano-displacement stage and an imaging microscope. The measured piece is fixed on the three-axis nanometer displacement platform, and the imaging microscope acquires the image of the measured piece. The distance between the measured piece and the optical center of the lens of the imaging microscope is the object distance. The object distance can be adjusted and imaging can be clearly realized by adjusting the Z-axis direction of the three-axis nano displacement table. In an actual test, under the influence of factors such as temperature change, external vibration and the like, the position of a tested piece in the Z-axis direction can move and deviate from a focus, so that imaging is fuzzy. Therefore, the object distance needs to be continuously adjusted by using a focus following algorithm to realize clear imaging in the test.
The existing methods include a defocus depth method and an in-focus depth method. The accuracy of the out-of-focus depth method is poor, the heat reflection temperature measurement has high out-of-focus sensitivity, the imaging microscope has small focal depth, and the out-of-focus depth method cannot ensure the test accuracy. The focusing depth method is characterized in that images are collected at different object distances, the definition is evaluated, and a focusing evaluation curve is fitted to determine a peak point as an optimal object distance position. The larger the range of the object distance is, the more the image is acquired, and the more accurate the object distance determined by the focusing depth method is. The in-focus depth method can achieve higher focusing accuracy, but requires determination of an in-focus position within a larger focusing range. When the focus is deviated too much, for example beyond the depth of focus, the image is blurred. However, the heat reflection temperature measurement generally needs to continuously acquire images, so that the focusing range of the focusing depth method is too large, the image cannot be continuously kept clear, and the method is not suitable for continuous real-time focus following in the heat reflection temperature measurement application.
Disclosure of Invention
The embodiment of the invention provides a focus following method and device of a heat reflection microscopic thermal imaging system and electronic equipment, and aims to solve the problems that when the heat reflection microscopic thermal imaging system tests the temperature of a tested piece, the focus depth method has an overlarge focusing range, cannot continuously keep an image clear, and is not suitable for continuous real-time focus following.
In a first aspect, an embodiment of the present invention provides a focus following method for a thermal reflection microscopic thermal imaging system, including:
an initial image is acquired, wherein the initial image is an image acquired at an initial object distance.
And acquiring a first image, wherein the first image is acquired after the object distance is adjusted to a first object distance from the initial object distance, and the first object distance is the initial object distance minus a preset step length.
And acquiring a second image, wherein the second image is acquired after the object distance is adjusted from the first object distance to the initial object distance.
And repeatedly executing the first image acquisition and the second image acquisition to obtain multiple groups of first images and second images.
And respectively calculating the image definition of the initial image, each first image and each second image.
And if the first average definition is larger than the second average definition, adjusting the object distance to the initial object distance minus a preset step length. The first average definition is an average value of image definitions of the first images, and the second average definition is an average value of image definitions of the initial image and the second images.
And if the first average definition is less than or equal to the second average definition, adjusting the object distance to the initial object distance plus a preset step length.
And circularly acquiring images by taking the adjusted object distance as an initial object distance, calculating average definition and adjusting the object distance.
In a possible implementation manner, before the acquiring the initial image, the method further includes: and acquiring a first step length with a preset fixed length as a preset step length, wherein the first step length is smaller than the focal depth of the heat reflection micro thermal imaging system.
In a possible implementation manner, before the acquiring the initial image, the method further includes: and acquiring a second step size and a third step size, wherein the second step size is smaller than the focal depth of the thermal reflection micro thermal imaging system. The third step size is greater than the second step size. And when the tested piece is tested at constant temperature by adopting a heat reflection microscopic thermal imaging system, taking the second step length as a preset step length. And when the tested piece is subjected to temperature change test by adopting a heat reflection microscopic thermal imaging system, taking the third step length as a preset step length, wherein the third step length is in direct proportion to the amplitude of the temperature change.
In a possible implementation manner, after the circularly acquiring images with the adjusted object distance as an initial object distance, calculating an average definition, and performing object distance adjustment, the method further includes: and if the object distance adjusting directions in two continuous cycles are the same, increasing the length of the preset step length. And if the object distance adjusting directions in two continuous cycles are opposite, reducing the length of the preset step length.
In a possible implementation manner, the separately calculating the image sharpness of the initial image, each of the first images, and each of the second images includes: respectively calculating the image definition of the initial image, each first image and each second image based on the following formulas:
wherein, c i The gray value of each pixel of the image, P is the total pixel number of the image, and s is the image definition.
In a possible implementation manner, after the acquiring the initial image, the method further includes: calculating an image sharpness based on the initial image. Correspondingly, the acquiring the first image comprises: and if the image definition of the initial image is lower than a preset threshold value, acquiring a first image.
In a second aspect, an embodiment of the present invention provides a focus tracking apparatus of a thermal reflection microscopy thermal imaging system, where the apparatus includes:
the first acquisition module is used for acquiring an initial image, wherein the initial image is an image acquired at an initial object distance.
And the second acquisition module is used for acquiring a first image, wherein the first image is acquired after the object distance is adjusted to the first object distance from the initial object distance, and the first object distance is the initial object distance minus a preset step length.
And the third acquisition module is used for acquiring a second image, wherein the second image is acquired after the object distance is adjusted from the first object distance to the initial object distance.
And the first circulation module is used for repeatedly executing the acquisition of the first image and the acquisition of the second image to obtain a plurality of groups of first images and second images.
And the definition calculating module is used for calculating the image definition of the initial image, each first image and each second image respectively.
And the first adjusting module is used for adjusting the object distance to the initial object distance minus a preset step length if the first average definition is greater than the second average definition. The first average definition is an average value of image definitions of the first images, and the second average definition is an average value of image definitions of the initial image and the second images.
And the second adjusting module is used for adjusting the object distance to the initial object distance and adding a preset step length if the first average definition is less than or equal to the second average definition.
And the second circulation module is used for circularly acquiring images by taking the adjusted object distance as the initial object distance, calculating the average definition and adjusting the object distance.
In one possible implementation, the apparatus further includes: and the step length acquisition module is used for acquiring a first step length with a preset fixed length as a preset step length before the initial image is acquired, wherein the first step length is smaller than the focal depth of the thermal reflection microscopic thermal imaging system.
In a third aspect, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method according to the first aspect or any possible implementation manner of the first aspect when executing the computer program.
In a fourth aspect, the present invention provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the method according to the first aspect or any one of the possible implementation manners of the first aspect.
The embodiment of the invention provides a focus following method, a focus following device and electronic equipment of a heat reflection microscopic thermal imaging system. And acquiring a first image, wherein the first image is acquired after the object distance is adjusted to a first object distance from the initial object distance, and the first object distance is the initial object distance minus a preset step length. And acquiring a second image, wherein the second image is acquired after the object distance is adjusted from the first object distance to the initial object distance. And repeatedly executing the steps of acquiring the first image and acquiring the second image to acquire multiple groups of first images and second images. And respectively calculating the image definition of the initial image, each first image and each second image. And if the first average definition is larger than the second average definition, adjusting the object distance to the initial object distance minus a preset step length. The first average definition is the average value of the image definitions of the first images, and the second average definition is the average value of the image definitions of the initial images and the second images. And if the first average definition is less than or equal to the second average definition, adjusting the object distance to the initial object distance plus a preset step length. And circularly acquiring images by taking the adjusted object distance as an initial object distance, calculating average definition and adjusting the object distance. The invention alternately adjusts the object distance and collects images according to the preset step length at the initial object distance position and the adjacent position of the initial object distance, compares the average image definition under the two adjacent object distances, determines the adjustment direction of the object distance, adjusts the object distance according to the preset step length, and then continuously and alternately adjusts the object distance collection images, confirms the adjustment direction and adjusts the object distance by taking the adjusted object distance as the reference. In the focus following process, the range of each adjustment of the object distance is within a preset step length, the deviation from the initial object distance is small, namely the deviation from the focus is small, and the image can be ensured to be clear. The method can be circularly executed in the temperature testing process of the heat reflection microscopic thermal imaging system, the object distance is adjusted in real time, the image is continuously kept clear, and continuous real-time focus following is realized.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a schematic diagram of an exemplary configuration of a thermal reflectance micro thermal imaging system;
FIG. 2 is a flowchart of an implementation of a focus following method of a thermal reflection microscopy thermal imaging system according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a method of continuously acquiring images;
FIG. 4 is a schematic diagram of a method for acquiring images by alternating object distances according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a focus following device of a thermal reflection microscopy thermal imaging system according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an electronic device provided in an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following description is made by way of specific embodiments with reference to the accompanying drawings.
The heat reflection temperature measurement technology is a non-contact temperature measurement technology, and is based on the heat reflection phenomenon. The basic characteristic of the heat reflection phenomenon is that the reflectivity of an object changes with the temperature of the object. The change in reflectance with temperature can be considered linear and therefore can be characterized by a coefficient of rate of change, commonly referred to as the heat reflection coefficient or heat reflection calibration coefficient, using C TR Is defined as:
in the formula, R is a reference reflectivity, Δ R is a reflectivity variation, and Δ T is a temperature variation.
For most metal and semiconductor materials, C TR Is usually in the range of (10) -2 ～10 -5 )K -1 And is related to the material, the wavelength of the incident light, and the angle of incidence. If the surface of the object has a multi-layer structure, the material composition of each layer and the interference of light among the multi-layer materials also directly affect C TR Is generally carried out by selecting a suitable measurement wavelength for each sample (type or model) and determining C TR Commonly referred to as C TR Calibrating and using measured C TR Temperature measurements are taken.
At C TR In known cases, the temperature can be calculated by measuring the change in reflectivity of the object according to the following formula
In the formula (2), T x To be the temperature to be measured, T 0 For reference temperature, R x Is the reflectance at the temperature to be measured, R 0 Is the reflectance at the reference temperature.
Since it is actually the rate of change (R) of the reflectivity that is of interest x -R 0 )/R 0 Therefore, a beam of detection light (incident light) can be projected to the surface of the measured object, and then the temperature measurement can be realized by measuring the change rate of the intensity of the reflected light, which is also the realization mode of the heat reflection temperature measurement technology. The rate of change of reflectivity in equation (2) can be equivalent to the rate of change of the detector reading, assuming that the detected light intensity is constant, i.e. equation (2) changes to
In the formula (3), c x For the reading of the detector at the temperature to be measured, c 0 Is the detector reading at the reference temperature.
In order to realize high-spatial-resolution microthermal imaging, a microlight-reflecting thermal imaging device is generally constructed based on a high-performance optical microscope. The probe light is provided by the illumination system of the optical microscope, the microscopic image is recorded using a high performance camera, and the output camera reading is taken as the measured value c.
Due to C TR Low magnitude, in order to guarantee measurement accuracy, at acquisition c 0 And c x It usually needs to take the average value of multiple frames of images, and the total number of measured frames is recorded as N, if there is
According to the principle, the data on each pixel of the camera and the spatial position of the measured surface need to have a stable corresponding relationship in the whole measuring process, and if the corresponding relationship is interfered, the accuracy of the temperature measuring result is influenced. The tested piece deviates from the focus due to corresponding thermal expansion, namely, the focus is out of focus, so that the picture is blurred, multiple times of focusing are needed in the test, the focusing consistency is good enough to ensure the stable and consistent corresponding relation, and otherwise, additional errors are introduced.
Fig. 1 is a schematic diagram of a typical structure of a thermal reflection micro thermal imaging system. Referring to fig. 1, a typical thermal reflection microscopy thermography system comprises an imaging microscope, a temperature controlled stage, a three-axis nano-displacement stage, a computer, and the like. The measured piece is fixed on the three-axis nanometer displacement platform, and the imaging microscope acquires the image of the measured piece. The distance between the measured piece and the optical center of the lens of the imaging microscope is the object distance. The object distance can be adjusted and imaging can be clearly realized by adjusting the Z-axis direction of the three-axis nano displacement table. In an actual test, under the influence of factors such as temperature change, external vibration and the like, the position of a tested piece in the Z-axis direction can move and deviate from a focus, so that imaging is blurred. Therefore, the object distance needs to be continuously adjusted by using a focus following algorithm in the test to realize clear imaging.
The existing methods include a defocus depth method and an in-focus depth method. The accuracy of the out-of-focus depth method is poor, the heat reflection temperature measurement has high out-of-focus sensitivity, the imaging microscope has small focal depth, and the out-of-focus depth method cannot ensure the test accuracy. The focusing depth method is characterized in that images are collected at different object distances, the definition is evaluated, and a focusing evaluation curve is fitted to determine a peak point as an optimal object distance position. The larger the range of the collected object distance is, the more the collected images are, and the more accurate the object distance determined by the focusing depth method is. The in-focus depth method can achieve higher focusing accuracy, but requires determination of an in-focus position within a larger focusing range. When the focus is deviated too much, for example beyond the depth of focus, the image is blurred. However, the heat reflection temperature measurement generally needs to continuously acquire images, so that the focusing range of the focusing depth method is too large, the image cannot be continuously kept clear, and the method is not suitable for continuous real-time focus following in the heat reflection temperature measurement application.
The embodiment of the invention provides a focus following method and device of a heat reflection microscopic thermal imaging system and electronic equipment, and aims to solve the problems that when the heat reflection microscopic thermal imaging system tests the temperature of a tested piece, the focus depth method has an overlarge focusing range, cannot continuously keep an image clear, and is not suitable for continuous real-time focus following.
Fig. 2 is a flowchart of an implementation of a focus following method of a thermal reflection microscopic thermal imaging system according to an embodiment of the present invention. Referring to fig. 2, the method includes:
s1, obtaining an initial image, wherein the initial image is an image acquired at an initial object distance.
Illustratively, the initial image is obtained by acquiring an image of the measured object through an imaging microscope of the thermal reflection micro thermal imaging system. Illustratively, the initial image is acquired by acquiring an image at an initial object distance. When the image is acquired, the measured object is usually at the focus position, and the initial object distance may be the object distance at the focus. The measured object may be off focus due to a variety of factors, and the initial object distance may also be the object distance off focus.
S2, acquiring a first image, wherein the first image is acquired after the object distance is adjusted to the first object distance from the initial object distance, and the first object distance is the initial object distance minus a preset step length.
And adjusting the object distance from the initial object distance to a first object distance, and then acquiring an image to obtain a first image. Illustratively, the first object distance may also be the initial object distance plus a preset step size. The object distance represents the relative distance of the three-axis nano-displacement stage from the imaging microscope. Illustratively, the object distance is adjusted by controlling the Z-axis direction movement of the three-axis nano displacement table. The relative distance between the measured piece and the imaging microscope is the actual object distance of the measured piece. Because the relative distance between the measured piece and the three-axis nanometer displacement platform changes due to thermal expansion, the relative distance between the three-axis nanometer displacement platform and the imaging microscope cannot completely represent the relative distance between the measured piece and the imaging microscope. The object distance can be adjusted by controlling the Z-axis direction movement of the triaxial nano displacement table, the change of the relative distance between the measured piece and the triaxial nano displacement table is counteracted, and the focal position of the measured piece on the imaging microscope is kept, namely, the focus tracking is realized.
And S3, acquiring a second image, wherein the second image is acquired after the object distance is adjusted from the first object distance to the initial object distance.
Illustratively, after the first image is acquired, the object distance is adjusted from the first object distance back to the initial object distance, and the image is acquired to obtain the second image. Although the initial image and the second image are both acquired at the initial object distance, the initial image and the second image are not necessarily the same image because the relative distance between the measured piece and the three-axis nanometer displacement table is continuously changed, namely the position of the measured piece is shifted.
And S4, repeatedly executing the steps of obtaining the first image and obtaining the second image, and obtaining a plurality of groups of first images and second images.
And (3) repeatedly executing the steps S2 and S3, acquiring a group of first images and second images once in a loop, and acquiring multiple groups of first images and second images in a loop. Illustratively, the above steps S2, S3 are performed at least twice in a loop to obtain at least two sets of first and second images. The focus returns to the original focus position once per cycle.
And S5, respectively calculating the image definition of the initial image, each first image and each second image.
Illustratively, the image sharpness may be calculated by a sharpness evaluation function. The sharpness evaluation function is also called a focus evaluation function and comprises an image gray value variance, an image gray value standard deviation, a Tenengrad function, a Laplace function and the like.
And S6, if the first average definition is larger than the second average definition, adjusting the object distance to the initial object distance minus a preset step length. The first average definition is the average value of the image definitions of the first images, and the second average definition is the average value of the image definitions of the initial images and the second images.
Illustratively, the image sharpness of each first image is arithmetically averaged to obtain the first average sharpness. The first average sharpness represents an average sharpness of images acquired at the first object distance position. Illustratively, the image definitions of the initial image and each second image are arithmetically averaged to obtain a second average definition. The second average sharpness represents the average sharpness of each image acquired at the initial object distance position. Illustratively, a higher sharpness value indicates a sharper image, and a lower sharpness value indicates a blurry image.
And comparing the first average definition with the second average definition, if the first average definition is greater than the second average definition, namely if the image definition at the first object distance position is higher than the definition at the initial object distance position, adjusting the object distance to the initial object distance and subtracting a preset step length, namely adjusting the object distance to the first object distance from the initial object distance, and realizing the adjustment of the object distance to the clear position from the fuzzy position. The direction of adjustment of the object distance is the decreasing direction, alternatively referred to as the negative direction.
And S7, if the first average definition is smaller than or equal to the second average definition, adjusting the object distance to the initial object distance plus a preset step length.
And comparing the first average definition with the second average definition, if the first average definition is less than or equal to the second average definition, namely if the image definition at the first object distance position is less than or equal to the definition at the initial object distance position, adjusting the object distance to the initial object distance and adding a preset step length, and realizing that the object distance is adjusted to the definition position from the fuzzy position. The adjustment direction of the object distance is an increasing direction, or referred to as a positive direction.
For example, in steps S2 and S3, if the first object distance is the initial object distance plus the preset step length, step S6 is to adjust the object distance to the initial object distance plus the preset step length if the first average resolution is greater than the second average resolution. Correspondingly, in step S7, if the first average resolution is less than or equal to the second average resolution, the object distance is adjusted to the initial object distance minus the preset step length.
And S8, circularly acquiring images by taking the adjusted object distance as an initial object distance, calculating average definition and adjusting the object distance.
The object distance adjusted in step S6 or step S7 is taken as the initial object distance, and steps S1 to S7 are re-executed based on the initial object distance. And circularly executing the steps S1 to S8, continuously determining the object distance adjusting direction and adjusting the object distance based on the initial object distance, and realizing focus following.
According to the embodiment of the invention, the object distance and the collected image are alternately adjusted according to the preset step length at the position of the initial object distance and the adjacent position of the initial object distance, the average image definition under the two adjacent object distances is compared, the adjustment direction of the object distance is determined, after the object distance is adjusted according to the preset step length, the object distance collected image is continuously and alternately adjusted again by taking the adjusted object distance as the reference, the adjustment direction is confirmed, and the object distance is adjusted. In the focus following process, the range of each adjustment of the object distance is within a preset step length, the deviation from the initial object distance is small, namely the deviation from the focus is small, and the image can be ensured to be clear. The method can be circularly executed in the temperature testing process of the heat reflection microscopic thermal imaging system, the object distance is adjusted in real time, the image is continuously kept clear, and continuous real-time focus following is realized.
For example, the relative distance change between the measured object and the three-axis nanometer displacement platform can include two situations. The first condition is as follows: the relative distance between the measured piece and the three-axis nanometer displacement table is kept unchanged after being changed. Case two: the relative distance between the measured piece and the three-axis nanometer displacement table in the Z-axis direction continuously changes, namely the measured piece always drifts in position. The embodiment of the invention is suitable for the two situations.
In a case one, the embodiment of the present invention may execute steps S1 to S8 in a loop for multiple times, and the object distance is continuously adjusted until the object distance is adjusted to be near the focal point, and finally, the object distance is repeatedly adjusted in a loop at a position near the focal point, so as to keep the object distance near the focal point.
In case two, the embodiment of the invention can acquire images by alternating object distance positions for a plurality of times when the measured piece continuously and slowly drifts. Due to the position drift of the measured piece, the definition of images acquired at different time and the same object distance position may be different. Fig. 3 is a schematic diagram of a method of continuously acquiring images. Referring to fig. 3, the horizontal axis represents the object distance, the vertical axis represents the sharpness of the image, the solid line curve represents the sharpness curve at the initial position at the initial time, and each dotted line represents the sharpness curve at each time after the initial time as time passes. The sharpness curve continuously drifts to the right over time. The filled triangles represent the time at which the image was acquired and the object distance position. The three solid triangles on the left indicate three consecutive acquisitions of images at the same object distance position. The four solid triangles on the right indicate that the images were acquired four times in succession after changing the object distance position. The three solid triangles on the left correspond to a higher definition average than the four triangles on the right, and in fact the object distance to the right is increased, the definition will be higher.
Fig. 4 is a schematic diagram of a method for acquiring images by alternating object distances according to an embodiment of the present invention. Referring to fig. 4, solid dots indicate the time and object distance positions at which the images are acquired. According to the time sequence, firstly, acquiring images corresponding to the object distance (initial object distance) by using the solid curve at the initial moment, then reducing the object distance by d (namely reducing the preset step length) to acquire the images, and then returning to the position of the initial object distance to acquire the images. And (4) performing in a circulating manner, alternately acquiring images at the object distance positions, and comparing the average definition of the images acquired at the two object distance positions, so that the object distance adjustment direction judgment error caused by continuous drift of the position of the detected piece in the focus following process can be avoided.
In one possible implementation, before acquiring the initial image, the method further includes: and acquiring a first step length of a preset fixed length as a preset step length, wherein the first step length is smaller than the focal depth of the heat reflection microscopic thermal imaging system.
The preset step size may be a fixed value during the loop execution of steps S1 to S8. The first step size may be acquired as the preset step size before step S1. Illustratively, the first step length is less than a depth of focus of the thermal reflectance micro-thermography system. The first step length is smaller than the focal depth, the object distance is changed by a distance of the first step length, and the image can still keep clear. Illustratively, the first step size is greater than the object distance difference corresponding to the noise level of the sharpness curve. That is, the change in image sharpness when the object distance is moved a preset step distance around the positive focus position is greater than the noise of the sharpness curve.
When the object distance is near the focus, the preset step length with the length as short as possible can ensure that the object distance is adjusted in the focus following process, the image keeps clear, and the focus following precision is high. In practical application of the thermal reflection microscopic thermal imaging system, the object distance may deviate from the focus to a relatively long distance, the preset step length is relatively small, the range of object distance adjustment is relatively small after steps S1 to S8 are executed circularly each time, and the efficiency of adjusting the object distance to the focus is relatively low.
In one possible implementation, before acquiring the initial image, the method further includes: and acquiring a second step length and a third step length, wherein the second step length is smaller than the focal depth of the thermal reflection microscopic thermal imaging system. The third step size is greater than the second step size. And when the tested piece is tested at constant temperature by adopting the heat reflection microscopic thermal imaging system, taking the second step length as a preset step length. When the tested piece is tested in a temperature-changing mode by adopting the heat reflection microscopic thermal imaging system, the third step length is taken as a preset step length, wherein the third step length is in direct proportion to the amplitude of the temperature change.
A smaller second step size and a larger third step size are set. When the measured piece is tested by adopting the constant temperature of the heat reflection micro thermal imaging system, the thermal expansion change of the measured piece is small, the position drift of the measured piece is small, and the focus following precision can be ensured by adopting a small second step length. When the heat reflection microscopic thermal imaging system is used for testing the tested piece in a temperature changing manner, the thermal expansion change of the tested piece is large, the position drift of the tested piece is large, a third step length is large, the object distance adjusting range is large, and the focus following efficiency can be ensured. Illustratively, the third step size may be a variation value. For example, the third step size may be proportional to the magnitude of the temperature change. Illustratively, the third step size is 5 to 10 times the second step size. For example, the temperature change test includes heating or cooling the part under test to change the temperature of the part under test. For example, the temperature-changing test may further include applying different voltages or currents to the tested object to change the operating temperature of the tested object.
According to the embodiment of the invention, the working state of the heat reflection microscopic thermal imaging system is monitored by setting the smaller second step length and the larger third step length, wherein the working state comprises a constant temperature test and a variable temperature test. And selecting preset step lengths with different lengths based on different working states. In the process of circularly executing the steps S1 to S8 each time, a smaller step length or a larger step length is selected based on an application scene, and the precision and the efficiency of focus following are both considered.
In a possible implementation manner, after cyclically acquiring images by taking the adjusted object distance as an initial object distance, calculating an average definition, and adjusting the object distance, the method further includes: and if the object distance adjusting directions in two continuous cycles are the same, increasing the length of the preset step length. And if the object distance adjusting directions in two continuous cycles are opposite, reducing the length of the preset step length.
Illustratively, in the step S1 to step S8, the direction of the object distance adjustment includes an increasing direction and a decreasing direction of the object distance. Illustratively, the same direction of the object distance adjustment in two consecutive cycles includes performing steps S1 to S8 in two consecutive cycles, where the direction of the object distance adjustment is either an increasing direction or a decreasing direction.
Illustratively, the opposite directions of the object distance adjustment in two consecutive cycles include performing steps S1 to S8 in two consecutive cycles, the direction of the object distance adjustment being an increasing direction and a decreasing direction in sequence, or the direction of the object distance adjustment being a decreasing direction and an increasing direction in sequence.
For example, if the object distance adjustment direction is the same in two consecutive cycles, the length of the preset step length is increased by a preset factor. d '= α d, α >1, d' represents the object distance after adjustment, d represents the object distance before adjustment, and α represents a preset coefficient. If the adjustment directions of the object distance in two continuous cycles are the same, the object distance is not adjusted to the peak point of the definition curve, the length of the preset step length can be increased, and the focus following efficiency is improved.
For example, if the object distance adjustment direction in two consecutive cycles is opposite, the length of the preset step length is decreased by a preset factor. d '= β d, β <1, d' represents the object distance after adjustment, d represents the object distance before adjustment, and β represents the preset coefficient. If the adjustment direction of the object distance in two consecutive cycles is opposite, it can indicate that the object distance is adjusted repeatedly on both sides of the peak point of the sharpness curve. Reducing the length of the preset step length can improve the accuracy of the follow focus.
In one possible implementation, the calculating the image sharpness of the initial image, each first image, and each second image separately comprises: the image sharpness of the initial image, each first image and each second image is calculated respectively based on the following formulas:
wherein, c i Is the gray value of each pixel of the image, P is the total number of pixels of the image, and s is the image definition.
The embodiment of the invention determines the definition based on the gray value of the collected image, and the image is clearest at the focus position and has the largest definition value. The output of the sharpness calculation method is theoretically not influenced by illumination intensity change, so that the influence of illumination intensity drift can be effectively inhibited, and a better focus following effect is obtained.
In a possible implementation manner, after the acquiring the initial image, the method further includes: an image sharpness is calculated based on the initial image. Accordingly, acquiring the first image includes: and if the image definition of the initial image is lower than a preset threshold value, acquiring a first image.
The embodiment of the invention is based on whether the current image definition really needs to adjust the object distance, namely whether the steps S1 to S8 are executed to adjust the object distance.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by functions and internal logic of the process, and should not limit the implementation process of the embodiments of the present invention in any way.
The following are embodiments of the apparatus of the invention, reference being made to the corresponding method embodiments described above for details which are not described in detail therein.
Fig. 5 is a schematic structural diagram of a focus tracking apparatus of a thermal reflection microscopy thermal imaging system according to an embodiment of the present invention, which only shows a part related to the embodiment of the present invention for convenience of description, and the following details are provided:
as shown in fig. 5, a focus tracking apparatus 5 of a thermal reflection microscopic thermal imaging system includes:
the first acquiring module 51 is configured to acquire an initial image, where the initial image is an image acquired at an initial object distance.
The second obtaining module 52 is configured to obtain a first image, where the first image is an image acquired after the object distance is adjusted from the initial object distance to the first object distance, and the first object distance is obtained by subtracting a preset step length from the initial object distance.
And a third obtaining module 53, configured to obtain a second image, where the second image is an image acquired after the object distance is adjusted from the first object distance to the initial object distance.
And a first loop module 54, configured to repeatedly perform acquiring the first image and acquiring the second image to obtain multiple sets of the first image and the second image.
And a definition calculating module 55, configured to calculate image definitions of the initial image, each of the first images, and each of the second images, respectively.
And a first adjusting module 56, configured to adjust the object distance to the initial object distance minus a preset step length if the first average sharpness is greater than the second average sharpness. The first average definition is the average value of the image definitions of the first images, and the second average definition is the average value of the image definitions of the initial images and the second images.
And a second adjusting module 57, configured to adjust the object distance to the initial object distance plus the preset step length if the first average resolution is smaller than or equal to the second average resolution.
And a second cyclic module 58, configured to cyclically acquire images using the adjusted object distance as an initial object distance, calculate average definition, and perform object distance adjustment.
According to the embodiment of the invention, the object distance and the collected image are alternately adjusted according to the preset step length at the position of the initial object distance and the adjacent position of the initial object distance, the average image definition under the two adjacent object distances is compared, the adjustment direction of the object distance is determined, after the object distance is adjusted according to the preset step length, the object distance collected image is continuously and alternately adjusted again by taking the adjusted object distance as the reference, the adjustment direction is confirmed, and the object distance is adjusted. In the focus following process, the range of each adjustment of the object distance is within a preset step length, the deviation from the initial object distance is small, namely the deviation from the focus is small, and the image can be ensured to be clear. The method can be circularly executed in the temperature testing process of the heat reflection microscopic thermal imaging system, the object distance is adjusted in real time, the image is continuously kept clear, and continuous real-time focus following is realized.
In one possible implementation, the apparatus further includes: the step length obtaining module is used for obtaining a first step length with a preset fixed length as a preset step length before obtaining the initial image, and the first step length is smaller than the focal depth of the heat reflection micro thermal imaging system.
In one possible implementation, before acquiring the initial image, the method further includes: and acquiring a second step length and a third step length, wherein the second step length is smaller than the focal depth of the thermal reflection microscopic thermal imaging system. The third step size is larger than the second step size. And when the tested piece is tested at constant temperature by adopting a heat reflection microscopic thermal imaging system, taking the second step length as a preset step length. When the tested piece is tested in a temperature-changing mode by adopting the heat reflection microscopic thermal imaging system, the third step length is taken as a preset step length, wherein the third step length is in direct proportion to the amplitude of the temperature change.
In a possible implementation manner, after cyclically acquiring images by taking the adjusted object distance as an initial object distance, calculating an average definition, and adjusting the object distance, the method further includes: and if the object distance adjusting directions in two continuous cycles are the same, increasing the length of the preset step length. And if the object distance adjusting directions in two continuous cycles are opposite, reducing the length of the preset step length.
In one possible implementation, the calculating the image sharpness of the initial image, each first image, and each second image separately comprises: respectively calculating the image definition of the initial image, each first image and each second image based on the following formulas:
wherein, c i The gray value of each pixel of the image, P is the total pixel number of the image, and s is the image definition.
In a possible implementation manner, after the acquiring the initial image, the method further includes: an image sharpness is calculated based on the initial image. Accordingly, acquiring the first image includes: and if the image definition of the initial image is lower than a preset threshold value, acquiring a first image.
Fig. 6 is a schematic diagram of an electronic device provided in an embodiment of the present invention. As shown in fig. 6, the electronic apparatus 6 of this embodiment includes: a processor 60, a memory 61 and a computer program 62 stored in said memory 61 and executable on said processor 60. The processor 60, when executing the computer program 62, implements the steps in the embodiments of the focus following method of the various thermal reflection microscopy thermography systems described above, such as steps S1-S8 shown in fig. 2. Alternatively, the processor 60, when executing the computer program 62, implements the functions of the modules/units in the above-mentioned device embodiments, such as the functions of the modules 51 to 58 shown in fig. 5.
Illustratively, the computer program 62 may be partitioned into one or more modules/units, which are stored in the memory 61 and executed by the processor 60 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing certain functions, which are used to describe the execution of the computer program 62 in the electronic device 6. For example, the computer program 62 may be divided into the modules 51 to 58 shown in fig. 5.
The electronic device 6 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing device. The electronic device 6 may include, but is not limited to, a processor 60, a memory 61. Those skilled in the art will appreciate that fig. 6 is merely an example of an electronic device 6, and does not constitute a limitation of the electronic device 6, and may include more or fewer components than shown, or some of the components may be combined, or different components, e.g., the electronic device may also include an input-output device, a network access device, a bus, etc.
The Processor 60 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or a memory of the electronic device 6. The memory 61 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the electronic device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 61 is used for storing the computer program and other programs and data required by the electronic device. The memory 61 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/electronic device and method may be implemented in other ways. For example, the above-described apparatus/electronic device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the method of the embodiments described above may be implemented by a computer program, which may be stored in a computer readable storage medium and used by a processor to implement the steps of the embodiments of the focus following method of the thermal reflection microscopy thermal imaging system described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A focus tracking method for a thermal reflection microscopy thermal imaging system, comprising:
acquiring an initial image, wherein the initial image is an image acquired at an initial object distance;
acquiring a first image, wherein the first image is acquired after the object distance is adjusted to a first object distance from an initial object distance, and the first object distance is the initial object distance minus a preset step length;
acquiring a second image, wherein the second image is acquired after the object distance is adjusted from the first object distance to the initial object distance;
repeatedly executing the first image acquisition and the second image acquisition to obtain a plurality of groups of first images and second images;
respectively calculating the image definition of the initial image, each first image and each second image;
if the first average definition is larger than the second average definition, adjusting the object distance to the initial object distance minus a preset step length; the first average definition is an average value of image definitions of the first images, and the second average definition is an average value of image definitions of the initial images and the second images;
if the first average definition is smaller than or equal to the second average definition, adjusting the object distance to the initial object distance plus a preset step length;
and taking the adjusted object distance as an initial object distance to circularly acquire images, calculating average definition and adjusting the object distance.
2. The method of tracking a thermal reflectance microscopy thermography system according to claim 1 further comprising, prior to said acquiring an initial image:
and acquiring a first step length with a preset fixed length as a preset step length, wherein the first step length is smaller than the focal depth of the heat reflection micro thermal imaging system.
3. The method of tracking a thermal reflectance microscopy thermography system according to claim 1 further comprising, prior to said acquiring an initial image:
acquiring a second step length and a third step length, wherein the second step length is smaller than the focal depth of the thermal reflection microscopic thermal imaging system; the third step size is greater than the second step size;
when a tested piece is tested at constant temperature by adopting a heat reflection microscopic thermal imaging system, taking the second step length as a preset step length;
and when the tested piece is subjected to temperature change test by adopting a heat reflection microscopic thermal imaging system, taking the third step length as a preset step length, wherein the third step length is in direct proportion to the amplitude of the temperature change.
4. The focus tracking method of the thermal reflection microscopy thermal imaging system according to claim 1, wherein after the cyclically acquiring the images with the adjusted object distance as the initial object distance, calculating the average definition and performing the object distance adjustment, further comprising:
if the adjustment directions of the object distance in two continuous cycles are the same, increasing the length of a preset step length;
and if the object distance adjusting directions in two continuous cycles are opposite, reducing the length of the preset step length.
5. The method of tracking a thermal reflectance microscopy thermography system according to claim 1 wherein said separately calculating the image sharpness of the initial image, each of the first images and each of the second images comprises:
respectively calculating the image definition of the initial image, each first image and each second image based on the following formulas:
wherein, c i Is the gray value of each pixel of the image, P is the total number of pixels of the image, and s is the image definition.
6. The method of tracking a thermal reflectance microscopy thermal imaging system according to claim 1 further comprising, after said acquiring an initial image:
calculating an image sharpness based on the initial image;
correspondingly, the acquiring the first image comprises:
and if the image definition of the initial image is lower than a preset threshold value, acquiring a first image.
7. A focus following device of a thermal reflection microscopic thermal imaging system is characterized by comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring an initial image, and the initial image is an image acquired at an initial object distance;
the second acquisition module is used for acquiring a first image, wherein the first image is acquired after the object distance is adjusted to a first object distance from the initial object distance, and the first object distance is the preset step length subtracted from the initial object distance;
the third acquisition module is used for acquiring a second image, wherein the second image is acquired after the object distance is adjusted from the first object distance to the initial object distance;
the first circulation module is used for repeatedly executing the acquisition of the first image and the acquisition of the second image to obtain a plurality of groups of first images and second images;
the definition calculating module is used for calculating the image definition of the initial image, each first image and each second image respectively;
the first adjusting module is used for adjusting the object distance to the initial object distance minus a preset step length if the first average definition is larger than the second average definition; the first average definition is an average value of image definitions of the first images, and the second average definition is an average value of the image definitions of the initial image and the second images;
the second adjusting module is used for adjusting the object distance to the initial object distance plus a preset step length if the first average definition is smaller than or equal to the second average definition;
and the second circulation module is used for circularly acquiring images by taking the adjusted object distance as the initial object distance, calculating the average definition and adjusting the object distance.
8. The focus tracking apparatus of a thermal reflectance microscopy thermal imaging system according to claim 7, further comprising:
and the step length acquisition module is used for acquiring a first step length with a preset fixed length as a preset step length before the initial image is acquired, wherein the first step length is smaller than the focal depth of the heat reflection micro thermal imaging system.
9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor when executing the computer program implements the steps of the focus following method of the thermal reflectance microscopy thermography system according to any of the claims 1 to 6 above.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method for tracking a thermal reflectance microscopy thermography system as claimed in any of the claims 1 to 6 above.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211324332.6A CN115657264A (en) | 2022-10-27 | 2022-10-27 | Focus following method and device of heat reflection microscopic thermal imaging system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211324332.6A CN115657264A (en) | 2022-10-27 | 2022-10-27 | Focus following method and device of heat reflection microscopic thermal imaging system and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115657264A true CN115657264A (en) | 2023-01-31 |
Family
ID=84993456
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211324332.6A Pending CN115657264A (en) | 2022-10-27 | 2022-10-27 | Focus following method and device of heat reflection microscopic thermal imaging system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115657264A (en) |
-
2022
- 2022-10-27 CN CN202211324332.6A patent/CN115657264A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107084794B (en) | Flame three-dimensional temperature field measuring system and method based on light field layered imaging technology | |
| US8711275B2 (en) | Estimating optical characteristics of a camera component using sharpness sweep data | |
| CN109813435B (en) | Static light reflection micro thermal imaging method and device and terminal equipment | |
| CN108362226B (en) | Double four-step phase shift method for improving phase measurement precision of image overexposure area | |
| EA011851B1 (en) | Method and device for measuring translation surface | |
| CN109587475B (en) | Method for measuring dynamic resolution of digital camera | |
| US11512946B2 (en) | Method and system for automatic focusing for high-resolution structured light 3D imaging | |
| Gao et al. | Stereo camera calibration for large field of view digital image correlation using zoom lens | |
| CN113624358B (en) | Three-dimensional displacement compensation method and control device for photothermal reflection microscopic thermal imaging | |
| CN115657264A (en) | Focus following method and device of heat reflection microscopic thermal imaging system and electronic equipment | |
| CN114993505A (en) | Method and device for realizing automatic refocusing of thermal reflection imaging temperature measurement | |
| TWI760417B (en) | Three-dimensional shape measurement method using scanning white interference microscope | |
| CN115086549B (en) | Shooting method, shooting device, computer equipment and storage medium | |
| Kang et al. | Evaluating binary defocusing quantitatively in real-time for fringe projection profilometry | |
| Iovenitti et al. | Three-dimensional measurement using a single image | |
| Chung | Improved least-squares method for phase-to-height relationship in fringe projection profilometry | |
| CN113655610B (en) | Automatic focusing method and control device for photothermal reflection microscopic thermal imaging | |
| JP7191632B2 (en) | Eccentricity measurement method | |
| Kim et al. | A novel depth-from-focus-based measurement system for the reconstruction of surface morphology with depth discontinuity | |
| Wei et al. | Global depth from defocus with fixed camera parameters | |
| Feng et al. | Automatic exposure control method for 3D shape measurement of specular surface based on digital fringe projection | |
| Heikkinen et al. | Perspective error reduction in 2D Digital Image Correlation measurements by combination with Defocused Speckle Imaging | |
| US11380015B2 (en) | Method for the optical determination of an intensity distribution | |
| Scott et al. | Real-time photon beam localization methods using high-resolution imagers and parallel processing using a reconfigurable system | |
| Lazdinas et al. | An analysis of the optical system of a length measurement comparator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |