CN114608451A - Neural network-based off-plane displacement measurement method and device - Google Patents

Abstract

The invention discloses an off-plane displacement measurement method and device based on a neural network. The method includes: acquiring characteristic information of reflection surface interference fringes and diffuse reflection interference fringes; according to the characteristics of the reflection surface interference fringes and diffuse reflection interference fringes According to the single-point off-plane displacement measurement system, the reflection surface interference experiment and the diffuse reflection surface interference experiment are carried out, and the displacement end is translated to obtain the fringe image; according to the fringe image, the The back-propagation neural network identifies the magnitude of the out-of-plane displacement in one cycle; according to the fringe image, the convolutional neural network pair is used to determine the out-of-plane displacement direction. The invention can improve processing efficiency and reduce labor cost, and can be widely used in the technical field of off-plane displacement measurement.

Description

A kind of out-of-plane displacement measurement method and device based on neural network

technical field

The invention relates to the technical field of out-of-plane displacement measurement, in particular to an out-of-plane displacement measurement method and device based on a neural network.

Background technique

The rapid development of scientific research and engineering technology has put forward higher requirements for high resolution, large range and real-time performance of out-of-plane displacement measurement. Among the non-contact measurement methods, high-sensitivity laser interferometry is widely used. The fringe division and counting method, the phase shift method, the time phase estimation, and the Fourier transform method are often used for fringe pattern analysis of out-of-plane measurements. The fringe subdivision technique in the laboratory environment achieves a measurement accuracy of the order of one hundredth of a wavelength, but the range is affected by speckle de-correlation. The related art proposes a vibration measurement method based on interference fringe subdivision technology, which achieves a measurement accuracy of 1/400 wavelength. The time-series fringe image phase extraction method can reach the measurement range of several hundreds of microns, but the real-time performance is not high. A measurement system based on laser feedback interferometry was proposed later, with a resolution of 0.51 nm, a measurement range of 850 μm, and a positioning accuracy of 5 nm in 2 minutes. It can be seen that it is still a challenge to achieve high-resolution, large-scale, real-time out-of-plane displacement measurement at the same time.

With the wide application and rapid development of out-of-plane displacement measurement, an effective, high-resolution, large-scale, real-time out-of-plane displacement measurement method is increasingly required. However, the existing out-of-plane displacement measurement methods have various defects. Specifically:

(1) It is difficult to take into account the requirements of high resolution, real-time performance and large range.

(2) It does not reach the level of high machine processing, the efficiency is low, and a large amount of manual participation is required.

SUMMARY OF THE INVENTION

In view of this, embodiments of the present invention provide an efficient and low labor cost, out-of-plane displacement measurement method and device based on a neural network.

One aspect of the present invention provides an out-of-plane displacement measurement method based on a neural network, comprising:

Obtain the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes;

According to the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes, a single-point off-plane displacement measurement system is constructed;

According to the single-point off-plane displacement measurement system, a reflection surface interference experiment and a diffuse reflection surface interference experiment are performed, and the displacement end is translated to obtain a fringe image;

According to the fringe image, a back-propagation neural network is used to identify the magnitude of the out-of-plane displacement within one cycle;

From the fringe image, a convolutional neural network pair is used to determine the out-of-plane displacement direction.

Optionally, the single-point out-of-plane displacement measurement system includes a Michelson interferometer, a piezoelectric ceramic nano-translation stage, a CCD camera, and a computing device;

In the single-point off-plane displacement measurement system, the emitted laser is divided into two paths through a beam splitting prism, the optical path at the displacement end returns through the first reflecting mirror, and the optical path at the reference end returns through the second reflecting mirror;

Wherein, the first reflecting mirror and the second reflecting mirror are coupled on the piezoelectric ceramic nano-translation stage.

Optionally, performing a reflection surface interference experiment and a diffuse reflection surface interference experiment according to the single-point off-plane displacement measurement system, and translating the displacement end to obtain a fringe image, including:

The normalized convolution value of the interferogram sub-area at any time and the interferogram sub-area at the initial moment is used as the reference value of the degree of similarity;

The displacement tracking is performed according to the nonlinear mapping relationship between the normalized convolution value and the displacement, and the displacement of the measured object at any time is obtained.

Optionally, performing a reflection surface interference experiment and a diffuse reflection surface interference experiment according to the single-point off-plane displacement measurement system, and translating the displacement end to obtain a fringe image, further comprising:

In one cycle, the fringe image is divided into two states according to the gray matrix of the central sub-area of the fringe image, moving to the left for half a cycle and moving to the right for half a cycle, and transforming the displacement direction judgment problem into an image recognition problem .

Optionally, according to the fringe image, using a back-propagation neural network to identify the magnitude of the out-of-plane displacement in one cycle, including:

A five-layer BP neural network is used to approximate the nonlinear mapping relationship between the normalized convolution value and the displacement;

Calculate the normalized convolution value of the interferogram at each moment and the interferogram at the initial moment, use the normalized convolution value as the input value of the BP neural network, and use the corresponding known displacement as the output value of the network. Network training; wherein, the learning rate is set to 0.001 in the process of the training;

Wherein, the input layer of the five-layer BP neural network contains one neuron, which corresponds to the input normalized convolution value; the number of neurons contained in the three hidden layers of the five-layer BP neural network is 100, respectively. 200, 200; the output layer of the five-layer BP neural network contains a neuron, corresponding to the displacement of the output; any two layers are fully connected, the initial value of the connection parameter weight is a random number of Gaussian distribution, and the initial value of the bias term is a constant.

Optionally, the convolutional neural network includes three convolutional layers, three pooling layers, a fully connected layer and a softmax function layer, and the convolutional neural network is used to determine the out-of-plane relationship according to the fringe image. Displacement directions, including:

Taking the sub-region of the interference map as the input value of the network, when passing through the first convolution layer, the size of the convolution kernel is (3, 3, 1, 32), and the image becomes a three-dimensional matrix of size 600×600×32;

Passing the three-dimensional matrix through a maximum pooling layer, the step size in the length and width directions is 2, and the size of the three-dimensional matrix is changed to 300×300×32;

After the second convolution layer, the size of the convolution kernel is (3, 3, 32, 64), and the size of the three-dimensional matrix is changed to 300×300×64;

After a maximum pooling layer with a step size of 2 in both the length and width directions, the size of the three-dimensional matrix becomes 150×150×64;

After the third convolution layer, the size of the convolution kernel is (3, 3, 64, 64), and the size of the three-dimensional matrix becomes 300×300×64;

After the max pooling layer, the 3D matrix size becomes 75×75×64;

After a fully connected layer with 1000 neurons and a softmax function layer;

After passing through the output layer with the number of neurons being 2, the output is a probability distribution, and the probability distribution is used to represent the probability that the off-plane displacement direction is left or right;

The out-of-plane displacement direction is determined according to the direction with a larger probability value in the probability distribution.

Another aspect of the embodiments of the present invention also provides an out-of-plane displacement measurement device based on a neural network, including:

The first module is used to obtain the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes;

The second module is used to construct a single-point off-plane displacement measurement system according to the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes;

The third module is used to perform the reflection surface interference experiment and the diffuse reflection surface interference experiment according to the single-point off-plane displacement measurement system, and translate the displacement end to obtain fringe images;

The fourth module is used to identify the off-plane displacement size within one cycle by using a back-propagation neural network according to the fringe image;

The fifth module is used for determining the out-of-plane displacement direction by using the convolutional neural network pair according to the fringe image.

Another aspect of the embodiments of the present invention further provides an electronic device, including a processor and a memory;

the memory is used to store programs;

The processor executes the program to implement the method as described above.

Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, where the storage medium stores a program, and the program is executed by a processor to implement the aforementioned method.

Another aspect of the embodiments of the present invention also provides a computer program product, including a computer program, which implements the aforementioned method when the computer program is executed by a processor.

The embodiment of the present invention first obtains the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes; builds a single-point off-plane displacement measurement system according to the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes; The point-off-plane displacement measurement system performs the reflection surface interference experiment and the diffuse reflection surface interference experiment, and translates the displacement end to obtain a fringe image; according to the fringe image, the back-propagation neural network is used to identify the out-of-plane displacement in one cycle; From the fringe image, a convolutional neural network pair is used to determine the out-of-plane displacement direction. The present invention can improve processing efficiency and reduce labor cost.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a flow chart of the overall steps provided by an embodiment of the present invention;

2 is a schematic diagram of a diffuse reflection surface interferometric measurement system provided by an embodiment of the present invention;

3 is a schematic diagram of a single-point off-plane displacement measurement system provided by an embodiment of the present invention;

4 is a schematic diagram of a normalized convolution value-displacement curve provided by an embodiment of the present invention;

5 is a schematic diagram of a BP network provided by an embodiment of the present invention;

FIG. 6 is a schematic diagram of a CNN network provided by an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

In view of the existing problems in the prior art, an aspect of the present invention provides a method for measuring out-of-plane displacement based on a neural network, as shown in FIG. 1 , the method includes the following steps:

Obtain the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes;

According to the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes, a single-point off-plane displacement measurement system is constructed;

According to the single-point off-plane displacement measurement system, a reflection surface interference experiment and a diffuse reflection surface interference experiment are performed, and the displacement end is translated to obtain a fringe image;

According to the fringe image, a back-propagation neural network is used to identify the magnitude of the out-of-plane displacement within one cycle;

From the fringe image, a convolutional neural network pair is used to determine the out-of-plane displacement direction.

Optionally, the single-point out-of-plane displacement measurement system includes a Michelson interferometer, a piezoelectric ceramic nano-translation stage, a CCD camera, and a computing device;

In the single-point off-plane displacement measurement system, the emitted laser is divided into two paths through a beam splitting prism, the optical path at the displacement end returns through the first reflecting mirror, and the optical path at the reference end returns through the second reflecting mirror;

Wherein, the first reflecting mirror and the second reflecting mirror are coupled on the piezoelectric ceramic nano-translation stage.

Optionally, performing a reflection surface interference experiment and a diffuse reflection surface interference experiment according to the single-point off-plane displacement measurement system, and translating the displacement end to obtain a fringe image, including:

The normalized convolution value of the interferogram sub-area at any time and the interferogram sub-area at the initial moment is used as the reference value of the degree of similarity;

The displacement tracking is performed according to the nonlinear mapping relationship between the normalized convolution value and the displacement, and the displacement of the measured object at any time is obtained.

Optionally, performing a reflection surface interference experiment and a diffuse reflection surface interference experiment according to the single-point off-plane displacement measurement system, and translating the displacement end to obtain a fringe image, further comprising:

In one cycle, the fringe image is divided into two states according to the gray matrix of the central sub-area of the fringe image, moving to the left for half a cycle and moving to the right for half a cycle, and transforming the displacement direction judgment problem into an image recognition problem .

Optionally, according to the fringe image, using a back-propagation neural network to identify the magnitude of the out-of-plane displacement in one cycle, including:

A five-layer BP neural network is used to approximate the nonlinear mapping relationship between the normalized convolution value and the displacement;

Calculate the normalized convolution value of the interferogram at each moment and the interferogram at the initial moment, use the normalized convolution value as the input value of the BP neural network, and use the corresponding known displacement as the output value of the network. Network training; wherein, the learning rate is set to 0.001 in the process of the training;

Wherein, the input layer of the five-layer BP neural network contains one neuron, which corresponds to the input normalized convolution value; the number of neurons contained in the three hidden layers of the five-layer BP neural network is 100, respectively. 200, 200; the output layer of the five-layer BP neural network contains a neuron, corresponding to the displacement of the output; any two layers are fully connected, the initial value of the connection parameter weight is a random number of Gaussian distribution, and the initial value of the bias term is a constant.

Optionally, the convolutional neural network includes three convolutional layers, three pooling layers, a fully connected layer and a softmax function layer, and the convolutional neural network is used to determine the out-of-plane relationship according to the fringe image. Displacement directions, including:

Taking the sub-region of the interference map as the input value of the network, when passing through the first convolution layer, the size of the convolution kernel is (3, 3, 1, 32), and the image becomes a three-dimensional matrix of size 600×600×32;

Passing the three-dimensional matrix through a maximum pooling layer, the step size in the length and width directions is 2, and the size of the three-dimensional matrix is changed to 300×300×32;

After the second convolution layer, the size of the convolution kernel is (3, 3, 32, 64), and the size of the three-dimensional matrix is changed to 300×300×64;

After a maximum pooling layer with a step size of 2 in both the length and width directions, the size of the three-dimensional matrix becomes 150×150×64;

After the third convolution layer, the size of the convolution kernel is (3, 3, 64, 64), and the size of the three-dimensional matrix becomes 300×300×64;

After the max pooling layer, the 3D matrix size becomes 75×75×64;

After a fully connected layer with 1000 neurons and a softmax function layer;

After passing through the output layer with the number of neurons being 2, the output is a probability distribution, and the probability distribution is used to represent the probability that the off-plane displacement direction is left or right;

The out-of-plane displacement direction is determined according to the direction with a larger probability value in the probability distribution.

Another aspect of the embodiments of the present invention also provides an out-of-plane displacement measurement device based on a neural network, including:

The first module is used to obtain the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes;

The second module is used to construct a single-point off-plane displacement measurement system according to the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes;

The third module is used to perform the reflection surface interference experiment and the diffuse reflection surface interference experiment according to the single-point off-plane displacement measurement system, and translate the displacement end to obtain fringe images;

The fourth module is used to identify the off-plane displacement size within one cycle by using a back-propagation neural network according to the fringe image;

The fifth module is used for determining the out-of-plane displacement direction by using the convolutional neural network pair according to the fringe image.

Another aspect of the embodiments of the present invention further provides an electronic device, including a processor and a memory;

the memory is used to store programs;

The processor executes the program to implement the method as described above.

Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, where the storage medium stores a program, and the program is executed by a processor to implement the aforementioned method.

Another aspect of the embodiments of the present invention also provides a computer program product, including a computer program, which implements the aforementioned method when the computer program is executed by a processor.

The specific implementation principle of the present invention will be described in detail below in conjunction with the accompanying drawings:

A method for streak motion detection and displacement measurement based on neural network proposed by the present invention is applied to various out-of-plane displacement measurement work, and the specific implementation principles can be divided into:

a. According to the optical principle, the light intensity and other characteristics of the reflection surface interference fringes and the diffuse reflection interference fringes are sorted out, so as to facilitate the design and conduct of subsequent experiments.

b. An off-plane displacement measurement system is designed, and the experimental procedures are designed for reflection surface interference and diffuse reflection surface interference respectively.

c. Carry out the experiment, after the displacement end M2 is translated, the image of the moved interference fringes and the gray matrix of the central sub-area are acquired, and the acquired interference pattern sub-area at any time and the initial moment interference pattern sub-area ( The normalized convolution value Conv of 600 × 600 pixels at the center of the image is used as the reference value of similarity.

d. Using back-propagation neural network (BP) to identify the magnitude of the out-of-plane displacement in one cycle.

e. The judgment of the displacement direction is regarded as a binary classification problem in image recognition, and a convolutional neural network (CNN) is used to learn and judge it.

In the step (a), the optical principles and characteristics of the reflection surface interference fringes and the diffuse reflection interference fringes are sorted out. details as follows:

1) The formation principle of interference fringes on the reflecting surface

When the two light waves meet the same frequency, the same vibration direction, and the phase difference stability condition are superimposed when they meet in space, the light intensity is re-stabilized and unevenly distributed, and the phenomenon of bright and dark or colored fringes is called interference. A light wave as an electromagnetic wave can be described as:

是电场矢量，

是电场复振幅；

是磁场矢量，

是磁场复振幅，w，

分别代表了角速度和初始位相，p代表空间中的任意位置。而本文所谈及的光扰动皆受光波中电场的影响，因此讨论的是电场矢量的振动，即某一时刻的光矢量仅表示为：here,

is the electric field vector,

is the complex amplitude of the electric field;

is the magnetic field vector,

is the complex amplitude of the magnetic field, w,

represent the angular velocity and initial phase, respectively, and p represents any position in space. The light disturbances mentioned in this article are all affected by the electric field in the light wave, so the discussion is about the vibration of the electric field vector, that is, the light vector at a certain moment is only expressed as:

When light propagates in a certain direction, the scalar wave function of its plane monochromatic light wave can be expressed as:

为初始位相。当两束振动方向相同的光波在空间传播：Among them, r, k represent the propagation distance and wave number, respectively,

is the initial phase. When two light waves with the same vibration direction propagate in space:

Using the complex amplitude method, it is expressed as:

When encounter superposition occurs, the superimposed light intensity is as follows:

代表了两束光在p点的相位差，根据

光强可变换为：make

represents the phase difference between the two beams of light at point p, according to

The light intensity can be transformed into:

I(P)=E ₁₀ ² (p)+E ₂₀ ² (p)+2E ₁₀ E ₂₀ cosθ(p)

Among them, E ₁₀ ² (p) and E ₂₀ ² (p) respectively represent the light intensity of the two light waves at point p. Therefore, the superposition of two coherent beams causes not simply the addition of light intensities, but an additional term of 2 ₁₀ E ₂₀ cosθ(p), where θ(p) is related to the propagation distance of the point light source and the initial phase.

at this time

θ(p)=2nπ(n=0,±1,±2…)

When the light intensity takes the maximum value, the two vector vibrations are in the same pace, completely superimposed, and reinforce each other. The intensity is:

I(P)=E ₁₀ ² (p)+E ₂₀ ² (p)+2E ₁₀ E ₂₀ =(E ₁₀ +E ₂₀ ) ²

and when

θ(p)=(2n+1)π(n=0,±1,±2…)

The light intensity takes the minimum value, the vibration steps are completely opposite, and they weaken each other. At this time, the intensity is:

I(P)=E ₁₀ ² (p)+E ₂₀ ² (p)-2E ₁₀ E ₂₀ =(E ₁₀ -E ₂₀ ) ²

Therefore, the light intensity of the spatial point in the overlapping area can be obtained by calculation. The two beams of light that meet the coherence conditions at the receiving end interfere to form bright and dark interference fringes, which are recorded by the CCD. At this time, the shape of the interference fringes, that is, the light intensity of each point in the overlapping area, is determined by the respective optical path difference.

2) The formation principle of diffuse reflection interference fringes

Considering that the measured object is a diffuse reflection surface, replace M2 and M3 with frosted glass, and other optical path components remain unchanged. As shown in Figure 2, adjust the CCD position and simultaneously image the speckle field on the M2 and M3 surfaces. At this time, the diffuse reflection is scattered. The speckle fields are superimposed on each other. In Figure 2, LASER stands for laser; Dis stands for off-plane displacement to be obtained; M2 (Object) stands for ground glass; BS stands for half mirror; SF stands for spatial filter; LENS stands for prism; Compensation stands for compensation end displacement; M3 stands for ground glass; CCD stands for camera;

When a laser is irradiated on the surface of a diffusely reflective object, the randomly distributed light and dark spots formed by interference on the surface and the front space are called laser speckles. Laser speckles can be described as:

分别代表了复振幅，振幅和相位。ξ也可以表示为where ξ, A,

represent the complex amplitude, amplitude and phase, respectively. ξ can also be expressed as

In the electron speckle interference (ESPI) experiment, the light and dark interference fringes can also be obtained by subtracting the speckle fields at different times. The mathematical expression of the interference fringes at this time is:

Among them, I _a -I _b represent the light intensity subtraction of the speckle field at different times a and b, and z ₁ and z ₂ represent the two ends of the Michelson interferometer M2 and M3 (imaged at the same position on the CCD).

可得到：because

available:

上式可以改写为When the speckle patterns are correlated at two moments at the observation position, that is,

The above formula can be rewritten as

分别代表了散斑干涉图在不同时刻(位移前后)的位相变化，

代表干涉散斑图位相变化之差，为慢变项，而

是快变项，因此上式表明散斑(快变项)会受到条纹(慢变项)的调制。in

respectively represent the phase change of the speckle interferogram at different times (before and after displacement),

represents the difference between the phase changes of the interference speckle pattern, which is a slow variable term, and

is a fast variable term, so the above equation shows that the speckle (fast variable term) will be modulated by fringes (slow variable term).

(m＝0,±1,±2…)；即位移引起的位相变化刚好是2π的整数倍时When the pixel phase is satisfied

(m=0,±1,±2...); that is, when the phase change caused by the displacement is just an integer multiple of 2π

Therefore, the light intensity of these pixels does not change at different times (before and after displacement), and they are called dark spots.

It can be seen from the above analysis that when there is no displacement, the interference speckle pattern after subtraction will be all dark spots. When displacement occurs, the light intensity will change, and bright and dark spots will appear. However, it is difficult to extract the exact displacement size and direction only from the light and dark interference field. Therefore, carrier fringes are introduced here, and the magnitude and direction of the displacement are judged by the movement of the carrier fringes when the displacement occurs, and the light of the interference field after the carrier fringes is introduced. Strong as follows:

代表了载波条纹位相。因此在漫反射面干涉测量系统引入载波条纹，并实时将此时的散斑场与原始散斑场相减后的图像显示在编写软件的图片显示界面，图像的载波条纹上每一点的光强由对应像素点的位移决定。in,

represents the carrier fringe phase. Therefore, a carrier fringe is introduced into the diffuse reflection surface interferometry system, and the image obtained by subtracting the speckle field at this time from the original speckle field is displayed on the picture display interface of the writing software in real time, and the light intensity of each point on the carrier fringe of the image is displayed. It is determined by the displacement of the corresponding pixel point.

In the step (b), a single-point off-plane displacement measurement system is designed, as shown in FIG. 3 . In Figure 3, LASER represents laser; SF represents spatial filter; Dis represents off-plane displacement to be measured; M2 (Object) represents mirror; LENS represents prism; BS represents half mirror; Compensation represents compensation end displacement; M3 represents Mirror; CCD stands for camera.

The measurement system consists of a Michelson interferometer, a piezoelectric ceramic (PZT) nano-translation stage, a CCD camera, and a computing device. As shown in Figure 3, a beam of laser is divided into two paths through a beam splitter prism, the optical path at the displacement end returns through mirror M2, and the optical path at the reference end returns through mirror M3, which is coupled to the PZT nano-translation stage (the model is Xinming , the range is 210 microns). Considering that the measured object is a mirror surface, the mirror M2 is used as the measured object, and it is coupled to the nano-translation stage (model PI: P-622.1CL, the range is 800 microns, and the positioning accuracy is 10 nanometers). The two paths of light are combined by the beam splitting prism BS and recorded by the CCD at the receiving end.

In the step (c), an experiment is performed. After the displacement end M2 is translated, the image of the shifted interference fringes and the grayscale matrix of the central sub-region are acquired, and the interference map sub-region at any time and the interference map at the initial time are obtained. The normalized convolution value Conv of the sub-region (600×600 pixels at the center of the image) is used as the reference value of similarity:

Because M2 only translates along the optical axis, the fringes move, but the fringe spacing and angle do not change, that is, only the phase changes. When there is no translation, the fringe state does not change, the interferogram at any time is consistent with the interferogram at the initial time, and the similarity is the largest. In half a cycle, as the displacement gradually increases, the similarity gradually decreases. Therefore, The normalized convolution value Conv of the interferogram sub-area at any time and the interferogram sub-area at the initial time (600×600 pixels at the center of the image) is used as the reference value of the degree of similarity, which is defined as follows:

分别代表了当前状态和初始状态干涉条纹图的每个像素点的灰度值。Here M represents the total number of pixels in the sub-region,

respectively represent the gray value of each pixel of the current state and the initial state of the interference fringe map.

Figure 4 shows the relationship between the normalized convolution value and the displacement. It can be seen that the normalized convolution value in one cycle has symmetry. When the stripe moves half a cycle, the convolution value is the smallest. When the stripe moves one cycle, the convolution value is the smallest. The value returns to the maximum value. Therefore, the accurate tracking displacement can be given according to the nonlinear mapping relationship between the normalized convolution value and the displacement, so as to obtain the displacement of the measured object M2 at any time, avoiding complex phase calculation. Similarly, in one cycle, the fringe image is divided into two states according to the gray matrix of the central sub-area of the fringe image, moving to the left for half a cycle and moving to the right for half a cycle, and transforming the displacement direction judgment problem into an image Identify the problem.

In the step (d), a back-propagation neural network (BP) is used to identify the magnitude of the out-of-plane displacement within one cycle.

Universal approximation theory shows that a 3-layer perceptron neural network can approximate a multi-dimensional continuous nonlinear function with arbitrary precision. In this paper, a five-layer BP neural network is used to approximate the nonlinear mapping relationship between the normalized convolution value and the displacement. As shown in Figure 5, the input layer contains one neuron, which corresponds to the normalized convolution value of the input, the number of neurons contained in the three hidden layers are 100, 200, and 200, respectively, and the output layer contains one neuron, corresponding to the output displacement value. Fully connected between two adjacent layers. The initial value of the connection parameter weight is a random number from a Gaussian distribution, and the initial value of the bias term is a constant. Considering that the essence of the problem is a regression problem, the mean square error (MSE value) is used as the loss function of the network.

For a Michelson interferometer, the measured object is shifted by half a wavelength and the fringes are shifted by one period. This experiment uses a helium-neon laser with a laser wavelength of 632.8nm, so one cycle of fringe movement corresponds to a displacement of 316.4nm of the measured object. In half a cycle, the larger the displacement, the smaller the convolution value. Move the measured object by 10nm each time until it moves 158.2nm (1/4 wavelength), calculate the normalized convolution value of the interferogram at each moment and the interferogram at the initial moment, and use it as the input of the BP neural network value, the corresponding known displacement is used as the output value of the network for network training, and the learning rate is set to 0.001 during the training process.

In the step (e), the judgment of the displacement direction is regarded as a binary classification problem in image recognition, and a convolutional neural network (CNN) is used to learn and judge it.

As shown in Figure 6, the convolutional neural network used in this paper contains three convolutional layers, three pooling layers, a fully connected layer and a softmax function layer. Taking the sub-region of the interference map (600 × 600 pixels in size) as the input value of the network, when passing through the first convolutional layer, the size of the convolution kernel is (3, 3, 1, 32), and the image becomes 600 × 600 × A three-dimensional matrix of size 32, after a maximum pooling layer, the step size in the length and width directions is 2, and the matrix size becomes 300 × 300 × 32. Then through the second convolution layer, the size of the convolution kernel is (3, 3, 32, 64), the image becomes a three-dimensional matrix of size 300 × 300 × 64, and a maximum pool with a step size of 2 in the length and width directions is passed. Layer, the matrix size becomes 150×150×64. The size of the convolution kernel of the third convolutional layer is (3, 3, 64, 64), and the image becomes a three-dimensional matrix of size 300 × 300 × 64. Similarly, after the maximum pooling layer, the size is reduced to 75 × 75 ×64. Then through a fully connected layer with 1000 neurons and a softmax function layer, and finally through the output layer with 2 neurons, the output is a probability distribution, representing the probability of left and right respectively, and the final displacement The direction of is given according to the one with the larger probability value. After layer-by-layer convolution, the features of the interference pattern are gradually extracted and combined, and the finally extracted features are classified according to certain classifier rules. The cross entropy is used as the loss function of the network to identify the displacement direction of the image.

To sum up, the present invention proposes a fringe motion detection method based on a neural network to realize displacement compensation measurement, and achieves out-of-plane displacement measurement taking into account high resolution, large range and real-time performance. Compared with the traditional out-of-plane displacement measurement method, this method can take into account multiple indicators, such as high resolution, large range and real-time performance, and at the same time, the machine processing is carried out through the neural network method, which improves the overall measurement efficiency.

In order to verify the effectiveness of the tracking measurement algorithm, a reflection surface interference experiment and a diffuse reflection surface interference experiment are respectively carried out in the embodiment of the present invention.

(1) Reflector interference experiment

In this example, a reflection surface interference experiment system based on Michelson interference is used. The light source is a helium-neon laser (HNL210L). A laser beam is emitted from the helium-neon laser and expanded into a group of parallel lights after passing through a spatial filter. The half mirror is divided into two beams, which pass through the mirror respectively and converge on the CCD target surface. The out-of-plane displacement of the measured end is provided by the nano-translation stage and applied to the mirror of the measured end, and the compensation displacement of the compensation end is provided by the PZT nano-translation stage and applied to the mirror of the compensation end. The digital signal of the CCD end is transmitted to the computing device in real time to monitor the movement size and direction of the interference fringes in real time. Once the tracking displacement is detected to be greater than the set threshold (10nm in this paper), the tracking command is triggered, so that the interference image remains unchanged. . The CCD industrial camera used in this experiment is from Basler Company, the model is Basler-ace 1600-20gm (resolution is 1600×1200 pixels, and the maximum frame rate is 20 frame/s). The computer CPU is Intel i5-4460, and the graphics card model is GeForce GTX1080. The tracking algorithm, including the judgment of displacement size and direction, and sending commands to actuate the PZT nano-translation stage are all written in Python language.

Considering that the range of the PZT nano-translation stage at the tracking end is 210 microns, set the translation displacement of the measured end to be 40 nm each time, repeat 5000 times, and the interval time is 400 ms.

(2) Diffuse reflection surface interference experiment

In this example, a diffuse reflection surface interference experimental system is built based on Michelson interference. The measured end and the tracking end replace the mirrors with ground glass, adjust the focal length of the CCD camera at the receiving end, and image the surface of the ground glass, so that the diffuse reflection interference speckle fields are mutually Superimposition, the computer software side displays the image after the subtraction of the current superimposed speckle field and the initial superimposed speckle field in real time. After the carrier is applied, bright and dark stripes appear. After the displacement end is translated, the carrier fringes move. Similarly, the software monitors the moving size and direction of the carrier stripe in real time. Once the tracking displacement is detected to be greater than the set threshold (10 nm in this paper), the tracking command is triggered, so that the image remains unchanged. Also set each translational displacement of the measured end to 40nm, repeat 5000 times, and the interval time is 400ms.

In the experiment, it is difficult to ensure that the optical axis of the compensation end and the measured end are completely perpendicular, so a correction coefficient K is introduced to correct the error between the displacement of the compensation end and the displacement of the measured end. The correction coefficient K can be obtained by theoretical derivation of the geometric relationship of the placement position of the experimental device or by calibration experiments. In this paper, the calibration experiment is used to obtain the results. The two systems respectively conduct five sets of calibration experiments, calculate the ratio of the known measured end displacement and the tracking displacement calculated by the compensation end, and obtain the average value to obtain the correction coefficient K. The K value calculated by the diffuse reflection surface interference experiment system is 0.954, and the K value of the reflection surface interference experiment system is 0.912.

The time interval between every two translations set in the experiment was 400ms, and the translations were repeated 5000 times. According to the experimental results, the compensated measured displacement curve is in good agreement with the real displacement curve. The measurement resolution of the device depends on a preset displacement threshold. When the detected displacements were larger than 10 nm, the PZT nanotranslation stage was actuated to restore the fringe pattern to its initial state. When the displacement is less than 10 nm, the state is regarded as unchanged. When the fluctuation is around 10 nm, which corresponds to the measurement resolution, the fluctuation remains stable and does not increase with the increase of displacement. Because the compensation displacement calculation of each step is compared with the initial fringe state, the residual error of the previous step can be compensated in the next step of compensation tracking, so there is no accumulated error. When the error of the error displacement is maintained at about 10 nm, the effectiveness of the tracking process is verified. The errors of both reflector interference experiments and diffuse reflector interference experiments are affected by background noise, including ambient vibration, air disturbance, temperature fluctuation and random CCD noise.

Table 1 shows the data comparison between the tracking measurement displacement and the real displacement during the reflection surface interference experiment and the diffuse reflection surface interference experiment. It can be seen that the relative error is less than 0.5%. It can be seen that for both reflective surface objects and diffuse reflection surface objects, the system can achieve accurate tracking measurement within the maximum stroke of the PZT device (the system is 210 microns). If the displacement of the object continues to increase, the reflection surface interference system is still effective, while the diffuse reflection surface interference system will be de-correlated. At this time, a wider range of measurement can be achieved by segmented measurement, but error accumulation will be introduced.

Table 1

Table 2 shows the time-consuming tracking of each step, where the set program isochronous time is 200ms, including the time required for the movement of the nano-translation stage, the PZT driving time, the mode stabilization time and the time required for image acquisition. The sum of the average calculation time of the displacement size and the average judgment time of the displacement direction is less than 200ms, which shows that the tracking measurement algorithm can realize real-time tracking measurement.

Table 2

Time (ms) Displacement direction judgment Displacement size calculation Program waiting time Diffuse surface 105 63 200 Reflective surface 123 55 200

The resolution of the measurement system is determined by the set displacement threshold. When the displacement of the measured object is greater than the set threshold, the compensation condition is triggered. At this time, the fringe returns to the initial position and the compensation displacement is recorded. However, the displacement threshold is limited by background noise, including environmental vibration, air disturbance, temperature fluctuation and random CCD noise. Excessive noise will affect the accuracy of displacement calculation and even cause displacement calculation errors. Therefore, the resolution can be further improved by effective noise suppression.

The maximum measurement range of this system is only 210μm, which is limited by the maximum stroke of the PZT nano-translation stage. Therefore, the measurement range can be increased by replacing the nano-translation stage with a larger stroke. The tracking time of this system is about 400ms each time, including the time required for the nanotranslation platform to move, the time required for the calculation of the displacement size and direction, the PZT driving time, the mode stabilization time and the image acquisition time. Image acquisition time can be effectively reduced by using a CCD camera with a higher frame rate (20 frames per second here). The fringe settling time can be reduced by using a PZT device with a faster drive response. Computation time can be effectively reduced by improving computer hardware configuration, including CPU and GPU.

The compensation algorithm realizes the off-plane displacement measurement by accumulating each compensation displacement, uses the neural network to process the interference fringe pattern, and outputs the compensation displacement size and direction, which avoids the accurate phase calculation and overcomes the influence of noise to a certain extent.

In some alternative implementations, the functions/operations noted in the block diagrams may occur out of the order noted in the operational diagrams. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/operations involved. Furthermore, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of the various operations are altered and in which sub-operations described as part of larger operations are performed independently.

Furthermore, while the invention is described in the context of functional modules, it is to be understood that, unless stated to the contrary, one or more of the described functions and/or features may be integrated in a single physical device and/or or software modules, or one or more functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to understand the present invention. Rather, given the attributes, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the modules will be within the routine skill of the engineer. Accordingly, those skilled in the art, using ordinary skill, can implement the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are illustrative only and are not intended to limit the scope of the invention, which is to be determined by the appended claims along with their full scope of equivalents.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus.

More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, The scope of the invention is defined by the claims and their equivalents.

The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments, and those skilled in the art can also make various equivalent deformations or replacements on the premise of not violating the spirit of the present invention, These equivalent modifications or substitutions are all included within the scope defined by the claims of the present application.

Claims (10)

1. an out-of-plane displacement measurement method based on neural network, is characterized in that, comprises: Obtain the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes; According to the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes, a single-point off-plane displacement measurement system is constructed; According to the single-point off-plane displacement measurement system, a reflection surface interference experiment and a diffuse reflection surface interference experiment are performed, and the displacement end is translated to obtain a fringe image; According to the fringe image, a back-propagation neural network is used to identify the magnitude of the out-of-plane displacement within one cycle; From the fringe image, a convolutional neural network pair is used to determine the out-of-plane displacement direction. 2. A kind of out-of-plane displacement measurement method based on neural network according to claim 1, is characterized in that, described single-point out-of-plane displacement measurement system comprises Michelson interferometer, piezoelectric ceramic nano-translation stage, CCD camera and computing equipment; In the single-point off-plane displacement measurement system, the emitted laser is divided into two paths through a beam splitting prism, the optical path at the displacement end returns through the first reflecting mirror, and the optical path at the reference end returns through the second reflecting mirror; Wherein, the first reflecting mirror and the second reflecting mirror are coupled on the piezoelectric ceramic nano-translation stage. 3. a kind of out-of-plane displacement measurement method based on neural network according to claim 1, is characterized in that, described according to described single-point out-of-plane displacement measurement system to carry out reflection surface interference experiment and diffuse reflection surface interference experiment, to Translate the displacement end to obtain fringe images, including: The normalized convolution value of the interferogram sub-area at any time and the interferogram sub-area at the initial moment is used as the reference value of the degree of similarity; The displacement tracking is performed according to the nonlinear mapping relationship between the normalized convolution value and the displacement, and the displacement of the measured object at any time is obtained. 4. A kind of out-of-plane displacement measurement method based on neural network according to claim 3, is characterized in that, described according to described single-point out-of-plane displacement measurement system to carry out reflection surface interference experiment and diffuse reflection surface interference experiment, to The displacement end is translated to obtain the fringe image, which also includes: In one cycle, the fringe image is divided into two states according to the gray matrix of the central sub-area of the fringe image, moving to the left for half a cycle and moving to the right for half a cycle, and transforming the displacement direction judgment problem into an image recognition problem . 5. A kind of out-of-plane displacement measurement method based on neural network according to claim 1, is characterized in that, described according to described fringe image, adopts back-propagation neural network to identify the out-of-plane displacement size in one cycle, including : A five-layer BP neural network is used to approximate the nonlinear mapping relationship between the normalized convolution value and the displacement; Calculate the normalized convolution value of the interferogram at each moment and the interferogram at the initial moment, use the normalized convolution value as the input value of the BP neural network, and use the corresponding known displacement as the output value of the network. Network training; wherein, the learning rate is set to 0.001 in the process of the training; Wherein, the input layer of the five-layer BP neural network contains one neuron, which corresponds to the input normalized convolution value; the number of neurons contained in the three hidden layers of the five-layer BP neural network is 100, respectively. 200, 200; the output layer of the five-layer BP neural network contains a neuron, corresponding to the displacement of the output; any two layers are fully connected, the initial value of the connection parameter weight is a random number of Gaussian distribution, and the initial value of the bias term is a constant. 6. A neural network-based out-of-plane displacement measurement method according to claim 1, wherein the convolutional neural network comprises three convolutional layers, three pooling layers, a fully connected layer and a The softmax function layer, according to the fringe image, uses a convolutional neural network to determine the direction of the off-plane displacement, including: Taking the sub-region of the interference map as the input value of the network, when passing through the first convolution layer, the size of the convolution kernel is (3, 3, 1, 32), and the image becomes a three-dimensional matrix of size 600×600×32; Passing the three-dimensional matrix through a maximum pooling layer, the step size in the length and width directions is 2, and the size of the three-dimensional matrix is changed to 300×300×32; After the second convolution layer, the size of the convolution kernel is (3, 3, 32, 64), and the size of the three-dimensional matrix is changed to 300×300×64; After a maximum pooling layer with a step size of 2 in both the length and width directions, the size of the three-dimensional matrix becomes 150×150×64; After the third convolution layer, the size of the convolution kernel is (3, 3, 64, 64), and the size of the three-dimensional matrix becomes 300×300×64; After the max pooling layer, the 3D matrix size becomes 75×75×64; After a fully connected layer with 1000 neurons and a softmax function layer; After passing through the output layer with the number of neurons being 2, the output is a probability distribution, and the probability distribution is used to represent the probability that the off-plane displacement direction is left or right; The out-of-plane displacement direction is determined according to the direction with a larger probability value in the probability distribution. 7. An out-of-plane displacement measuring device based on a neural network, characterized in that, comprising: The first module is used to obtain the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes; The second module is used to construct a single-point off-plane displacement measurement system according to the characteristic information of the reflection surface interference fringes and the diffuse reflection interference fringes; The third module is used to perform the reflection surface interference experiment and the diffuse reflection surface interference experiment according to the single-point off-plane displacement measurement system, and translate the displacement end to obtain fringe images; The fourth module is used to identify the off-plane displacement size within one cycle by using a back-propagation neural network according to the fringe image; The fifth module is used for determining the out-of-plane displacement direction by using the convolutional neural network pair according to the fringe image. 8. An electronic device, comprising a processor and a memory; the memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 6 . 9 . A computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 6 . 10. A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images