CN116612007A

CN116612007A - Method and device for splicing sub-apertures on surface of optical element

Info

Publication number: CN116612007A
Application number: CN202310651290.5A
Authority: CN
Inventors: 王红军; 王颖茹; 朱学亮; 刘丙才; 岳鑫; 田爱玲; 王大森
Original assignee: Xian Technological University
Current assignee: Xian Technological University
Priority date: 2023-06-03
Filing date: 2023-06-03
Publication date: 2023-08-18
Anticipated expiration: 2043-06-03

Abstract

The invention discloses a method and a device for splicing sub-apertures on the surface of an optical element, and relates to the field of optical measurement. The method can solve the problems of longer detection time and low detection efficiency of the existing sub-aperture splicing method aiming at the detection of the surface defects of the large-caliber precise optical element. The method comprises the following steps: preprocessing the scanned sub-aperture image and extracting an overlapping area data unit, and establishing a sub-aperture data unit corresponding to each preprocessed image according to the overlapping area data unit; according to the position relation between the overlapping region codes and the sub-aperture regions included in the overlapping region data unit, the sequence of the overlapping region codes is adjusted, and overlapping region matching pairs with matching relation and overlapping part transformation matrixes are obtained; and compressing and storing data according to the initial position of the overlapped part transformation matrix and the corresponding sub-aperture area in the global coordinate system and the sparse matrix corresponding to the sub-aperture area to obtain a sub-aperture spliced image.

Description

Method and device for splicing sub-apertures on surface of optical element

Technical Field

The invention relates to the field of optical measurement, in particular to a method and a device for splicing sub-apertures on the surface of an optical element.

Background

Along with the application of the optical element in the fields of large astronomical telescope systems, X-ray laser systems, inertial confinement nuclear fusion systems and the like, the requirements on the surface quality of the optical element are higher and higher for meeting the imaging quality of the system, ensuring the operation safety of the optical system and the use function of the system. The corresponding realization of the rapid and accurate detection of the surface defects of the optical element has very important significance.

In many optical element defect detection methods, microscopic scattering dark field imaging detection methods are based on the scattering effect of optical element surface defects on light beams, and optical element surface defect images of dark background bright defects are obtained by collecting element surface scattered light and utilizing CMOS (Complementary Metal Oxide Semiconductor, complementary metal oxide field effect transistor) imaging. The method takes the defects on the surface of the element as luminous points, the widening effect enables the defects to be more obvious on a dark background image, the detection sensitivity is improved, the defects on the surface of the element with the size smaller than the diffraction limit of an imaging optical system can be detected, super-resolution detection is realized, and meanwhile, online defect detection can be realized by combining an image processing technology.

Because the resolution and the view field of the optical imaging system cannot be obtained, when the defect detection is carried out on the surface of the large-caliber optical element, the large-view field range can be provided while the optical resolution is ensured by a sub-aperture splicing method. For large-caliber precise optical elements, because the surface defects are fewer and distributed discretely, a large number of sub-aperture images acquired through microscopic scattering dark field imaging are not in or have fewer defect images, so that the storage data size is increased, and the splicing errors are accumulated easily in the splicing process, so that the problems of longer detection time and low detection efficiency exist.

Disclosure of Invention

The embodiment of the invention provides a sub-aperture splicing method and device for the surface of an optical element, which can solve the problems of longer detection time and low detection efficiency of the existing sub-aperture splicing method aiming at the detection of the surface defects of a large-caliber precise optical element.

The embodiment of the invention provides a sub-aperture splicing method for the surface of an optical element, which comprises the following steps:

preprocessing the scanned sub-aperture image and extracting an overlapping area data unit, and establishing a sub-aperture data unit corresponding to each preprocessed image according to the overlapping area data unit; each sub-aperture data unit at least comprises sub-aperture area codes, scanning data and overlapping area data units;

dividing the overlapped region codes into left and right region codes and upper and lower region codes according to the position relation between the overlapped region codes and the sub-aperture regions included in the overlapped region data unit;

the left and right region codes determine corresponding rows of each left and right region code according to the number of longitudinal scanning steps in corresponding scanning data, the left and right region codes included in each row are ordered from small to large according to the number of transverse scanning steps, and the sequence of the left and right region codes is adjusted according to a transverse ordering result; and/or the upper and lower region codes determine corresponding columns of each upper and lower region code according to the transverse step number in the corresponding scanning data, the upper and lower region codes included in each column are ordered from small to large according to the longitudinal step number, and the sequence of the upper and lower region codes is adjusted according to the longitudinal ordering result;

Splitting the upper and lower region codes and/or the left and right region codes with the adjustment sequence according to columns to obtain overlapping region matching pairs with matching relations;

obtaining a constant-torque Euclidean distance according to Hu invariant moment of each defect included in the overlapping region matching pair, and determining an optimal defect matching pair set included in the overlapping region matching pair according to the constant-torque Euclidean distance; obtaining an overlapped part transformation matrix according to the mass center coordinates of the defects corresponding to the matched defect pairs in the optimal defect matching pair set;

and according to the initial positions of the overlapped partial transformation matrixes and the corresponding sub-aperture areas in the global coordinate system, obtaining the accurate positions of the sub-aperture areas in the global coordinate system, and according to the accurate positions of the sub-aperture areas in the global coordinate system and the sparse matrixes corresponding to the sub-aperture areas, compressing and storing data, and obtaining the sub-aperture spliced image.

Preferably, the left and right region codes determine corresponding rows of each left and right region code according to the number of longitudinal scanning steps in the corresponding scanning data, the left and right region codes included in each row are ordered from small to large according to the number of transverse scanning steps, the order of the left and right region codes is adjusted according to the transverse ordering result, and the left and right region codes with the adjusted order are split according to columns to obtain overlapping region matching pairs with matching relationship, and the method specifically comprises the steps of:

Dividing the left and right region codes into a left region code and a right region code;

the left region codes determine corresponding rows of each left region code according to the longitudinal scanning step number in the corresponding scanning data, the left region codes included in each row are ordered from small to large according to the transverse scanning step number, and the sequence of the left region codes corresponding to the transverse scanning step number is adjusted according to the transverse ordering result; the right region codes determine corresponding rows of each right region code according to the longitudinal scanning step number in the corresponding scanning data, the right region codes included in each row are ordered from small to large according to the transverse scanning step number, and the sequence of the right region codes corresponding to the transverse scanning step number is adjusted according to the transverse ordering result; sequencing the left region codes with the adjustment sequence and the right region codes with the adjustment sequence in sequence, wherein the left region codes and the right region codes which are positioned in the same column are overlapped region matching pairs with matching relations;

the upper and lower region codes determine corresponding columns of each upper and lower region code according to the transverse step number in the corresponding scanning data, the upper and lower region codes included in each column are ordered from small to large according to the longitudinal step number, the sequence of the upper and lower region codes is adjusted according to the longitudinal ordering result, and the upper and lower region codes with the adjusted sequence are split according to the columns to obtain overlapping region matching pairs with matching relations, and the method specifically comprises the following steps:

Dividing the upper and lower region codes into an upper region code and a lower region code;

the upper region codes determine corresponding columns of each upper region code according to the transverse scanning step number in the corresponding scanning data, the upper region codes included in each column are ordered from small to large according to the longitudinal scanning step number, and the sequence of the upper region codes corresponding to the longitudinal scanning step number is adjusted according to the longitudinal ordering result; the lower region codes determine corresponding columns of each lower region code according to the transverse scanning step number in the corresponding scanning data, the lower region codes included in each column are ordered from small to large according to the longitudinal scanning step number, and the sequence of the lower region codes corresponding to the longitudinal scanning step number is adjusted according to the longitudinal ordering result; and ordering the upper region codes with the adjustment sequence and the lower region codes with the adjustment sequence in sequence, wherein the upper region codes and the lower region codes which are positioned in the same column are matched pairs of overlapping regions with matching relations.

Preferably, after splitting the upper and lower region codes and/or the left and right region codes with the adjustment sequence according to columns to obtain overlapping region matching pairs with matching relations, the method further includes:

according to the overlapping region matching pair, an upper region code and a lower region code are included, a sub-aperture code corresponding to the lower region code is determined to be a reference aperture code, and a sub-aperture code corresponding to the upper region code is determined to be an aperture code to be matched; or alternatively

And determining the sub-aperture code corresponding to the right region code as a reference aperture code and determining the sub-aperture code corresponding to the left region code as an aperture code to be matched according to the left region code and the right region code included in the overlapping region matching pair.

Preferably, after obtaining the overlapping transformation matrix according to the centroid coordinates of the matched defects in the optimal defect matching set, the method further includes:

constructing a sub-aperture spliced data unit by using reference aperture codes, scanning data, sparse matrix compressed storage data corresponding to right area codes and to-be-matched aperture codes, scanning data and sparse matrix compressed storage data corresponding to left area codes included in the overlapping area matching pairs and overlapping part transformation matrixes; or (b)

And constructing the reference aperture code, the scanning data, the sparse matrix compressed storage data corresponding to the lower region code and the aperture code, the scanning data and the sparse matrix compressed storage data to be matched corresponding to the upper region code, which are included in the overlapping region matching pair, and the overlapping part transformation matrix into sub-aperture spliced data units.

Preferably, the Hu invariant moment of each defect included in the matching pair according to the overlapping area obtains the invariant moment euclidean distance, and the invariant moment euclidean distance determination formula is as follows:

Wherein d _ij Indicating the degree of similarity of the corresponding defects,a kth invariant moment value indicative of an ith defect in the reference overlap region R, < ->A kth invariant moment value representing a jth defect in the overlap region S to be matched, i=1, 2,..n; j=1, 2,..m; k=1, 2..7, m represents the number of defects in the overlap region S to be matched, and n represents the number of defects in the reference overlap region R.

Preferably, the preprocessing and overlapping area data unit extraction of the scanned sub-aperture image specifically includes:

carrying out image filtering treatment, image binarization treatment and digital morphology treatment on the scanned sub-aperture image to obtain a preprocessed image; cutting the preprocessed image according to the overlapping area to obtain an overlapping area image, wherein the width of the cut overlapping area is determined according to the nominal moving distance of the translation stage;

performing eight-neighborhood detection and marking on defects included in the preprocessed image and the overlapped area image by a connected area marking method, and determining the number of marks as the number of defects included in the preprocessed image and the overlapped area image;

determining the length-width ratio of the defects included in the overlapped area image based on a minimum circumscribed rectangle algorithm, and determining defect type data included in the overlapped area image according to the length-width ratio of each defect;

The Hu invariant moment corresponding to the overlapped area image is determined through the Hu moment, and the Hu invariant moment is determined to be the shape characteristic of the overlapped area defect;

each sub-aperture data unit further comprises a defect number; the overlapping area data unit comprises a defect type, hu invariant moment and defect centroid coordinates.

The embodiment of the invention provides an optical element surface sub-aperture splicing device, which comprises:

the establishing unit is used for preprocessing the scanned sub-aperture images and extracting overlapping area data units, and establishing sub-aperture data units corresponding to each preprocessed image according to the overlapping area data units; each sub-aperture data unit at least comprises sub-aperture area codes, scanning data and overlapping area data units;

the first obtaining unit is used for dividing the overlapped region codes into left and right region codes and upper and lower region codes according to the position relation between the overlapped region codes and the sub-aperture regions included in the overlapped region data unit; the left and right region codes determine corresponding rows of each left and right region code according to the number of longitudinal scanning steps in corresponding scanning data, the left and right region codes included in each row are ordered from small to large according to the number of transverse scanning steps, and the sequence of the left and right region codes is adjusted according to a transverse ordering result; and/or the upper and lower region codes determine corresponding columns of each upper and lower region code according to the transverse step number in the corresponding scanning data, the upper and lower region codes included in each column are ordered from small to large according to the longitudinal step number, and the sequence of the upper and lower region codes is adjusted according to the longitudinal ordering result;

The second obtaining unit is used for splitting the upper and lower region codes and/or the left and right region codes with the adjustment sequence according to columns to obtain overlapping region matching pairs with matching relations; obtaining a constant-torque Euclidean distance according to Hu invariant moment of each defect included in the overlapping region matching pair, and determining an optimal defect matching pair set included in the overlapping region matching pair according to the constant-torque Euclidean distance; obtaining an overlapped part transformation matrix according to the mass center coordinates of the defects corresponding to the matched defect pairs in the optimal defect matching pair set;

and the third obtaining unit is used for obtaining the accurate position of the sub-aperture area in the global coordinate system according to the initial position of the overlapping part transformation matrix and the sub-aperture area corresponding to the overlapping part transformation matrix in the global coordinate system, compressing and storing data according to the accurate position of the sub-aperture area in the global coordinate system and the sparse matrix corresponding to the sub-aperture area, and obtaining the sub-aperture spliced image.

An embodiment of the present invention provides a computer device, where the computer device includes a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute a sub-aperture splicing method for an optical element surface according to any one of the foregoing embodiments.

An embodiment of the present invention provides a computer readable storage medium storing a computer program, where the computer program when executed by a processor causes the processor to perform a method for stitching sub-apertures on a surface of an optical element according to any one of the above embodiments.

The embodiment of the invention provides a sub-aperture splicing method and device for the surface of an optical element, wherein the method comprises the steps of carrying out image preprocessing and overlapping area data unit extraction on acquired dark field sub-aperture images, establishing sub-aperture data units corresponding to each preprocessed image according to the overlapping area data units, and reducing data storage capacity by taking the sub-aperture data units as the basis of full-caliber splicing processing; further, based on the overlapping region codes included in the overlapping region data unit and sub-aperture region scanning data corresponding to the overlapping region codes, overlapping region matching pairs are obtained; and (3) based on the shape characteristics of the defects of the corresponding overlapping region, obtaining an overlapping part transformation matrix, compressing and storing data according to the coordinate positions of the overlapping part transformation matrix and the sub-aperture region and the sub-aperture sparse matrix, and realizing the rapid and accurate splicing of the sub-apertures on the surface of the optical element. The method reduces splicing errors and improves splicing efficiency.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic flow chart of a method for splicing sub-apertures on a surface of an optical element according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a microscopic dark field imaging detection apparatus according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a sub-aperture scan trajectory according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a sub-aperture data unit structure according to an embodiment of the present invention;

FIG. 5 is a sequence diagram of a 3×3 sub-aperture area according to an embodiment of the present invention

Fig. 6 is a schematic diagram of a sub-aperture spliced data unit according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a sub-aperture splicing device for a surface of an optical element according to an embodiment of the present invention;

wherein, CMOS-1, microscope objective-2, annular light source-3, measured element-4, electric control translation stage-5 and stepping motor controller-6.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Fig. 1 is a schematic flow chart of a method for splicing sub-apertures on a surface of an optical element according to an embodiment of the present invention, and the method for splicing sub-apertures on a surface of an optical element according to an embodiment of the present invention is described in detail below according to fig. 1. Specifically, the method comprises the following steps:

step 101, preprocessing sub-aperture images obtained by scanning and extracting overlapping area data units, and establishing sub-aperture data units corresponding to each preprocessed image according to the overlapping area data units; each sub-aperture data unit at least comprises sub-aperture area codes, scanning data and overlapping area data units;

102, dividing the overlapped region codes into left and right region codes and upper and lower region codes according to the position relation between the overlapped region codes and the sub-aperture regions included in the overlapped region data unit;

Step 103, the left and right region codes determine corresponding rows of each left and right region code according to the number of longitudinal scanning steps in the corresponding scanning data, the left and right region codes included in each row are ordered from small to large according to the number of transverse scanning steps, and the sequence of the left and right region codes is adjusted according to the transverse ordering result; and/or the upper and lower region codes determine corresponding columns of each upper and lower region code according to the transverse step number in the corresponding scanning data, the upper and lower region codes included in each column are ordered from small to large according to the longitudinal step number, and the sequence of the upper and lower region codes is adjusted according to the longitudinal ordering result;

step 104, splitting the upper and lower region codes and/or the left and right region codes with the adjustment sequence according to columns to obtain overlapping region matching pairs with matching relations;

step 105, obtaining a constant torque Euclidean distance according to Hu invariant moment of each defect included in the overlapping region matching pair, and determining an optimal defect matching pair set included in the overlapping region matching pair according to the constant torque Euclidean distance; obtaining an overlapped part transformation matrix according to the mass center coordinates of the defects corresponding to the matched defect pairs in the optimal defect matching pair set;

and 106, obtaining the accurate position of the sub-aperture area in the global coordinate system according to the initial position of the overlapping part transformation matrix and the sub-aperture area corresponding to the overlapping part transformation matrix in the global coordinate system, and compressing and storing data according to the accurate position of the sub-aperture area in the global coordinate system and the sparse matrix corresponding to the sub-aperture area to obtain the sub-aperture spliced image.

Before step 101, a microscopic dark field imaging measurement device is described according to fig. 2, and as shown in fig. 2, the device mainly includes a CMOS1, a microscope objective 2, a measured element 4, an annular light source 3, an electronically controlled translation stage 5, a stepper motor controller 6, and a computer. Specifically, the light source irradiates the tested element, and back scattered light caused by surface defects of the tested element enters the microscopic imaging system through the microscopic objective lens and is converged on the target surface of the CMOS, and reflected light does not enter the microscopic imaging system to form a dark field, so that a bright image under a dark background is formed by a defect image.

Further, the sub-aperture scanning scheme applied simultaneously with the microscopic dark field imaging measuring device is that the annular light source irradiates the surface of the measured element, the translation stage is controlled by the stepping motor controller to start scanning from the upper left corner of the measured element along the X axis and scan right along the scanning track shown in fig. 3, each time a step length is moved, the CMOS acquires a sub-aperture image until the right edge of the measured element is scanned, then the translation stage moves a step length along the Y axis direction, then scans left along the X axis, acquires an image every time a step length is moved, the scanning step number of the translation stage corresponding to the sub-aperture image is stored, and the like until the whole surface of the optical element is acquired.

In practice, the translation stage scan steps may be referred to as scan data, which may be expressed as (I, J) (i=0, 1,2.

When the microscopic dark field imaging measuring device provided by fig. 2 scans the measured element and acquires the sub-aperture image, the computer can synchronously perform preprocessing and overlapping area data unit extraction on the scanned sub-aperture image.

In step 101, pre-processing and overlapping area data unit extraction are performed on the scanned sub-aperture image. The preprocessing of the sub-aperture image mainly comprises the following steps: the method comprises the steps of image filtering processing, image binarization processing, digital morphology processing and overlapping region clipping, wherein the preprocessed image is clipped according to the overlapping region to obtain an overlapping region image, and the clipping overlapping region width is determined according to the nominal moving distance of the translation stage.

The preprocessing of the image is described in detail below:

the image filtering process adopts median filtering to remove noise caused by dust and the like in the dark field image, and the median filtering is a nonlinear smoothing filtering method which can remove the noise and simultaneously retain the characteristic information of the image.

The image binarization is to traverse the sub-aperture image by determining a threshold T as a dividing limit, setting a pixel value greater than T in the sub-aperture image to 1, and setting a pixel value less than or equal to T to 0 as a background point, thereby distinguishing a target from a background in the sub-aperture image, and reserving a region of interest to the maximum extent, specifically, as shown in formula (1):

Where g (x, y) represents an output image and f (x, y) represents an input image.

The digital morphological processing comprises two parts, namely corrosion and expansion, wherein the corrosion is that the target boundary is contracted inwards, the expansion is that the target boundary is expanded outwards, the opening operation is that the image is corroded first and then expanded, the closing operation is that the image is corroded first and then corroded, and the closing operation can connect the disconnected adjacent target and fill the tiny holes in the target area, so that the binarized sub-aperture image is subjected to digital morphological processing by using the closing operation, and the closing operation formula is shown in (2):

wherein F is a binary image, G is a structural element,indicating that the former is inflated by the latter, +.>Indicating that the former is corroded by the latter.

By the three preprocessing methods, the sub-aperture image obtained by scanning is obtained into a preprocessed image, further, the preprocessed image is required to be subjected to overlap region clipping to obtain an overlap region image, in practical application, the clipping overlap region width is determined according to the nominal moving distance of the translation stage, the nominal moving distance of the translation stage is obtained by calculating parameters such as CMOS resolution, field size, scanning system moving step length and the like, and the clipping overlap region width is represented by delta M.

In the embodiment of the invention, the determination of the data unit of the overlapped area mainly comprises the determination of the number of the defects, the data of the types of the defects and the shape characteristics of the defects of the overlapped area, and the following description is given of the determination process of the number of the defects, the data of the types of the defects and the shape characteristics of the defects of the overlapped area:

Number of defects: because the pretreatment image and the overlapping area image are obtained before, eight neighborhood detection and marking can be carried out on the defect targets in the pretreatment image and the overlapping area image by a connected area marking method on the basis of the pretreatment image and the overlapping area image until the detection of the whole image is finished, and the statistic marking quantity is the defect quantity of the area.

Defect type data:

identifying defect targets in the overlapped area images by utilizing a minimum circumscribed rectangle algorithm to obtain eachThe minimum circumscribed rectangle coordinate value of each defect, and calculating the minimum circumscribed rectangle length-width ratio to distinguish the defect types, wherein the defect length l and width w pass through the four coordinate values x of the minimum circumscribed rectangle _max ,x _min ,y _max ,y _min Solving, as shown in formula (3):

the aspect ratio P is determined by equation (4):

further, the type of defect can be judged from the aspect ratio P. Furthermore, when the transverse aspect ratio is less than 4:1, defined by the U.S. Pat. Nos. MIL-PRP-13830 and MIL-O-13830, the pits may be identified, and conversely, the scratches may be identified.

Shape feature data of the defects in the overlapped area:

in practical application, the Hu moment is a common method for describing regional characteristics, is statistics of all relevant pixel points in an image region, has invariance of translation, rotation and scaling, is first proposed by M.K. Hu in 1962, and is widely applied to image recognition after decades of development.

In the embodiment of the invention, the Hu invariant moment corresponding to the defect in the overlapped area image is determined through the Hu moment, and the Hu invariant moment is determined as the shape characteristic data of the defect in the overlapped area.

In the Cartesian coordinate system, let the gray image function be f (x, y), its (p+q) order moment definition is as shown in equation (5):

m _pq ＝∑ _x ∑ _y x ^p y ^q f(x,y) (5)

further, the geometric moment that moves the origin of the coordinate system to the centroid of the image is referred to as the center geometric moment, which is defined as shown in equation (6):

μ _pq ＝∑ _x ∑ _y (x-x ₀ ) ^p (y-y ₀ ) ^q f(x,y) (6)

wherein, (x) ₀ ,y ₀ ) The normalized center-to-center distance for the centroid coordinates, f (x, y), is shown in equation (7):

where r=1+ (p+q)/2, p+q=2, 3.

The geometric moment of the image represents the spatial characteristics of the gray distribution of the image, but the geometric moment of different orders represents different physical meanings, and the meanings of the geometric moment of each order are as follows:

zero order geometrical moment m ₀₀ The sum of the gray values of the image is represented, and when the image is a binary image, the area of the image is represented. First order geometric moment m ₀₁ 、m ₁₀ May be used to represent the centroid of the image, assuming that the image centroid coordinates are (x ₀ ，y ₀ ) Can be represented by formula (8)

In the embodiment of the invention, the mass center coordinates of the defects are also determined as the shape characteristic data of the defects in the overlapped area.

The second order geometric moment can be used to calculate the direction of the shape. The first order second order center moment of the image can be used to determine a uniform ellipse that is equal to both the second order moment of the original image and the gray level sum of the original image. The third order geometric moment represents the degree of distortion of the image projection, which is a measure of the degree of deviation from the mean symmetric distribution. Fourth order geometric moments can be used to calculate kurtosis coefficients. With normalized second and third order central moments, the 7 moment invariant formulas proposed by Hu through nonlinear combination are as follows:

φ ₁ ＝η ₂₀ +η ₀₂ (9-1)

φ ₂ ＝(η ₂₀ -η ₀₂ ) ² +4η ₁ ² ₁ (9-2)

φ ₃ ＝(η ₃₀ -3η ₁₂ ) ² +(3η ₂₁ -η ₀₃ ) ² (9-3)

φ ₄ ＝(η ₃₀ +η ₁₂ ) ² +(η ₂₁ +η ₀₃ ) ² (9-4)

φ ₅ ＝(η ₃₀ -3η ₁₂ )(η ₃₀ +η ₁₂ )[(η ₃₀ +η ₁₂ ) ² -3(η ₁₂ +η ₀₃ ) ² ]

+(3η ₂₁ -η ₀₃ )(η ₂₁ +η ₀₃ )[3(η ₃₀ +η ₁₂ ) ² -(η ₂₁ +η ₃₀ ) ² ](9-5)

φ ₆ ＝(η ₂₀ -η ₀₂ )[(η ₃₀ +η ₁₂ ) ² -(η ₂₁ +η ₀₃ ) ² ]+

4η ₁₁ (η ₃₀ +η ₁₂ )(η ₂₁ +η ₀₃ )(9-6)

φ ₇ ＝(3η ₂₁ -η ₀₃ )(η ₃₀ +η ₂₁ )[(η ₃₀ +η ₁₂ ) ² -3(η ₂₁ +η ₀₃ ) ² ]

+(3η ₁₂ -η ₃₀ )(η ₂₁ +η ₀₃ )[3(η ₃₀ +η ₁₂ ) ² -(η ₂₁ +η ₀₃ ) ² ](9-7)

In the embodiment of the invention, phi is utilized ₁ ～φ ₇ The 7 Hu invariant moment calculation formulas calculate invariant moment of each defect area in the overlapped area image as shape characteristic data of the corresponding defect, namely shape characteristic data of the defects of the overlapped area.

In the embodiment of the invention, after the scanned sub-aperture image is subjected to image binarization and morphological processing, a small number of pixel values of a defect area are 1, and a large number of pixel values of a background area are 0, and non-zero element values and positions in the image are saved by using a sparse matrix compression storage mode so as to reduce the storage data volume. In particularThe compression storage adopts a triple node method, and the triple node is defined as [ i, j, a ] _ij ]Wherein i represents a row number, j represents a column number, a _ij And representing element values, wherein each non-zero element in the two-dimensional array is uniquely determined by a triplet node, so that compressed storage of a sparse matrix is realized, and compressed storage data of the sparse matrix is obtained.

In the embodiment of the invention, when the scanned sub-aperture image is preprocessed to obtain a preprocessed image, the preprocessed image is cut according to the overlapping area to obtain an overlapping area image; then, after the number of defects, the types of defects and the shape characteristics of the defects in the overlapped area are sequentially acquired, a sub-aperture data unit and a sub-aperture data set are required to be constructed.

Fig. 4 is a schematic structural diagram of a sub-aperture data unit provided in an embodiment of the present invention, where, as shown in fig. 4, each sub-aperture data unit includes a sub-aperture region code, scan data, a defect number, an overlap region data unit, and sparse matrix compressed storage data, where the overlap region defect shape feature includes a Hu invariant moment and a centroid coordinate, and the overlap region data unit includes an overlap region code, a defect number, a defect type, a Hu invariant moment, and a centroid coordinate.

In practical application, as only a small amount of defects exist on the surface of the tested element, it can be determined that most of scanned sub-aperture images do not contain defects after pretreatment, so that for sub-aperture images which do not contain defects, sub-aperture data units corresponding to sub-aperture data sets only contain two information of sub-aperture area codes and scanned data, and other information units related to defects are attached with blank values.

In step 102, the overlapping region codes are extracted from the sub-aperture dataset constructed in step 101, and the overlapping region codes can be divided into upper and lower region codes and left and right region codes according to the positional relationship between the overlapping region codes and the sub-aperture region.

In the embodiment of the present invention, the sub-aperture data unit includes sub-aperture area codes, scan data, defect number, overlapping area data unit, and sparse matrix compressed storage data, and the sub-aperture area codes corresponding to the overlapping area codes may be determined by the sub-aperture data unit according to the overlapping area codes. In the embodiment of the invention, a partial overlapping area is reserved between two adjacent sub-aperture images when the sub-aperture images are scanned, and further, when the sub-aperture images are preprocessed and cut, the width of the cut overlapping area is determined according to the nominal moving distance of the translation stage.

The sub-aperture image scan acquisition path as shown in FIG. 3, wherein sub-aperture area A ₀₀ Comprises two overlapped area images respectively positioned at A ₀₀ Right and lower sides of (a); sub-aperture area A ₁₀ Comprises three overlapped area images respectively positioned at A ₁₀ Left, right and lower sides of (a); sub-aperture area A ₀₁ It includes three overlapping area images, respectively positioned at A ₀₁ Upper, lower and right sides of (a); sub-aperture area A ₁₁ Comprises four overlapped area images respectively positioned at A ₁₁ Upper side, lower side, left side and right side of (c).

In the embodiment of the invention, the overlapping area corresponding to each sub-aperture area is encoded, namely, the overlapping area is encoded. The determination of the overlapping region codes is performed by cutting out the overlapping region, and therefore, according to each overlapping region code, the position of the overlapping region code in the corresponding sub-aperture region can be determined.

For example, as shown in the 3×3 sub-aperture area sequence diagram of fig. 5, the overlapping area codes with defects in fig. 5 are respectively: 1-y, 2-z, 4-z, 5-y, 6-y, 8-y, 9-z; from the above-described overlapping region codes and the respective sub-aperture regions, the positions thereof in the sub-aperture regions, respectively, can be determined. 1-y is located to the right of sub-aperture region 1 (0, 0) and 2-z is located to the left of sub-aperture region 2 (1, 0).

In step 103, after the overlapped region codes are divided into the upper and lower region codes and the left and right region codes, the order of the upper and lower region codes and the left and right region codes may be adjusted according to the scan data corresponding to the upper and lower region codes and the left and right region codes, respectively.

Specifically, taking the order adjustment of the left and right region coding as an example, the detailed procedure is described:

since the left and right region codes are located at the left side or the right side of the sub-aperture region, the left and right region codes need to be divided into left region codes and right region codes when the order of the left and right region codes is adjusted; and then respectively adjusting the sequence of the left region coding and the right region coding.

Under the condition, the left region codes determine the corresponding row of each left region code according to the longitudinal scanning step number in the corresponding scanning data, determine the left region code included in each row, then sort the transverse scanning step number from small to large, and adjust the sequence of the left region codes corresponding to the transverse scanning step number according to the transverse sorting result.

Illustratively, as shown in fig. 5, the left region codes in the figure are respectively: 2-z, 4-z, 5-z, 9-z, wherein the correspondence between left region codes and scan data is respectively: [ 2-z, (1, 0) ], [ 4-z, (2, 1) ], [ 5-z, (1, 1) ], [ 9-z, (2, 2) ]; dividing four left area codes into three rows which are respectively the first row 2-z according to the number of longitudinal scanning steps; second rows 4-z, 5-z; third row 9-z; when a line includes only one left region code, there is no need to adjust the sequence of the left region code included in the line, in the embodiment of the present invention, when a line includes at least two left region codes, there is a need to adjust the sequence of the left region code located in the line according to the scan data corresponding to the left region code, specifically, because the scan data corresponding to the two left region codes included in the second line are (2, 1) and (1, 1) respectively, when the two scan data included in the second line are ordered from small to large according to the number of horizontal scan steps, the positional relationship of the two scan data included in the second line becomes (1, 1) and (2, 1); further, the order of the left region codes corresponding to the number of the lateral scanning steps needs to be adjusted according to the lateral ordering result, and the order of the left region codes originally included in the second line is adjusted from 4-z to 5-z to 4-z. Further, the sequence of the left region codes is adjusted from original 2-z, 4-z, 5-z and 9-z to 2-z, 5-z, 4-z and 9-z.

Under the condition, the right region codes determine the corresponding row of each right region code according to the longitudinal scanning step number in the corresponding scanning data, determine the right region code included in each row, then sort the transverse scanning step number from small to large, and adjust the sequence of the right region codes corresponding to the transverse scanning step number according to the transverse sorting result.

Illustratively, as shown in fig. 5, the right region codes in the figure are respectively: 1-y, 5-y, 6-y and 8-y, wherein the corresponding relation between the right region codes and the scanning data is as follows: [ 1-y, (0, 0) ], [ 5-y, (1, 1) ], [ 6-y, (0, 1) ], [ 8-y, (1, 2) ]; dividing four right region codes into three rows, namely a first row 1-y respectively, according to the number of longitudinal scanning steps; second rows 5-y, 6-y; third row 8-y; because the first row and the third row only comprise one right region code, the right region codes of the two rows do not need to be sequentially adjusted, the second row comprises two right region codes, the scanning data corresponding to the two right region codes are (1, 1) and (0, 1) respectively, and when the scanning data are ordered from small to large according to the transverse scanning step number, the position relationship of the two scanning data of the second row is changed into (0, 1) and (1, 1); further, the right region coding sequence corresponding to the number of the transverse scanning steps is required to be adjusted according to the transverse sorting result, and the left region coding sequence originally included in the second row is adjusted from 5-y and 6-y to 6-y and 5-y. Further, the coding sequence of all right regions is adjusted from original 1-y, 5-y, 6-y and 8-y to 1-y, 6-y, 5-y and 8-y.

In step 104, after the order adjustment of the left region code and the right region code is completed, the left region code with the adjusted order and the right region code with the adjusted order are ordered in order, and the left region code and the right region code located in the same column are matched pairs of overlapping regions with matching relationship.

As shown in fig. 5, the left region encoding of the adjustment order is sequentially: 2-z, 5-z, 4-z, 9-z; the right region coding of the adjustment sequence is as follows: 1-y, 6-y, 5-y, 8-y; wherein 2-z and 1-y are in the same column, 5-z and 6-y are in the same column, 4-z and 5-y are in the same column, and 9-z and 8-y are in the same column; therefore, it is possible to determine that [ 2-z, 1-y ] located in the same column are overlapping region matching pairs having a matching relationship, that [ 5-z, 6-y ] are overlapping region matching pairs having a matching relationship, that [ 4-z, 5-y ] are overlapping region matching pairs having a matching relationship, and that [ 9-z, 8-y ] are overlapping region matching pairs having a matching relationship.

Further, after determining the overlapping region matching pair, if the overlapping region matching pair includes an upper region code and a lower region code, a sub-aperture code corresponding to the upper region code may be determined as the aperture code to be matched, and a sub-aperture code corresponding to the lower region code may be determined as the reference aperture code.

After the overlapping region matching pair is determined, if the overlapping region matching pair includes a left region code and a right region code, a sub-aperture code corresponding to the left region code may be determined as the aperture code to be matched, and a sub-aperture code corresponding to the right region code may be determined as the reference aperture code.

In step 105, it can be seen from the above steps that there is a matching pair of overlapping regions in a matching relationship, which includes two overlapping region codes, i.e. two groups of sub-aperture data units respectively corresponding to each other.

In the embodiment of the invention, in order to distinguish the overlapping region matching pair with the matching relationship respectively corresponds to a group of parameters with consistent names, two overlapping regions included in the overlapping region pair with the matching relationship are respectively named as a reference sub-aperture overlapping region and a sub-aperture overlapping region to be matched.

Further, according to the overlapping region matching pair with the matching relationship, the Hu invariant moment of each defect corresponding to the overlapping region of the reference sub-aperture and the Hu invariant moment of each defect corresponding to the overlapping region of the sub-aperture to be matched are respectively extracted.

Wherein n defects in the reference sub-aperture overlap region R are expressed asThe m defects in the overlapping area S of the sub-apertures to be matched are expressed as +. >Then->And->The invariant euclidean distance of (c) is determined by the following equation (10):

It should be noted that formula d _ij Representing the degree of similarity of the corresponding defects, a smaller value representing the similarity of the two defects, by setting a threshold d _T Screening out d _ij ≤d _T Form an optimal set of pairs of defect matches.

In the embodiment of the invention, the Euclidean distance of invariant moment is obtained according to the Hu invariant moment of each defect included in the overlapping region matching pair, and the optimal defect matching pair set included in the overlapping region matching pair is obtained according to the Euclidean distance of invariant moment.

Further, according to the mass center coordinates of the defects corresponding to the matched defect pairs in the optimal defect matching pair set, an overlapped part transformation matrix can be obtained, and the overlapped part transformation matrix is specifically shown as a formula (11);

in the formula, Δx and Δy represent the translation amounts in the horizontal direction and the vertical direction, respectively, and Δx=

Further, the method comprises the steps of, ⁱ x ₁ represents the abscissa of the centroid point of the matched defect in the optimal defect matched pair set in the overlapping area of the reference sub-aperture, ⁱ x ₂ representing the abscissa of the centroid point of the matched defects in the optimal defect matching pair set in the overlapping area of the sub-apertures to be matched, ⁱ y ₁ representing the ordinate of the centroid point of the matched defect in the optimal defect matched pair set in the overlapping area of the reference sub-aperture, ⁱ y ₂ and the ordinate of the centroid point of the matched defect in the optimal defect matching pair set in the overlapping area of the sub-apertures to be matched is represented, and N represents the number of the matched defect pairs in the optimal matching defect pair set in the overlapping area.

In the embodiment of the invention, after the overlapping region matching pair is determined, an overlapping part transformation matrix is obtained according to the overlapping region matching pair, and further, a sub-aperture spliced data unit can be constructed.

Specifically, since each overlapping region matching pair may be combined by left region encoding and right region encoding, it may also be combined by upper region encoding and lower region encoding. Further, the sub-aperture region codes corresponding to the right (left) region codes in the overlapping region matching pair determined by the left and right regions determine a reference (to-be-matched) aperture code; the sub-aperture region codes corresponding to the lower (upper) region codes in the overlapping region matching pair determined by the upper and lower regions determine the reference (to be matched) aperture codes. Therefore, the sub-aperture spliced data unit constructed in the embodiment of the invention at least comprises the following cases:

And constructing the reference aperture code, the scanning data, the sparse matrix compressed storage data corresponding to the right region code and the aperture code, the scanning data and the sparse matrix compressed storage data to be matched corresponding to the left region code included in the overlapping region matching pair, and the overlapping part transformation matrix into sub-aperture spliced data units.

When the sub-aperture spliced data unit is constructed, when the right region code corresponds to the reference aperture code, the scan data and the sparse matrix compressed storage data may also be referred to as reference aperture scan data and reference aperture sparse matrix compressed storage data; correspondingly, the left area code corresponds to the aperture code to be matched, the aperture scanning data to be matched and the aperture sparse matrix compressed storage data to be matched. The names corresponding to the lower region code and the upper region code are consistent with the right region code and the left region code, and will not be described again here.

As shown in fig. 6, the sub-aperture spliced data unit provided by the embodiment of the invention mainly includes a pair of reference aperture codes and aperture codes to be matched, a pair of reference aperture scan data and aperture scan data to be matched, a pair of reference aperture sparse matrix compressed storage data and aperture sparse matrix compressed storage data to be matched, and an overlapping partial transformation matrix.

The sub-aperture data set formed by the sub-aperture data units and the sub-aperture spliced data set formed by the sub-aperture spliced data units are stored in two data sets, and when the sub-aperture data set is applied, searching and matching are performed through common information between sub-aperture numbers and reference aperture numbers (aperture codes to be matched).

In step 106, the initial position of each sub-aperture area corresponding to the overlap transformation matrix in the global coordinate system is determined according to the overlap transformation matrix, and in practical application, the number of scanning steps of the translation stage moving along the X, Y direction during acquisition of each sub-aperture image corresponding to each sub-aperture area is represented as (I, J) (i=0, 1,2.., j=0, 1, 2.), where I is the number of scan steps moving in the X-axis direction, J is the number of scan steps moving in the Y-axis direction, and with the upper left corner of the sub-aperture area as the origin of coordinates, the initial position of each sub-aperture area under the global coordinate system can be expressed as (i× (L- Δm), j× (W- Δm)), where L, W is the length and width values of each sub-aperture area, respectively, and Δm is the clipping overlap area width;

And then, the initial positions of the sub-aperture areas are readjusted according to the overlapping part transformation matrix determined in the steps, and the accurate positions of all the sub-aperture areas under the global coordinate system are obtained.

The initial position of the sub-aperture area is adjusted according to the overlap transformation matrix, that is, by adding Δx and Δy in the overlap transformation matrix to the initial position of the sub-aperture area, the accurate position of the sub-aperture area in the global coordinate system can be obtained, and the accurate position of the sub-aperture of the overlap area without defects is given according to the initial position.

Further, after determining the accurate position of the sub-aperture area under the global coordinate system, coordinate conversion needs to be performed on the defect data included in the sub-aperture area, and the defect data correspond to the corresponding position in the global coordinate system.

When sub-aperture splicing is carried out on the defect information in each sub-aperture area, firstly, carrying out coordinate conversion on row and column values of a defect triplet node array in each sub-aperture according to the position information of each sub-aperture under a global coordinate system; specifically, the defect position before coordinate conversion is [ x, y ]Wherein x is triplet node data [ i, j, a ] _ij ]In (a) is the value of i, y is the data of the triple node [ i, j, a ] _ij ]J value in (a), the position of the defect [ x ', y ' ] after coordinate conversion ']The calculation formula is shown as (12):

secondly, integrating the three-dimensional group data of the defects in each sub-aperture after coordinate conversion into an array, and based on [ i ', j', a ] in the array _ij ]Creating a sparse matrix A by three vectors, wherein the size of the sparse matrix A is calculated by the size data of all sub-apertures and overlapping parts thereof;

wherein [ i ', j', a ] _ij ]The i 'value in (a) corresponds to the x' value after coordinate conversion, and the j 'value corresponds to the y' value after coordinate conversion. And finally, converting the sparse matrix A into a full matrix storage form and displaying an image, thereby completing sub-aperture splicing based on the characteristic data set.

Based on the same inventive concept, the embodiment of the invention provides an optical element surface sub-aperture splicing device, and because the principle of the device for solving the technical problem is similar to that of an optical element surface sub-aperture splicing method, the implementation of the device can be referred to the implementation of the method, and the repetition is omitted.

As shown in fig. 7, the apparatus includes: a unit 701, a first obtaining unit 702, a second obtaining unit 703 and a third obtaining unit 704 are established.

The establishing unit 701 is configured to perform preprocessing and overlapping area data unit extraction on the scanned sub-aperture image, and establish a sub-aperture data unit corresponding to each preprocessed image according to the overlapping area data unit; each sub-aperture data unit at least comprises sub-aperture area codes, scanning data and overlapping area data units;

a first obtaining unit 702, configured to divide the overlapping region code into a left-right region code and an upper-lower region code according to a positional relationship between the overlapping region code and the sub-aperture region included in the overlapping region data unit; the left and right region codes determine corresponding rows of each left and right region code according to the number of longitudinal scanning steps in corresponding scanning data, the left and right region codes included in each row are ordered from small to large according to the number of transverse scanning steps, and the sequence of the left and right region codes is adjusted according to a transverse ordering result; and/or the upper and lower region codes determine corresponding columns of each upper and lower region code according to the transverse step number in the corresponding scanning data, the upper and lower region codes included in each column are ordered from small to large according to the longitudinal step number, and the sequence of the upper and lower region codes is adjusted according to the longitudinal ordering result;

A second obtaining unit 703, configured to split the upper and lower region codes and/or the left and right region codes in the adjustment order according to columns to obtain overlapping region matching pairs with matching relationships; obtaining a constant-torque Euclidean distance according to Hu invariant moment of each defect included in the overlapping region matching pair, and determining an optimal defect matching pair set included in the overlapping region matching pair according to the constant-torque Euclidean distance; obtaining an overlapped part transformation matrix according to the mass center coordinates of the defects corresponding to the matched defect pairs in the optimal defect matching pair set;

and a third obtaining unit 704, configured to obtain an accurate position of the sub-aperture area in the global coordinate system according to the initial position of the overlapping partial transformation matrix and the sub-aperture area corresponding to the overlapping partial transformation matrix in the global coordinate system, and compress and store data according to the accurate position of the sub-aperture area in the global coordinate system and the sparse matrix corresponding to the sub-aperture area, so as to obtain a sub-aperture spliced image.

Preferably, the first obtaining unit 702 is specifically configured to:

Preferably, the first obtaining unit 702 is further configured to:

Preferably, the second obtaining unit 703 is further configured to:

Preferably, the second obtaining unit 703 is further configured to: the Hu invariant moment of each defect included according to the overlapping area matching pair obtains an invariant moment Euclidean distance, and the invariant moment Euclidean distance determining formula is as follows:

Preferably, the establishing unit 701 is specifically configured to:

It should be understood that the above optical element surface sub-aperture splicing device includes units that are only logically divided according to functions implemented by the device, and in practical applications, the above units may be stacked or split. The function achieved by the optical element surface sub-aperture splicing device provided in this embodiment corresponds to one-to-one with the optical element surface sub-aperture splicing method provided in the above embodiment, and the more detailed process flow achieved by the device is described in detail in the above method embodiment one, which is not described in detail here.

Another embodiment of the present invention also provides a computer apparatus, including: a processor and a memory; the memory is used for storing computer program codes, and the computer program codes comprise computer instructions; when the processor executes the computer instructions, the electronic device executes each step of an optical element surface sub-aperture stitching method in the method flow shown in the method embodiment.

In another embodiment, the present invention further provides a computer readable storage medium, where computer instructions are stored, where the computer instructions, when executed on a computer device, cause the computer device to perform the steps of a sub-aperture splicing method for a surface of an optical element in a method flow shown in the foregoing method embodiment.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. A method for sub-aperture splicing of a surface of an optical element, comprising:

2. The method of claim 1, wherein the left and right region codes determine corresponding rows of each left and right region code according to a number of longitudinal scan steps in corresponding scan data, the left and right region codes included in each row are ordered from small to large according to a number of transverse scan steps, an order of the left and right region codes is adjusted according to a transverse ordering result, and the left and right region codes with the adjusted order are split according to columns to obtain overlapping region matching pairs with matching relation, the method specifically comprising:

3. The method according to claim 1, wherein after splitting the upper and lower region codes and/or the left and right region codes in the adjustment order into overlapping region matching pairs having a matching relationship by columns, the method further comprises:

4. A method as in claim 1, wherein said obtaining an overlap transformation matrix from the coordinates of the centroid of the matched defect pair in the set of optimal matched defect pairs further comprises:

5. A method as claimed in claim 1, wherein said Hu invariant moment for each defect included in said matched pair of overlapping regions yields a invariant moment euclidean distance, said invariant moment euclidean distance being defined as follows:

wherein d _ij Indicating the degree of similarity of the corresponding defects,a kth invariant moment value indicative of an ith defect in the reference overlap region R, < ->A kth invariant moment value representing a jth defect in the overlap region S to be matched, i=1, 2,..n; j=1, 2,..m; k=1, 2..7, n represents the number of defects in the overlap region S to be matched and n represents the number of defects in the reference overlap region R.

6. The method of claim 1, wherein preprocessing and overlapping area data unit extraction are performed on the scanned sub-aperture image, specifically including:

7. An optical element surface sub-aperture splicing apparatus, comprising:

8. A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform a method of optical element surface sub-aperture stitching according to any of claims 1-6.

9. A computer-readable storage medium, characterized in that a computer program is stored, which, when being executed by a processor, causes the processor to perform a method of optical element surface sub-aperture stitching according to any of claims 1-6.