WO2024000268A1 - Image processing method and apparatus, and device and medium - Google Patents

Abstract

The present application relates to an image processing method and apparatus, and a device and a medium. The method comprises: acquiring a first image and a second image, wherein the first image and the second image comprise a plurality of trajectory lines and a plurality of trajectory points; next, selecting a first trajectory line in the first image, and rotating the first image on the basis of an included angle between the first trajectory line and a horizontal direction, i.e., rotating the first image to the horizontal direction, wherein the first image and the second image, which are initially obtained, may differ in terms of zoom ratio, and therefore the first image needs to be zoomed in or zoomed out to obtain a third image, such that the third image has the same ratio as the second image; and then, determining a target offset on the basis of the second image and the third image, and moving the third image on the basis of the target offset. By means of the image processing method provided in the embodiments of the present application, a first image and a second image can be registered at a pixel-level precision, thereby improving the accuracy of image processing.

Description

An image processing method, device, equipment and medium Technical field

The present application relates to the field of image processing technology, and in particular, to an image processing method, device, equipment and medium.

Background technique

Gene expression refers to the process of synthesizing genetic information from genes into functional gene products. The products of gene expression are usually proteins. All known life forms use gene expression to synthesize macromolecules of life. In spatial dimension gene expression analysis, in order to correspond the gene expression image to the actual spatial position of the biological sample, an image registration step is required to obtain accurate gene expression information.

Existing registration technology can perform rough position correspondence between microscope images and gene expression visualization images, where the gene expression visualization image is generated using a gene expression matrix containing spatial position information. Based on the characteristics of the image itself and supplemented by artificial visual adjustment, the spatial position of the image captured by the microscope and the gene expression visualization image can be matched. However, existing registration technology can only achieve position matching in the relatively macroscopic dimension of biological samples, resulting in low registration accuracy and inability to support more refined gene expression analysis.

Contents of the invention

In order to solve the above technical problems or at least partially solve the above technical problems, this application provides an image processing method, device, equipment and medium to improve the accuracy of image processing.

In a first aspect, embodiments of the present application provide an image processing method, which method includes:

Obtaining a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and/or a plurality of trajectory points, the trajectory points are formed by the intersection of the two trajectory lines , each of the plurality of trajectory lines has a corresponding index number;

Obtain the first trajectory line in the first image, and rotate the first image based on the angle between the first trajectory line and the horizontal direction;

Enlarging or reducing the first image to obtain a third image, the third image having the same proportion as the second image;

determining a target offset based on the second image and the third image;

The third image is moved based on the target offset.

In a possible implementation, enlarging or reducing the first image to obtain a third image includes:

Obtain a second trajectory line in the first image, and determine a first distance based on the first trajectory line and the second trajectory line;

Obtain the third trajectory line and the fourth trajectory line in the second image, and determine the second distance based on the third trajectory line and the fourth trajectory line, wherein the index number of the third trajectory line is the same as the index number of the third trajectory line. The index number of the first trajectory line is the same, the index number of the fourth trajectory line is the same as the index number of the second trajectory line;

A ratio value is determined based on the first distance and the second distance, and the first image is enlarged or reduced according to the ratio value to obtain a third image.

In a possible implementation, enlarging or reducing the first image to obtain a third image includes:

Enlarging or reducing the first image to obtain a fourth image, the fourth image having the same proportion as the second image;

The fourth image is moved according to a preset period and a preset rotation angle to obtain the third image.

In a possible implementation, enlarging or reducing the first image to obtain a fourth image includes:

Enlarging or reducing the first image to obtain a fifth image, where the fifth image has the same proportion as the second image;

Calculate the first barycenter coordinates of the fifth image and the second barycenter coordinates of the second image respectively;

Determine a center of gravity offset based on the first center of gravity coordinates and the second center of gravity coordinates;

The fifth image is moved based on the gravity center offset to obtain the fourth image.

In a possible implementation, the moving the fourth image according to a preset period and a preset rotation angle to obtain the third image includes:

Obtain the preset period and multiple preset rotation angles;

For any preset rotation angle, determine a first offset based on the fourth image and the second image;

Determine a plurality of second offsets corresponding to the first offset within the preset period, and each second offset in the plurality of second offsets is consistent with the preset period. Each cycle within has a one-to-one correspondence;

For any second offset, move the fourth image based on the second offset to obtain a sixth image;

determining a first degree of similarity between the sixth image and the second image;

Determine a first maximum similarity based on the plurality of preset rotation angles and the first similarity corresponding to each cycle;

The sixth image corresponding to the first maximum similarity is determined to be the third image.

In a possible implementation, determining the target offset based on the second image and the third image includes:

Determine the second offset corresponding to the third image as the target offset;

The moving the third image based on the target offset includes:

The third image is moved based on the preset rotation angle corresponding to the third image and the target offset.

In a possible implementation, the moving the fourth image according to a preset period and a preset rotation angle to obtain the third image includes:

Obtain the preset period and the plurality of preset rotation angles;

For any preset rotation angle, the fourth image is moved according to each period within the preset period to obtain a plurality of seventh images, and each seventh image in the plurality of seventh images is related to Each period within the preset period has a one-to-one correspondence;

For any seventh image, determine a second degree of similarity between the seventh image and the second image;

Determine a second maximum similarity based on the plurality of preset rotation angles and the second similarity corresponding to each cycle;

The seventh image corresponding to the second maximum similarity is determined to be the third image.

In a possible implementation, determining the target offset based on the second image and the third image includes:

determining a plurality of third offsets based on the second image and the third image;

determining the target offset based on the plurality of third offsets;

The moving the third image based on the target offset includes:

The third image is moved based on the preset rotation angle corresponding to the third image and the target offset.

In a possible implementation, determining the first offset based on the fourth image and the second image includes:

Acquire multiple sets of trajectory point sets. Each set of trajectory points in the multiple sets of trajectory point sets includes a first trajectory point and a second trajectory point. The first trajectory point is located in the fourth image, and the second trajectory point is located in the fourth image. The point is located in the second image, and the first trajectory point and the second trajectory point have the same index identifier;

For any set of trajectory points, determine a fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point under the set of trajectory points;

The first offset is determined based on the fourth offset corresponding to the plurality of sets of trajectory points.

In a possible implementation, when the second image has multiple sub-images, the trajectory line of each sub-image has the same index number, and the trajectory line based on the first trajectory point under the trajectory point set is The coordinates and the coordinates of the second trajectory point determine a fourth offset, including:

Determine the second trajectory point in each sub-image based on the index identification of the first trajectory point, where the first trajectory point and the second trajectory point have the same index identification;

Determine the Euclidean distance between the first trajectory point and any second trajectory point among the plurality of second trajectory points;

Determine the second trajectory point corresponding to the minimum Euclidean distance among the plurality of Euclidean distances;

The fourth offset is determined based on the coordinates of the first trajectory point and the coordinates of the second trajectory point corresponding to the minimum Euclidean distance.

In a possible implementation, determining the first similarity between the sixth image and the second image includes:

Perform frequency domain transformation on the sixth image to obtain a first frequency domain image;

Perform frequency domain transformation on the second image to obtain a second frequency domain image;

Calculate the Hamming distance between the first frequency domain image and the second frequency domain image as the first similarity;

Determining the first maximum similarity based on the plurality of preset rotation angles and the first similarity corresponding to each cycle includes:

Based on the plurality of preset rotation angles and the Hamming distance corresponding to each period, the minimum Hamming distance is determined to be the first maximum similarity.

In a possible implementation, the step of moving the fourth image according to each period within the preset period to obtain a seventh image includes:

Calculate the coordinates of the third center of gravity of the fourth image;

Taking the period where the third barycenter coordinate is located as the center, the fourth image is moved according to each period to obtain the seventh image.

In a possible implementation, determining a plurality of third offsets based on the second image and the third image includes:

Acquire multiple sets of trajectory point sets, each of the multiple sets of trajectory point sets includes a third trajectory point and a fourth trajectory point, the third trajectory point is located in the third image, and the fourth trajectory point The point is located in the second image, and the third trajectory point and the fourth trajectory point have the same index identifier;

For any set of trajectory points, a third offset is determined based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point under the set of trajectory points.

In a possible implementation, after moving the third image based on the target offset, the method further includes:

Based on the moved third image and the second image, multiple sets of trajectory point sets are obtained, each of the multiple sets of trajectory point sets includes a fifth trajectory point and a sixth trajectory point, The fifth trajectory point is located in the third image after the movement, the sixth trajectory point is located in the second image, and the fifth trajectory point and the six trajectory points have the same index identifier;

For any group of trajectory point sets, determine a fifth offset based on the coordinates of the fifth trajectory point and the coordinates of the sixth trajectory point under the trajectory point set;

Determine the final offset based on the fifth offset corresponding to the multiple sets of trajectory points;

The moved third image is moved based on the final offset.

In a possible implementation, obtaining the second image includes:

An eighth image is acquired, and convolution enhancement processing is performed on the eighth image to obtain the second image.

In a second aspect, embodiments of the present application provide an image processing device, which includes:

A first acquisition unit, configured to acquire a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and/or a plurality of trajectory points, and the trajectory points are composed of two The trajectory lines are formed by intersecting, and each trajectory line in the plurality of trajectory lines has a corresponding index number;

a second acquisition unit, configured to acquire the first trajectory line in the first image, and rotate the first image based on the angle between the first trajectory line and the horizontal direction;

A first processing unit configured to enlarge or reduce the first image to obtain a third image, where the third image has the same proportion as the second image;

a second processing unit configured to determine a target offset based on the second image and the third image;

A third processing unit configured to move the third image based on the target offset.

In a third aspect, embodiments of the present application provide an electronic device, where the electronic device includes: a memory and a processor;

The memory is used to store instructions or computer programs;

The processor is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the image processing method described in any implementation of the first aspect.

In the fourth aspect, embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium is used to store a computer program. The computer program is used to execute any of the implementation methods described in the first aspect. Image processing methods.

In a fifth aspect, embodiments of the present application provide a computer program product. The computer program product includes a program. When the program is run on a processor, it causes the computer or network device to execute any one of the implementation methods of the first aspect. The image processing method described.

Compared with the existing technology, the technical solution provided by the embodiment of the present application has the following advantages:

When processing images in this embodiment, the first image and the second image are first acquired. The first image and the second image include multiple trajectory lines and multiple trajectory points. The trajectory points are formed by the intersection of the two trajectory lines. , each trajectory line has a corresponding index number. Then select the first trajectory line in the first image, and rotate the first image based on the angle between the first trajectory line and the horizontal direction, that is, rotate the first image to the horizontal direction. The initially obtained first image and the second image may have different scaling ratios, so the first image needs to be enlarged or reduced to obtain a third image, so that the third image has the same ratio as the second image. The target offset is then determined based on the second image and the third image, and the third image is moved based on the target offset. Through the image processing method provided by the embodiment of the present application, the first image and the second image can be registered with pixel-level precision, thereby improving the accuracy of image processing.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the embodiments provided in the present application. , for those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

Figure 1 is a schematic flowchart of an image processing method provided by an embodiment of the present application;

Figure 2a is a schematic diagram of an image provided by an embodiment of the present application;

Figure 2b is another schematic diagram of an image provided by an embodiment of the present application;

Figure 3 is a schematic diagram of a first image provided by an embodiment of the present application;

Figure 4 is a schematic diagram of image rotation provided by an embodiment of the present application;

Figure 5a is a schematic diagram of a third image with multiple periods provided by an embodiment of the present application;

Figure 5b is a schematic diagram of a second image with multiple periods provided by an embodiment of the present application;

Figure 6 is a schematic diagram of a preset period provided by an embodiment of the present application;

Figure 7 is a schematic flow chart of another image processing method provided by an embodiment of the present application;

Figure 8 is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

Figure 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. The described embodiments are only exemplary implementations of the present application and are not all implementations. Those skilled in the art can combine the embodiments of the present application to obtain other embodiments without performing creative work, and these embodiments are also within the protection scope of the present application.

In order to facilitate understanding of the technical solutions provided by this application, the technical background involved in this application will be introduced below.

In spatial dimension gene expression analysis, in order to correspond the gene expression image to the actual spatial position of the biological sample, an image registration step is required to obtain accurate gene expression information. Existing registration technology is mostly based on specific graphic marks on the space-time chip to obtain the microscope image of the biological sample, and then roughly corresponds the position of the microscope image and the gene expression visualization image based on the position of the mark on the microscope image. Based on the characteristics of the image itself and supplemented by artificial visual adjustment, the spatial position of the image captured by the microscope and the gene expression visualization image can be matched. However, existing registration technology can only achieve position matching in the relatively macroscopic dimension of biological samples, resulting in low registration accuracy and inability to support more refined gene expression analysis.

Based on this, embodiments of the present application provide an image processing method to improve the accuracy of image processing. During specific implementation, the first image and the second image are first obtained, wherein the first image and the second image include multiple trajectory lines and multiple trajectory points. The trajectory points are formed by the intersection of the two trajectory lines. Each trajectory Lines have corresponding index numbers. Then select the first trajectory line in the first image, and rotate the first image based on the angle between the first trajectory line and the horizontal direction, that is, rotate the first image to the horizontal direction. The initially obtained first image and the second image may have different scaling ratios, so the first image needs to be enlarged or reduced to obtain a third image, so that the third image has the same ratio as the second image. The target offset is then determined based on the second image and the third image, and the third image is moved based on the target offset. Through the image processing method provided by the embodiment of the present application, the first image and the second image can be registered with pixel-level precision, thereby improving the accuracy of image processing.

The image processing method provided by the embodiment of the present application will be introduced below with reference to the accompanying drawings.

Referring to Figure 1, Figure 1 is a schematic flow chart of an image processing method provided by an embodiment of the present application.

The method specifically includes the following steps:

S101: Obtain the first image and the second image;

The first image and the second image include multiple trajectory lines and/or multiple trajectory points. The trajectory points are formed by the intersection of two trajectory lines, and each trajectory line has a corresponding index number.

Before processing the image, first obtain the first image and the second image. There are multiple trajectory lines and trajectory points in the first image. Each trajectory point is formed by the intersection of two trajectory lines, and each trajectory line has a corresponding index number. There are also multiple trajectory lines and trajectory points in the second image, and the index numbers of the trajectory lines correspond to the index numbers of the trajectory lines in the first image. As shown in Figure 2a and Figure 2b, Figure 2a is a partial image schematic diagram of the first image, and Figure 2b is a partial image schematic diagram of the second image. The numerical labels in Figure 2a represent the index numbers of the trajectory lines in the first image, and the black dots represents the trajectory point formed by the intersection of two trajectory lines. Similarly, the numerical label in Figure 2b represents the index number of the trajectory line in the second image, which corresponds to the trajectory line in the first image. The gray dot represents the two trajectory lines. The trajectory points formed by the intersection.

It should be noted that the schematic diagrams of the first image and the second image provided in the above embodiments are only one possible implementation manner, and do not limit the specific form of the images in any way.

In a possible implementation, the first image can be a microscope image, and the second image can be a gene expression visualization image, a gene expression image or a gene image, that is, pixel-level registration of the microscope image and the gene expression visualization image can be achieved. Among them, the microscope image is obtained by taking a biological sample on the spatio-temporal chip with a microscope, and the gene expression visualization image is based on the same biological sample and generated using a gene expression matrix. There are regularly arranged sites on the spatiotemporal chip for collecting genes of biological samples. Each site can capture the gene sequence of the biological sample. Through gene sequencing, the gene sequence or the number of bases at each site can be obtained, and then the gene sequence or the number of bases can be obtained. The number of gene sequences of the site is used as the value of the element in the gene expression matrix, so that the elements of the gene expression matrix correspond to the sites of the spatiotemporal chip (for example, one element corresponds to one site, the first row and the first column of the gene expression matrix The elements correspond to the position in the upper left corner of the space-time chip) and so on to obtain the gene expression matrix, and then draw a gene expression visualization image based on the gene expression matrix. Among them, the pixels in the gene expression visualization image correspond to the elements of the gene expression matrix, and the pixel values of the pixels correspond to the values of the elements. Therefore, the gene expression visualization image contains spatial information of the same biological sample. Chips can include spatiotemporal chips, sequencing chips, etc.

S102: Obtain the first trajectory line in the first image, and rotate the first image based on the angle between the first trajectory line and the horizontal direction.

When the first image is a microscope image, since the microscope image is obtained by shooting, the trajectory lines in the microscope image are not necessarily in the horizontal and vertical directions, so the first trajectory line in the microscope image can be selected as the benchmark to calculate The angle between the first trajectory line and the horizontal direction is located in the horizontal direction after the microscope image is rotated by the above-mentioned angle. Referring to Figure 3, a schematic diagram of a first image is shown. It can be seen from Figure 3 that the first image is not located in the horizontal direction, and there is an angle between the first trajectory line and the horizontal direction. Therefore, the first image can be rotated to the horizontal direction after calculating the included angle.

When the second image is a gene expression visualization image, since the gene expression visualization image is generated by a gene expression matrix and appears as many discrete points, it has a weak ability to express aggregate features or edge features of biological samples. In a preferred implementation, tissue segmentation can be performed on the gene expression visualization image to obtain the biological sample outline displayed by gene expression, and convolution enhancement processing is performed on the approximate area where the sample outline is located to enhance the edge features of the gene expression visualization image. , to facilitate subsequent registration of microscope images and gene expression visualization images.

S103: Enlarge or reduce the first image to obtain a third image, where the third image has the same proportion as the second image.

When the first image is an image captured by a microscope, since the second image is a gene expression visualization image, it can be used as a reference image, and there may be a difference in the scaling ratio between the first image captured by the microscope and the second image. In order to achieve registration of the first image and the second image, it is necessary to enlarge or reduce the first image to obtain a third image that is the same size as the second image.

In a possible implementation, the scaling ratio of the first image can be determined based on the distance between the trajectory lines corresponding to the index numbers in the first image and the second image. Specifically, the second trajectory line in the first image is obtained, and the first distance is determined based on the first trajectory line and the second trajectory line of the first image; the third trajectory line and the fourth trajectory line in the second image are obtained, wherein , the index number of the third trajectory line is the same as the index number of the first trajectory line, the index number of the fourth trajectory line is the same as the index number of the second trajectory line, and the second distance is determined based on the third trajectory line and the fourth trajectory line. Then, a scaling ratio value between the first image and the second image is determined based on the first spacing and the second spacing, and the first image is enlarged or reduced according to the above scaling value to obtain a third image with the same ratio as the second image. The first trajectory line and the second trajectory line are parallel; the third trajectory line and the fourth trajectory line are parallel.

When the first distance is determined based on the first trajectory line and the second trajectory line, it can be calculated using the trajectory points on the first trajectory line and the trajectory points on the second trajectory line. When the two trajectory points have the same abscissa or vertical coordinate, When coordinates are used, the difference between the corresponding ordinates or the abscissas can be used as the first spacing. Similarly, the second spacing between the third trajectory line and the fourth trajectory line can be determined based on the same principle. It should be noted that the method for calculating the scaling ratio of the first image and the second image provided in the above embodiments is only an exemplary description and is not limited to the above implementation.

S104: Determine the target offset based on the second image and the third image.

When the first image is scaled to the third image and has the same proportion as the second image, the target offset can be determined based on the second image and the third image. It can be seen from the above embodiment that after the original first image is rotated to the horizontal direction, there may be a 90-degree angle difference between the first image and the second image. In order to further improve the accuracy of image processing, the first image is scaled to After the third image has the same proportion as the second image, the third image can be rotated according to a preset rotation angle, and the corresponding target offset is determined at each rotation angle. The following will be explained with a specific application scenario.

As shown in Figure 4, S represents the third image, V represents the second image, the black point represents the center of gravity of the third image, and the gray point represents the center of gravity of the second image. The four areas in Figure 4 represent the original third image respectively. image and the image after rotating the third image clockwise by 90 degrees, 180 degrees, and 270 degrees. After the third image is rotated, the alignment can be performed based on the origin coordinates of the third image and the origin coordinates of the second image, and then the target offset is determined based on the aligned third image and the second image.

For any preset rotation angle, in order to process the image more conveniently and accurately, this embodiment provides an optimal implementation method, that is, first, the third image can be preliminarily processed according to the coordinates of the center of gravity of the third image and the second image. Position it to roughly the same position as the second image, and then calculate the target offset based on the moved third image to perform precise positioning. In specific implementation, the first image is enlarged or reduced to obtain a fifth image, where the fifth image has the same proportion as the second image. Then calculate the first gravity center coordinates of the fifth image and the second gravity center coordinates of the second image respectively, determine the gravity center offset based on the first gravity center coordinates and the second gravity center coordinates, and move the fifth image based on the gravity center offset, and get The fourth image is the fourth image from which preliminary positioning is obtained. When calculating the centroid coordinates of the fifth image and the second image, a density centroid algorithm based on pixel gray values can be used, which is not limited in this embodiment.

It should be noted that regarding the image movement in the intermediate process, the processing process can be implemented on the temporary image that copies the first image, that is, the first image is not moved and registered. After the calculation is completed, the final target offset is obtained. , and then move the first image all at once to register with the second image.

After the initially determined fourth image is obtained, the target offset can be determined based on the fourth image and the second image. In practical applications, the trajectory lines on the space-time chip may have multiple periods, so the microscope images and gene expression visualization images obtained based on the space-time chip may also have multiple periods, that is, the fourth image and the second image have multiple periods. cycle. There is a corresponding sub-image in each cycle, the number of trajectory lines and trajectory points in each sub-image is the same, and the index numbers of the trajectory lines between the sub-images correspond one to one. As shown in Figures 5a and 5b, Figure 5a shows a third image with multiple cycles, and Figure 5b shows a second image with multiple cycles. In Figure 5a, only the sub-images of the first period and the second period are shown. It can be seen that the sub-images of each period have the same number of index lines and index points, and each index line corresponds to an index number. The sub-images corresponding to the first period and the second period are also shown in Figure 5b. The schematic diagrams of the third image and the second image provided in the above embodiments are only one possible implementation manner, and do not limit the specific form of the images in any way. Therefore, when determining the target offset based on the fourth image and the second image, the influence of the period needs to be considered.

During specific implementation, first the first offset can be determined based on the fourth image and the second image, where the first offset is the undetermined offset between the fourth image and the second image, and then the first offset can be used to determine the offset between the fourth image and the second image. As the center, determine a plurality of second offsets corresponding to the first offset within the preset period, that is, each period within the preset period corresponds to a second offset. For any second offset, move the fourth image based on the second offset to obtain the sixth image, and then calculate the first similarity between the sixth image and the second image, thereby obtaining the result within the preset period. The first similarity corresponding to each cycle of . Since the above determined similarity is obtained at any rotation angle, it is necessary to re-execute the above steps at other preset rotation angles to obtain multiple first similarities corresponding to each rotation angle. Based on the plurality of first similarities determined by all rotation angles and all cycles, a first maximum similarity is determined, and the sixth image corresponding to the first maximum similarity is determined as the third image. Then the second offset corresponding to the third image can be determined as the target offset.

When determining the first offset based on the fourth image and the second image, one possible implementation is to first obtain multiple sets of trajectory point sets, where each set of trajectory point sets includes a first trajectory point and a second trajectory point. , the first trajectory point is located in the fourth image, the second trajectory point is located in the second image, and the first trajectory point and the second trajectory point have the same index identifier. Since a trajectory point is formed by two intersecting trajectory lines, and each trajectory line has an index number, the index identification of the trajectory point can be composed of the index numbers of the two intersecting trajectory lines of the trajectory point. In this embodiment, since there are multiple trajectory points in both the fourth image and the second image, it is necessary to obtain trajectory points with corresponding relationships before they can be used as a set of trajectory points, that is, trajectory points with the same index number. For any set of track points among the multiple sets of track points, the fourth offset may be determined based on the coordinates of the first track point and the coordinates of the second track point, and then the first offset may be determined based on a plurality of fourth offsets. Shift amount.

In order to facilitate a clearer understanding of this solution, before introducing the implementation method of the third offset, the implementation method of the fourth offset is first described.

During specific implementation, since the second image has multiple cycles, that is, multiple sub-images, the first trajectory point and the second trajectory point obtained with the same index identification may not belong to the corresponding cycle, so the determined first trajectory point There may be multiple trajectory points. When determining the fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point, any first trajectory point in the fourth image can be first selected, and the index identification of the first trajectory point is determined to have the same identification. multiple second trajectory points. In order to obtain the second trajectory point corresponding to the first trajectory point, the Euclidean distance between the first trajectory point and each second trajectory point can be calculated separately. After calculating multiple Euclidean distances, obtain the Euclidean distance corresponding to the minimum Euclidean distance. The second trajectory point is the second trajectory point corresponding to the first trajectory point. Then, the fourth offset is determined based on the coordinate difference between the second trajectory point corresponding to the minimum Euclidean distance and the first trajectory point. Since there are multiple trajectory points in the fourth image, that is, corresponding to multiple sets of trajectory points, it is necessary to traverse all the trajectory point sets, and determine the position corresponding to each set of trajectory points according to the above method of determining the basic offset. Fourth offset. In a possible implementation, among the plurality of determined fourth offsets, fourth offsets that exceed the preset range may be first excluded, and then the target offset is determined based on the remaining fourth offsets. For example, the median of multiple fourth offsets can be selected as the target offset, or the average of all fourth offsets can be calculated as the target offset to maximize the accuracy of determining the offset. sex. It should be noted that this embodiment does not limit the specific manner of determining the target offset based on the fourth offset.

The image movement within the preset period will be described below based on a specific application scenario.

Referring to FIG. 6 , FIG. 6 is a schematic diagram of a preset period determined with the first offset as the center.

In one possible implementation, the second image is used as the benchmark. As shown in Figure 6, the middle gray area represents the period in which the second trajectory point is located. After determining the first offset based on the first trajectory point and the second trajectory point, Taking the first offset (the period to which the corresponding second trajectory point belongs) as the center, select nearby 5*5 cycles to determine the second offset corresponding to each cycle, and the second offset corresponding to the adjacent cycle. The difference in shift amount is the width of the sub-image, and by analogy, the second offset corresponding to each cycle can be determined.

Then the fourth image can be moved according to the second offset corresponding to any period to obtain a sixth image, and the similarity between the sixth image and the second image can be calculated. When calculating the similarity, this embodiment provides a possible implementation method, that is, perform frequency domain transformation on the sixth image to obtain the first frequency domain image, perform frequency domain transformation on the second image to obtain the second frequency domain image, and calculate the third frequency domain image. The Hamming distance between the first frequency domain image and the second frequency domain image is used as the similarity. When the Hamming distance is smaller, it indicates that the similarity between the first frequency domain image and the second frequency domain image is greater. After determining the Hamming distances corresponding to all preset rotation angles and all preset periods, the minimum Hamming distance is used as the first maximum similarity.

Among them, discrete cosine transform can be used to obtain the frequency domain image, that is, discrete cosine transform is performed on the moved third image to obtain the first frequency domain image, and discrete cosine transform is performed on the second image to obtain the second frequency domain image, and then the first frequency domain image is obtained by discrete cosine transform. The low-frequency frequency domain image is intercepted from the frequency domain image. Based on the low-frequency frequency domain image, the pixel mean of the low-frequency frequency domain image is calculated. In the low-frequency frequency domain image, the pixel value of the pixel point with a pixel value higher than the pixel mean is set to 1. Set the pixel value of the pixel point whose pixel value is lower than the pixel mean value to 0, set the pixel value of the pixel point equal to the pixel mean value to 0 or 1, and then obtain the hash fingerprint based on the reset low-frequency frequency domain image. . Follow the same execution steps to obtain the hash fingerprint of the second frequency domain image. Then based on the hash fingerprint of the first frequency domain image and the hash fingerprint of the second frequency domain image, it is determined how many different values there are in the comparison of the two hash fingerprints, that is, as the Han of the first frequency domain image and the second frequency domain image. clear distance. It should be noted that the method of calculating similarity in the above embodiment is only an exemplary implementation, and is not limited to the above implementation. Other implementable methods of calculating image similarity are also within the protection scope of this application.

In the above-mentioned embodiment of the present application, the fourth image can be initially moved and positioned according to the offset of the center of gravity from the second image. In another possible implementation, when determining the third image, the third image is not The four images are initially positioned, and for any preset rotation angle, the fourth image is moved according to each period within the preset period to obtain a seventh image corresponding to each period. For example, the third barycenter coordinate of the fourth image can be calculated first, and the fourth image can be moved according to each period within the preset period with the period where the third barycenter coordinate is located as the center to obtain the seventh image. Then the second similarity between the seventh image and the second image is calculated, thereby obtaining the second similarity corresponding to each period within the preset period. Then repeat the above steps according to other preset rotation angles to obtain the second similarity corresponding to each preset rotation angle and each period, and determine the second maximum similarity based on the plurality of second similarities. . The seventh image corresponding to the second maximum similarity is used as the third image, and then the target offset is determined based on the third image and the second image.

Specifically, first, a plurality of third offsets may be determined based on the second image and the third image, and the target offset may be determined based on the plurality of third offsets. The calculation principle of the third offset is the same as the calculation principle of the fourth offset in the above embodiment. That is, multiple groups of trajectory point sets are obtained, wherein each group of trajectory point sets includes a third trajectory point and a fourth trajectory point, the third trajectory point is located in the third image, the fourth trajectory point is located in the second image, and the third trajectory point It has the same index identifier as the fourth track point. For any set of trajectory points, a third offset is determined based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point. For a specific implementation method of determining the third offset based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point, please refer to the specific implementation of determining the fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point. The implementation method will not be described again here.

Similarly, after determining the third offsets corresponding to the multiple sets of trajectory points, the third offsets that exceed the preset range can be first excluded, and then selected from a plurality of third offsets that meet the preset range. The median is used as the target offset, or the average of multiple third offsets is calculated as the target offset to maximize the accuracy of determining the offset.

S105: Move the third image based on the target offset.

After the target offset is determined based on the method provided by the above embodiment, a preset rotation angle corresponding to the target offset is determined, and the third image is moved based on the preset rotation angle and the target offset.

It should be noted that the image movement in the intermediate process is a process implemented on the temporary image that copies the first image, that is, the first image is not moved and registered. When the calculation is completed, the final target offset is obtained. After that, the first image is moved once again to register with the second image.

In the image processing method provided by the embodiments of the present application, the first image and the second image can be registered based on the image coordinates with pixel-level accuracy, thereby improving the accuracy of image processing and facilitating subsequent use of the registered images for research and development. analyze.

When the third image and the second image have multiple periods, since the offset corresponding to the preset period is inferred from the calculated offset using the theoretical period width, errors may exist in the actual image. In order to correct errors in theoretical reasoning and further improve the accuracy of image processing, in a preferred implementation, after moving the third image based on the target offset, based on the moved third image and the second image, the image can be re-imaged. Use the third image and the second image to determine the correction offset as the final offset, and continue to move the third image based on the determined final offset, and register the third image with the second image to improve Registration accuracy. For example, based on the moved third image and the second image, multiple groups of trajectory point sets are obtained, wherein each group of trajectory point sets includes a fifth trajectory point and a sixth trajectory point, and the fifth trajectory point is located in the third image, and the The sixth track point is located in the second image, and the fifth track point and the sixth track point have the same index identification. For any set of trajectory points, a fifth offset is determined based on the coordinates of the fifth trajectory point and the coordinates of the sixth trajectory point. After determining the fifth offset corresponding to each set of track points in the plurality of sets of track points, the final offset is determined based on the plurality of fifth offsets. The implementation of determining the fifth offset and determining the final offset based on multiple fifth offsets may refer to the above embodiments and will not be described again here.

The image processing method provided by the embodiment of the present application will be introduced below in conjunction with an application scenario. Referring to Figure 7, Figure 7 is a schematic flow chart of another image processing method provided by an embodiment of the present application.

In this application scenario, the first image is a microscope image, and the second image is a gene expression visualization image. After acquiring the microscope image and the gene expression visualization image, preprocess the image, for example, rotate the microscope image to the horizontal direction and scale the microscope image to the same scale as the gene expression visualization image. In addition, convolution enhancement processing can be performed on the gene expression visualization image to enhance the edge features of the image. In this embodiment, the microscope image can be rotated to four directions, namely the original horizontal direction, a clockwise rotation of 90 degrees, a clockwise rotation of 180 degrees, and a clockwise rotation of 270 degrees. The same process is performed in any direction. Image processing steps. That is, the center of gravity of the microscope image and the gene expression visualization image are calculated respectively, and the microscope image is initially positioned. The basic offset is then determined using the coordinates of the trajectory points identified by the corresponding index in the microscope image and gene expression visualization image. Taking the basic offset as the center, traverse the basic offset corresponding to the preset period, calculate the image similarity under the corresponding period, perform the above steps for each rotation direction, and determine the corresponding direction of each rotation and each period. Image similarity, select the period with the highest similarity as the benchmark, determine the basic offset corresponding to the period as the target offset, and then move the microscope image based on the target offset to achieve image registration.

The beneficial effect parameters of the image processing method provided by this embodiment are not described in detail here.

Based on the above method embodiments, embodiments of the present application also provide an image processing device. Referring to Figure 8, the device 800 includes:

The first acquisition unit 801 is used to acquire a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and/or a plurality of trajectory points, and the trajectory points are formed by Formed by the intersection of two of the trajectory lines, each of the plurality of trajectory lines has a corresponding index number;

The second acquisition unit 802 is used to acquire the first trajectory line in the first image, and rotate the first image based on the angle between the first trajectory line and the horizontal direction;

The first processing unit 803 is configured to enlarge or reduce the first image to obtain a third image, where the third image has the same proportion as the second image;

A second processing unit 804 configured to determine a target offset based on the second image and the third image;

The third processing unit 805 is configured to move the third image based on the target offset.

In a possible implementation, the first processing unit 803 is specifically configured to obtain the second trajectory line in the first image, and determine the first distance based on the first trajectory line and the second trajectory line; Obtain the third trajectory line and the fourth trajectory line in the second image, and determine the second distance based on the third trajectory line and the fourth trajectory line, wherein the index number of the third trajectory line is the same as the index number of the third trajectory line. The index number of the first trajectory line is the same, the index number of the fourth trajectory line is the same as the index number of the second trajectory line; the proportion value is determined based on the first spacing and the second spacing, and the The first image is enlarged or reduced according to the ratio value to obtain a third image.

In a possible implementation, the first processing unit 803 is specifically configured to enlarge or reduce the first image to obtain a fourth image, where the fourth image has the same proportion as the second image; The fourth image moves according to a preset period and a preset rotation angle to obtain the third image.

In a possible implementation, the first processing unit 803 is specifically configured to enlarge or reduce the first image to obtain a fifth image, where the fifth image and the second image have the same proportion; respectively Calculate the first gravity center coordinate of the fifth image and the second gravity center coordinate of the second image; determine the gravity center offset based on the first gravity center coordinate and the second gravity center coordinate; based on the gravity center offset The fifth image is moved to obtain the fourth image.

In a possible implementation, the first processing unit 803 is specifically configured to obtain the preset period and multiple preset rotation angles; for any preset rotation angle, based on the fourth image and the third The second image determines a first offset; determines a plurality of second offsets corresponding to the first offset within the preset period, and each second offset in the plurality of second offsets is determined. The offset corresponds to each period within the preset period; for any second offset, move the fourth image based on the second offset to obtain a sixth image; determine the a first similarity between the sixth image and the second image; determining a first maximum similarity based on the plurality of preset rotation angles and the first similarity corresponding to each period; determining the first similarity The sixth image corresponding to a maximum similarity is the third image.

In a possible implementation, the second processing unit 804 is specifically configured to determine the second offset corresponding to the third image as the target offset;

The third processing unit 805 is specifically configured to move the third image based on the preset rotation angle corresponding to the third image and the target offset.

In a possible implementation, the first processing unit 803 is specifically configured to obtain the preset period and the plurality of preset rotation angles; for any preset rotation angle, the fourth image is processed according to Each period within the preset period is moved to obtain a plurality of seventh images, and each seventh image in the plurality of seventh images corresponds to each period within the preset period; for any a seventh image, determining a second degree of similarity between the seventh image and the second image; determining a second degree of similarity based on the plurality of preset rotation angles and the second degree of similarity corresponding to each period. Maximum similarity; determine the seventh image corresponding to the second maximum similarity as the third image.

In a possible implementation, the second processing unit 804 is specifically configured to determine a plurality of third offsets based on the second image and the third image; determine a plurality of third offsets based on the plurality of third offsets. The target offset;

The third processing unit 805 is specifically configured to move the third image based on the preset rotation angle corresponding to the third image and the target offset.

In a possible implementation, the first processing unit 803 is specifically configured to obtain multiple sets of trajectory point sets, each of the multiple sets of trajectory point sets including a first trajectory point and a second trajectory point, The first trajectory point is located in the fourth image, the second trajectory point is located in the second image, and the first trajectory point and the second trajectory point have the same index identifier; for any group of trajectories point set, determining a fourth offset based on the coordinates of the first track point and the coordinates of the second track point under the track point set; based on the fourth offset corresponding to the multiple groups of track point sets The amount determines the first offset.

In a possible implementation, when the second image has multiple sub-images, the trajectory lines of each sub-image have the same index number, and the first processing unit 803 is specifically used to perform tracking based on the first trajectory point. The index identification determines the second trajectory point in each sub-image, and the first trajectory point and the second trajectory point have the same index identification; determines the first trajectory point and a plurality of the second trajectory points. The Euclidean distance between any second trajectory points among the trajectory points; determining the second trajectory point corresponding to the minimum Euclidean distance among a plurality of the Euclidean distances; based on the coordinates of the first trajectory point and the minimum Euclidean distance The coordinates of the corresponding second trajectory point determine the fourth offset amount.

In a possible implementation, the first processing unit 803 is specifically configured to perform frequency domain transformation on the sixth image to obtain a first frequency domain image; perform frequency domain transformation on the second image to obtain a second frequency domain image. image; calculate the Hamming distance between the first frequency domain image and the second frequency domain image as the first similarity; the first processing unit 803 is also configured to based on the plurality of preset rotation angles and the Describe the Hamming distance corresponding to each period, and determine the minimum Hamming distance as the first maximum similarity.

In a possible implementation, the first processing unit 803 is specifically configured to calculate the third barycenter coordinate of the fourth image; taking the period in which the third barycenter coordinate is located as the center, calculate the third barycenter coordinate according to each of the The fourth image is moved in cycles to obtain the seventh image.

In a possible implementation, the first processing unit 803 is specifically configured to obtain multiple sets of trajectory point sets, each of the multiple sets of trajectory point sets including a third trajectory point and a fourth trajectory point, The third trajectory point is located in the third image, the fourth trajectory point is located in the second image, and the third trajectory point and the fourth trajectory point have the same index identifier; for any group of trajectories Point set, determining a third offset based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point under the trajectory point set.

In a possible implementation, the device 800 is further configured to obtain multiple sets of trajectory point sets based on the moved third image and the second image. Each set of trajectories in the multiple sets of trajectory point sets is The point set includes a fifth trajectory point and a sixth trajectory point. The fifth trajectory point is located in the third image after the movement. The sixth trajectory point is located in the second image. The fifth trajectory point It has the same index identifier as the six track points; for any set of track points, the fifth offset is determined based on the coordinates of the fifth track point and the coordinates of the sixth track point under the set of track points. amount; determine the final offset amount based on the fifth offset amount corresponding to the plurality of sets of trajectory points; and move the moved third image based on the final offset amount.

In a possible implementation, the first acquisition unit 801 is specifically configured to acquire an eighth image, and perform convolution enhancement processing on the eighth image to obtain the second image.

Based on the above method embodiments and device embodiments, embodiments of the present application also provide an electronic device. Referring to Figure 9, the electronic device 900 includes: a memory 901 and a processor 902;

The memory 901 is used to store instructions or computer programs;

The processor 902 is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the image processing method described in the above method embodiment.

In addition, embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium being used to store a computer program, and the computer program being used to execute the image processing method described in the above method embodiment.

Embodiments of the present application also provide a computer program product. The computer program product includes a program. When the program is run on a processor, the computer or network device causes the computer or network device to execute the image processing method described in the above method embodiment.

It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on its differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. In particular, the system or device embodiments are described simply because they are basically similar to the method embodiments. For relevant parts, please refer to the partial description of the method embodiments. The device embodiments described above are only illustrative, in which units or modules illustrated as separate components may or may not be physically separated, and components shown as units or modules may or may not be physical modules, that is, It can be located in one place, or it can also be distributed to multiple network units. Some or all of the units or modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.

It should be understood that in this application, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And/or" is used to describe the relationship between associated objects, indicating that there can be three relationships. For example, "A and/or B" can mean: only A exists, only B exists, and A and B exist simultaneously. , where A and B can be singular or plural. The character "/" generally indicates that the related objects are in an "or" relationship. “At least one of the following” or similar expressions thereof refers to any combination of these items, including any combination of a single item (items) or a plurality of items (items). For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c" ”, where a, b, c can be single or multiple.

It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is no such actual relationship or sequence between them. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the stated element.

The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination of both. Software modules may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or anywhere in the field of technology. any other known form of storage media.

The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be practiced in other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (19)

1. An image processing method, characterized in that the method includes:

   Obtaining a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and/or a plurality of trajectory points, the trajectory points are formed by the intersection of the two trajectory lines , each of the plurality of trajectory lines has a corresponding index number;

   Obtain the first trajectory line in the first image, and rotate the first image based on the angle between the first trajectory line and the horizontal direction;

   Enlarging or reducing the first image to obtain a third image, the third image having the same proportion as the second image;

   determining a target offset based on the second image and the third image;

   The third image is moved based on the target offset.
2. The method of claim 1, wherein enlarging or reducing the first image to obtain a third image includes:

   Obtain a second trajectory line in the first image, and determine a first distance based on the first trajectory line and the second trajectory line;

   Obtain the third trajectory line and the fourth trajectory line in the second image, and determine the second distance based on the third trajectory line and the fourth trajectory line, wherein the index number of the third trajectory line is the same as the index number of the third trajectory line. The index number of the first trajectory line is the same, the index number of the fourth trajectory line is the same as the index number of the second trajectory line;

   A ratio value is determined based on the first distance and the second distance, and the first image is enlarged or reduced according to the ratio value to obtain a third image.
3. The method of claim 1, wherein enlarging or reducing the first image to obtain a third image includes:

   Enlarging or reducing the first image to obtain a fourth image, the fourth image having the same proportion as the second image;

   The fourth image is moved according to a preset period and a preset rotation angle to obtain the third image.
4. The method of claim 3, wherein enlarging or reducing the first image to obtain a fourth image includes:

   Enlarging or reducing the first image to obtain a fifth image, where the fifth image has the same proportion as the second image;

   Calculate the first barycenter coordinates of the fifth image and the second barycenter coordinates of the second image respectively;

   Determine a center of gravity offset based on the first center of gravity coordinates and the second center of gravity coordinates;

   The fifth image is moved based on the gravity center offset to obtain the fourth image.
5. The method according to claim 3 or 4, characterized in that said moving the fourth image according to a preset period and a preset rotation angle to obtain the third image includes:

   Obtain the preset period and multiple preset rotation angles;

   For any preset rotation angle, determine a first offset based on the fourth image and the second image;

   Determine a plurality of second offsets corresponding to the first offset within the preset period, and each second offset in the plurality of second offsets is consistent with the preset period. Each cycle within has a one-to-one correspondence;

   For any second offset, move the fourth image based on the second offset to obtain a sixth image;

   determining a first degree of similarity between the sixth image and the second image;

   Determine a first maximum similarity based on the plurality of preset rotation angles and the first similarity corresponding to each cycle;

   The sixth image corresponding to the first maximum similarity is determined to be the third image.
6. The method of claim 5, wherein determining the target offset based on the second image and the third image includes:

   Determine the second offset corresponding to the third image as the target offset;

   The moving the third image based on the target offset includes:

   The third image is moved based on the preset rotation angle corresponding to the third image and the target offset.
7. The method of claim 3, wherein moving the fourth image according to a preset period and a preset rotation angle to obtain the third image includes:

   Obtain the preset period and the plurality of preset rotation angles;

   For any preset rotation angle, the fourth image is moved according to each period within the preset period to obtain a plurality of seventh images, and each seventh image in the plurality of seventh images is related to Each period within the preset period has a one-to-one correspondence;

   For any seventh image, determine a second degree of similarity between the seventh image and the second image;

   Determine a second maximum similarity based on the plurality of preset rotation angles and the second similarity corresponding to each cycle;

   The seventh image corresponding to the second maximum similarity is determined to be the third image.
8. The method of claim 7, wherein determining the target offset based on the second image and the third image includes:

   determining a plurality of third offsets based on the second image and the third image;

   determining the target offset based on the plurality of third offsets;

   The moving the third image based on the target offset includes:

   The third image is moved based on the preset rotation angle corresponding to the third image and the target offset.
9. The method of claim 5, wherein determining the first offset based on the fourth image and the second image includes:

   Acquire multiple sets of trajectory point sets. Each set of trajectory points in the multiple sets of trajectory point sets includes a first trajectory point and a second trajectory point. The first trajectory point is located in the fourth image, and the second trajectory point is located in the fourth image. The point is located in the second image, and the first trajectory point and the second trajectory point have the same index identifier;

   For any set of trajectory points, determine a fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point under the set of trajectory points;

   The first offset is determined based on the fourth offset corresponding to the plurality of sets of trajectory points.
10. The method according to claim 9, characterized in that when the second image has multiple sub-images, the trajectory line of each sub-image has the same index number, and the trajectory line based on the trajectory point set is The coordinates of a trajectory point and the coordinates of the second trajectory point determine a fourth offset, including:

    Determine the second trajectory point in each sub-image based on the index identification of the first trajectory point, where the first trajectory point and the second trajectory point have the same index identification;

    Determine the Euclidean distance between the first trajectory point and any second trajectory point among the plurality of second trajectory points;

    Determine the second trajectory point corresponding to the minimum Euclidean distance among the plurality of Euclidean distances;

    The fourth offset is determined based on the coordinates of the first trajectory point and the coordinates of the second trajectory point corresponding to the minimum Euclidean distance.
11. The method of claim 5, wherein determining the first similarity between the sixth image and the second image includes:

    Perform frequency domain transformation on the sixth image to obtain a first frequency domain image;

    Perform frequency domain transformation on the second image to obtain a second frequency domain image;

    Calculate the Hamming distance between the first frequency domain image and the second frequency domain image as the first similarity;

    Determining the first maximum similarity based on the plurality of preset rotation angles and the first similarity corresponding to each cycle includes:

    Based on the plurality of preset rotation angles and the Hamming distance corresponding to each period, the minimum Hamming distance is determined to be the first maximum similarity.
12. The method according to claim 7, wherein said moving the fourth image according to each period within the preset period to obtain a seventh image includes:

    Calculate the coordinates of the third center of gravity of the fourth image;

    Taking the period where the third barycenter coordinate is located as the center, the fourth image is moved according to each period to obtain the seventh image.
13. The method of claim 8, wherein determining a plurality of third offsets based on the second image and the third image includes:

    Acquire multiple sets of trajectory point sets, each of the multiple sets of trajectory point sets includes a third trajectory point and a fourth trajectory point, the third trajectory point is located in the third image, and the fourth trajectory point The point is located in the second image, and the third trajectory point and the fourth trajectory point have the same index identifier;

    For any set of trajectory points, a third offset is determined based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point under the set of trajectory points.
14. The method of claim 1, wherein after moving the third image based on the target offset, the method further includes:

    Based on the moved third image and the second image, multiple sets of trajectory point sets are obtained, each of the multiple sets of trajectory point sets includes a fifth trajectory point and a sixth trajectory point, The fifth trajectory point is located in the third image after the movement, the sixth trajectory point is located in the second image, and the fifth trajectory point and the six trajectory points have the same index identifier;

    For any group of trajectory point sets, determine a fifth offset based on the coordinates of the fifth trajectory point and the coordinates of the sixth trajectory point under the trajectory point set;

    Determine the final offset based on the fifth offset corresponding to the multiple sets of trajectory points;

    The moved third image is moved based on the final offset.
15. The method according to any one of claims 1 to 14, wherein obtaining the second image includes:

    An eighth image is acquired, and convolution enhancement processing is performed on the eighth image to obtain the second image.
16. An image processing device, characterized in that the device includes:

    A first acquisition unit, configured to acquire a first image and a second image, wherein the first image and the second image include a plurality of trajectory lines and/or a plurality of trajectory points, and the trajectory points are composed of two The trajectory lines are formed by intersecting, and each trajectory line in the plurality of trajectory lines has a corresponding index number;

    a second acquisition unit, configured to acquire the first trajectory line in the first image, and rotate the first image based on the angle between the first trajectory line and the horizontal direction;

    A first processing unit configured to enlarge or reduce the first image to obtain a third image, where the third image has the same proportion as the second image;

    a second processing unit configured to determine a target offset based on the second image and the third image;

    A third processing unit configured to move the third image based on the target offset.
17. An electronic device, characterized in that the electronic device includes: a memory and a processor;

    The memory is used to store instructions or computer programs;

    The processor is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the image processing method according to any one of claims 1 to 15.
18. A computer-readable storage medium, characterized in that the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the image processing method described in any one of claims 1 to 15.
19. A computer program product, characterized in that the computer program product includes a program, which when the program is run on a processor, causes the computer or network device to execute the image processing method described in any one of claims 1 to 15.

Images