CN113643180B - Image registration method, device, equipment and medium - Google Patents

Abstract

The present disclosure relates to an image registration method, apparatus, device, and medium; wherein the method comprises the following steps: acquiring a reference image and an original image to be registered; respectively converting the reference image and the original image according to the Hilbert curve to obtain a first one-dimensional vector corresponding to the reference image and a second one-dimensional vector corresponding to the original image; extracting a first feature vector from the first one-dimensional vector, and extracting a second feature vector from the second one-dimensional vector; performing feature point matching on the first feature vector and the second feature vector to obtain feature point pairs; and carrying out affine transformation processing on the reference image according to the characteristic point pairs to obtain a target image registered with the original image. The present disclosure can increase the speed and accuracy of image registration.

Description

Image registration method, device, equipment and medium

Technical Field

The present disclosure relates to the field of image processing technology, and in particular to an image registration method, device, equipment and medium.

Background technique

Image registration is the process of matching and superimposing two or more images of the same object under different conditions (such as equipment, time, lighting, angle, etc.). At present, the general method of image registration is to first use image feature points for matching, and then calculate the homography matrix of the two images to be registered by random sampling and other methods to achieve image registration. However, in the face of images with complex features such as uneven lighting and angular distortion, the above general image registration method is less effective, resulting in low accuracy of image registration between images.

Summary of the invention

In order to solve the above technical problem or at least partially solve the above technical problem, the present disclosure provides an image registration method, device, equipment and medium.

The present disclosure provides an image registration method, comprising:

Acquire a reference image and an original image to be registered; transform the reference image and the original image respectively according to the Hilbert curve to obtain a first one-dimensional vector corresponding to the reference image and a second one-dimensional vector corresponding to the original image; wherein the first one-dimensional vector is used to characterize the sequence relationship between pixel points in the reference image, and the second one-dimensional vector is used to characterize the sequence relationship between pixel points in the original image; extract a first eigenvector from the first one-dimensional vector, and extract a second eigenvector from the second one-dimensional vector; perform feature point matching on the first eigenvector and the second eigenvector to obtain a feature point pair; perform affine transformation processing on the reference image according to the feature point pair to obtain a target image registered with the original image.

The present disclosure provides an image registration device, comprising:

An image acquisition module is used to acquire a reference image and an original image to be registered; an image conversion module is used to convert the reference image and the original image respectively according to the Hilbert curve to obtain a first one-dimensional vector corresponding to the reference image and a second one-dimensional vector corresponding to the original image; wherein the first one-dimensional vector is used to characterize the sequence relationship between pixel points in the reference image, and the second one-dimensional vector is used to characterize the sequence relationship between pixel points in the original image; a feature extraction module is used to extract a first feature vector from the first one-dimensional vector and a second feature vector from the second one-dimensional vector; a matching module is used to perform feature point matching on the first feature vector and the second feature vector to obtain a feature point pair; a registration module is used to perform affine transformation processing on the reference image according to the feature point pair to obtain a target image registered with the original image.

The present disclosure provides an electronic device, comprising: a processor; and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the above-mentioned image registration method.

The present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute an image registration method.

The present disclosure provides a computer program product, including a computer program, which implements the above-mentioned image registration method when executed by a processor.

Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

The disclosed embodiments provide an image registration method, apparatus, device and medium, which first converts the acquired reference image and original image respectively according to the Hilbert curve to obtain the corresponding first one-dimensional vector and second one-dimensional vector; then extracts the first feature vector from the first one-dimensional vector, and extracts the second feature vector from the second one-dimensional vector; finally, performs feature point matching on the first feature vector and the second feature vector, and performs affine transformation processing on the reference image according to the obtained feature point pairs to obtain a target image registered with the original image. The disclosed embodiments can effectively increase the speed and accuracy of image registration.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a flow chart of an image registration method provided by an embodiment of the present disclosure;

FIG2 is a schematic diagram of an original image and a reference image provided by an embodiment of the present disclosure;

FIG3 is a schematic diagram of the structure of an image registration device provided by an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

Detailed ways

In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and/or in parallel. In addition, the method embodiments may include additional steps and/or omit the steps shown. The scope of the present disclosure is not limited in this respect.

The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

When using existing image registration methods to process images with complex features, the accuracy of image registration is low and the effect is poor. For example, in a common application scenario, when faced with questions with semantic information (such as multiple choice, fill-in-the-blank, and true or false questions), users generally use a combination of photo-taking and image question banks to correct these questions. Due to various reasons such as writing habits and photo-taking scenes, the text images taken by users may have a lot of problems such as back-transparency (that is, writing on both sides of the same page causes one side to affect the other side), uneven lighting, photocopying, and incorrect shooting angles (refer to the original image shown in Figure 2); therefore, in this way, how to more accurately match the text image taken by the user with the answer area of the image in the question bank will have a great impact on the accuracy of photo-taking. Among them, a more effective way to achieve area correspondence is to use image registration. It can be seen that the accuracy of photo-taking is heavily dependent on the effect of image registration. However, the current image registration method has poor registration effect on text images with more complex changes, which in turn restricts the accuracy of photo-taking.

Based on the above considerations, the embodiments of the present disclosure provide an image registration method, device, equipment and medium, which can be widely used in target detection, model reconstruction, motion estimation, feature matching, tumor detection, lesion localization, angiography, geological exploration, aerial reconnaissance and other fields. For ease of understanding, the embodiments of the present disclosure are described in detail below.

Referring to the flowchart of an image registration method provided in FIG1 , the method may include the following steps:

Step S102: Acquire a reference image and an original image to be registered.

The reference image and the original image in this embodiment can be images in various scenarios requiring image registration, such as image enhancement, image shape analysis, and images in image stitching scenarios. The reference image and the original image are two-dimensional images containing the same content, wherein the original image is an image that needs to be registered with the reference image as the registration target. In practical applications, the original image can be an image obtained by a user through an image selection operation, an image shooting operation, or an image upload operation in a terminal, as shown in FIG2 , specifically, for example, a text image obtained by shooting a target test question, and correspondingly, the reference image specifically, for example, is an image including the target test question in a preset test question library.

In some implementations, the reference image and the original image may both be grayscale images that have been pre-processed in grayscale, that is, the reference image and the original image are both grayscale images.

Step S104, converting the reference image and the original image respectively according to the Hilbert curve to obtain a first one-dimensional vector corresponding to the reference image and a second one-dimensional vector corresponding to the original image; wherein the first one-dimensional vector is used to represent the sequence relationship between pixel points in the reference image, and the second one-dimensional vector is used to represent the sequence relationship between pixel points in the original image.

The Hilbert curve is a space-filling curve; a space-filling curve is a function curve that uses a one-dimensional curve to contain the entire two-dimensional or even multi-dimensional space. The discrete approximate representation method of the Hilbert curve is very practical. It can well preserve spatial proximity when converting multi-dimensional space into one-dimensional space. A high-order Hilbert curve can be used to fill the two-dimensional plane, and then the curve can be expanded. Adjacent pixels on the two-dimensional plane are still in adjacent positions on the one-dimensional Hilbert curve.

In this embodiment, the method of converting the reference image and the original image according to the Hilbert curve is the same, and the conversion method is described here only by taking the original image as an example. The Hilbert curve can be used to represent regular text or irregular text in text recognition scenarios of any shape; this embodiment can select a Hilbert curve of a suitable order according to the text density or pixel size of the original image, and arrange all the pixels in the original image into a one-dimensional vector according to the scanning order of the Hilbert curve to obtain a second one-dimensional vector corresponding to the original image. It can be understood that the elements with a sequence relationship in the second one-dimensional vector are the pixels in the original image, and the pixels in the original image correspond one-to-one to the elements in the second one-dimensional vector, and the value of the element is the pixel value of the corresponding pixel; wherein the pixel value can be represented by a gray value.

This embodiment converts a two-dimensional reference image and an original image into corresponding one-dimensional vectors, so that adjacent pixels in the two-dimensional space of the image become adjacent pixels arranged in sequence in the one-dimensional space in the one-dimensional vector, and converts the two-dimensional image registration problem, that is, the problem of obtaining the spatial transformation matrix, into a vector similarity matching problem in the one-dimensional space, so that the image registration problem becomes a matching problem, thereby avoiding obtaining the homography matrix based on simple descriptor screening and pairing. In addition, it should be noted that the image is usually divided into a key area including the object content and a background area not including the object content. In the one-dimensional vector, the adjacent pixels in the background area and the adjacent pixels in the key area are arranged separately. Therefore, the background noise caused by factors such as uneven lighting, angle deformity and back penetration can be ignored in the one-dimensional vector.

Step S106: extracting a first eigenvector from the first one-dimensional vector, and extracting a second eigenvector from the second one-dimensional vector.

In some implementations, the method of extracting the first feature vector from the first one-dimensional vector may refer to the following: sliding the window of the first one-dimensional vector based on a window of a preset size, and extracting the fourth pixel point whose pixel value changes dramatically in each window according to the pixel value of the pixel point in the first one-dimensional vector, and obtaining the first feature vector according to the sequence relationship of the fourth pixel point in the first one-dimensional vector; the first feature vector is used to characterize the sequence relationship between the fourth pixel points. Similarly, the method of extracting the second feature vector from the second one-dimensional vector includes: sliding the window of the second one-dimensional vector based on a window of a preset size, and extracting the third pixel point whose pixel value changes dramatically in each window according to the pixel value of the pixel point in the second one-dimensional vector; and determining the second feature vector based on the third pixel point.

Step S108: performing feature point matching on the first feature vector and the second feature vector to obtain a feature point pair.

In this embodiment, the feature points between the first feature vector and the second feature vector can be matched according to the feature point detection algorithm to obtain the feature points representing the same pixel point on the two feature vectors, thereby forming a feature point pair, or in other words, the feature point pair can represent a pixel point on the original image and a pixel point on the reference image. Among them, the feature point detection algorithm is such as SIFT (Scale-invariant feature transform) feature detection algorithm, SURF (Speeded Up Robust Features) algorithm, orb (Oriented FAST and Rotated BRIEF) algorithm, etc.

Step S110 , performing affine transformation processing on the reference image according to the feature point pairs to obtain a target image registered with the original image.

After obtaining the feature point pairs, all pixels in the reference image may be affine transformed based on the matching feature point pairs in the first feature vector and the second feature vector to obtain a target image aligned with the preset image. For example, based on the feature point pairs, the homography matrix of the reference image relative to the original image is calculated, and the reference image is affine transformed based on the homography matrix to obtain a target image aligned with the original image.

Among them, the homography matrix is used to describe the position mapping relationship of the object between the world coordinate system and the pixel coordinate system; in this embodiment, the homography matrix is used to represent the position mapping relationship of the object between the reference image and the original image, and the reference image and the original image are aligning to obtain the matching feature point pairs of the two images, and then the homography matrix is calculated based on the position of the feature point pairs on the image.

The image registration method provided by the embodiment of the present disclosure first converts the acquired reference image and original image respectively according to the Hilbert curve to obtain the corresponding first one-dimensional vector and second one-dimensional vector; then extracts the first eigenvector from the first one-dimensional vector, and extracts the second eigenvector from the second one-dimensional vector; finally, the first eigenvector and the second eigenvector are matched with feature points, and the reference image is affine transformed according to the obtained feature point pairs to obtain the target image registered with the original image. In the above technical scheme, the one-dimensional vector obtained by using the Hilbert curve ensures that the adjacent pixel points on the two-dimensional image are also adjacent in the one-dimensional space, which ensures that the feature points can represent local information; at the same time, in the feature vector extracted from the one-dimensional vector, each feature point is more representative and less in number. Compared with the two-dimensional image, the feature vector removes noise and redundancy, and only retains a small amount of key information, which effectively increases the speed and accuracy of image registration.

For ease of understanding, the image registration method provided by the embodiment of the present disclosure is described in detail below.

With respect to the above step S106, this embodiment takes the first one-dimensional vector as an example to provide a method for extracting a first feature vector from the first one-dimensional vector, including the following steps:

Step 1: input a first one-dimensional vector into a preset encoder, and compress the first one-dimensional vector through the encoder to obtain a low-dimensional first compressed vector corresponding to the first one-dimensional vector.

The encoder is obtained after training a preset VAE (Variational Autoencoder) model, in which the convolution kernel and deconvolution kernel of the VAE model are both one-dimensional.

The first one-dimensional vector is usually of high dimension, which will result in a very large amount of calculation. Based on this, the present embodiment can use an encoder to encode and compress the high-dimensional first one-dimensional vector, output a first compressed vector of lower dimension, and reduce the vector length. In a specific implementation, the compression ratio of the first compressed vector relative to the first one-dimensional vector can be set according to actual conditions. For example, if the compression ratio of the encoder is set to 1/4, the vector length of the first compressed vector output by the encoder is 1/4 of the first one-dimensional vector. The present embodiment uses an encoder to perform vector compression, which can reduce the amount of vector calculation, thereby speeding up the calculation speed and reducing the calculation deviation, and achieving faster and more accurate image registration.

Step 2: extracting pixels whose pixel values change dramatically from the first compression vector according to the pixel values of the pixels in the first compression vector, and determining the extracted pixels as target feature points.

In a specific embodiment, it can be achieved through the following steps 2.1 to 2.4:

Step 2.1, performing a differential operation on the first compressed vector to obtain a differential vector; the differential vector is used to represent the change in pixel values between pixel points. Specifically, performing a differential operation on the first compressed vector is to obtain a gradient vector of the first compressed vector to obtain a differential vector.

Step 2.2, sliding a preset first window on the first compression vector, and extracting the first pixel point with the largest pixel value change in each first window according to the differential vector.

In a specific embodiment, the first window is, for example, a window including 32 elements; in this case, a first window with a length of 32 is used to slide without overlapping on the first compression vector, and the first pixel with the largest change in pixel value in each first window is found based on the change in pixel value represented by the differential vector.

Step 2.3, sliding a preset second window on the local vector, and extracting the second pixel point with the largest difference in pixel value with the first pixel point in each second window according to the differential vector; wherein the local vector is a vector of a preset size centered on the first pixel point in the first compressed vector.

In the first compressed vector, a local vector of a preset size is determined with the position of the first pixel as the center, for example, 32 elements to the left and 32 elements to the right with the position of the first pixel as the center, thereby determining a local vector with a preset length of 64. The second window is, for example, a window including 4 elements; based on this, a second window with a length of 4 is used to slide the window on the local vector, and according to the change of the pixel value represented by the differential vector, the second pixel point with the largest difference in pixel value with the first pixel point in each second window is found. Based on this, 16 second pixel points corresponding to the first pixel point can be found on the local vector.

Step 2.4: taking the extracted first pixel point and the second pixel point as pixel points with drastic pixel value changes.

It is easy to understand that there are multiple first pixel points and second pixel points obtained on the first compression vector, and in this embodiment, all the first pixel points and second pixel points are determined as target feature points.

Step 3: Determine a first feature vector based on the target feature point. Specifically, the target feature point can be used as an element of the first feature vector, and the first feature vector is obtained according to the sequence relationship of the target feature point in the first compression vector; the first feature vector is used to characterize the sequence relationship between the target feature points.

It should be noted that the method of extracting the second eigenvector from the second one-dimensional vector is the same as the method of extracting the first eigenvector from the first one-dimensional vector described in the above embodiment, and will not be described in detail here.

After extracting the first feature vector from the first one-dimensional vector according to the above embodiment and extracting the second feature vector from the second one-dimensional vector, feature point matching is performed on the first feature vector and the second feature vector according to the following embodiment, including:

First, the matching degree of the feature points between the first eigenvector and the second eigenvector is determined according to the preset matching degree algorithm, and the feature points with matching degrees higher than the preset matching degree threshold are determined as candidate feature point pairs; then, the candidate feature point pairs are optimized according to the RANSAC (Random Sample Consensus) algorithm to obtain the final feature point pairs.

Before performing affine transformation processing on the reference image according to the feature points, the present embodiment may also pre-record a first correspondence between the position of the same pixel point on the reference image and the position on the first one-dimensional vector; and record a second correspondence between the position of the same pixel point on the original image and the position on the second one-dimensional vector.

In a specific embodiment, in the process of converting the reference image and the original image respectively according to the Hilbert curve, the pixel points on the two-dimensional image and the elements on the converted one-dimensional vector can be converted into a dictionary in a one-to-one correspondence, and the first correspondence between the pixel points on the reference image and the elements on the first one-dimensional vector is recorded through the first dictionary, and the second correspondence between the pixel points on the original image and the elements on the second one-dimensional vector is recorded through the second dictionary.

In practical applications, the first correspondence relationship can be used to determine the position coordinates of the feature points on the first feature vector mapped on the reference image; and the second correspondence relationship can be used to determine the position coordinates of the feature points on the second feature vector mapped on the original image.

According to the above embodiment, a specific implementation method of performing affine transformation processing on a reference image according to feature points is described, including:

According to the first corresponding relationship and the second corresponding relationship, the position coordinates of each feature point in the feature point pair mapped on the corresponding image are calculated. Among them, the feature points in the feature point pair include: a first feature point belonging to the first feature vector and a second feature point belonging to the second feature vector. In the specific calculation process of the position coordinates, the position coordinates of the first feature point in the feature point pair mapped on the reference image can be determined according to the first corresponding relationship and the compression ratio between the first compression vector and the first one-dimensional vector. Of course, the same method can be used to determine the position coordinates of the second feature point in the feature point pair mapped on the original image.

Based on the position coordinates of each feature point in the feature point pair, the homography matrix of the reference image relative to the original image is calculated; based on the homography matrix, the reference image is affine transformed to obtain a target image that is aligned with the original image.

For example, any two or more candidate feature point pairs are selected from the feature point pairs; a candidate homography matrix of the reference image relative to the original image is calculated based on the candidate feature point pairs, and the feature points in the reference image are affine transformed based on the candidate homography matrix; based on the feature points in the original image, it is determined whether the accuracy of the feature points obtained by the affine transformation is greater than a preset threshold; if not, the process returns to select any two or more candidate feature point pairs from the feature point pairs; if so, the candidate homography matrix is used as the final homography matrix; then, the reference image is affine transformed based on the final homography matrix to obtain a target image aligned with the original image.

In order to enable the encoder in the above embodiment to be directly used for vector compression, the VAE model containing the encoder needs to be trained in advance; the purpose of training the VAE model is to finally determine the parameters that can meet the requirements. Using the trained parameters, the encoder can achieve the expected vector compression. This embodiment provides a training method for the VAE model, as shown below:

(1) Obtain a sample image; wherein there are multiple sample images, and for each sample image, the sample image is converted into a first sample vector according to the Hilbert curve.

(2) Input the first sample vector into the encoder of the VAE model to be trained to output the first sample compressed vector.

(3) Inputting the first sample compression vector into the decoder of the VAE model to be trained, and decoding the first sample compression vector through the decoder to generate a restored image corresponding to the sample image.

(4) The loss function value between the sample image and the restored image is calculated based on the L1 loss function, and the parameters of the VAE model to be trained are adjusted according to the loss function value. The training is terminated when the loss function value converges to the preset value, and a trained VAE model is obtained.

(5) Keep the encoder in the trained VAE model.

Through the above steps, an encoder that can be directly used for vector compression is obtained.

In summary, the image registration method provided by the embodiment of the present disclosure uses the one-dimensional vector obtained by the Hilbert curve to ensure that adjacent pixel points on the two-dimensional image are also adjacent in the one-dimensional space, which also ensures that the feature points can represent local information; at the same time, in the feature vector extracted from the one-dimensional vector, each feature point is more representative and fewer in number. At the same time, after the vector is compressed, the vector length is reduced while retaining key information. Therefore, compared with the two-dimensional image, the feature vector removes noise and redundancy and only retains a small amount of key information, which has a better effect on image registration and effectively increases the speed and accuracy of image registration.

On the basis of improving the image registration effect through the above-mentioned method, further, beneficial effects can be achieved in practical applications. For example, in the face of complex text images, the accuracy of photo identification can be significantly improved.

Based on the image registration method provided in the above embodiment, this embodiment provides an image registration device. Referring to the structural schematic diagram of an image registration device shown in FIG3 , the device includes:

An image acquisition module 302 is used to acquire a reference image and an original image to be registered;

An image conversion module 304 is used to convert the reference image and the original image according to the Hilbert curve to obtain a first one-dimensional vector corresponding to the reference image and a second one-dimensional vector corresponding to the original image; wherein the first one-dimensional vector is used to represent the sequence relationship between the pixel points in the reference image, and the second one-dimensional vector is used to represent the sequence relationship between the pixel points in the original image;

A feature extraction module 306, configured to extract a first feature vector from the first one-dimensional vector and a second feature vector from the second one-dimensional vector;

A matching module 308, configured to perform feature point matching on the first feature vector and the second feature vector to obtain a feature point pair;

The registration module 310 is used to perform affine transformation processing on the reference image according to the feature point pairs to obtain a target image registered with the original image.

In one embodiment, the feature extraction module 306 includes:

a vector compression unit, configured to input the first one-dimensional vector into a preset encoder, and compress the first one-dimensional vector through the encoder to obtain a low-dimensional first compressed vector corresponding to the first one-dimensional vector;

A first feature point extraction unit, configured to extract pixel points whose pixel values change dramatically from the first compression vector according to the pixel values of the pixel points in the first compression vector, and determine the extracted pixel points as target feature points;

A vector determination unit is used to determine a first feature vector based on the target feature point.

In one embodiment, the feature point extraction unit is specifically used for:

Performing a differential operation on the first compressed vector to obtain a differential vector; the differential vector is used to represent a change in pixel values between pixel points;

Performing sliding window operation on the first compression vector with a preset first window, and extracting first pixel points with the largest pixel value change in each of the first windows according to the differential vector;

Sliding a preset second window on the local vector, and extracting the second pixel point having the largest difference in pixel value between the second window and the first pixel point according to the differential vector; wherein the local vector is a vector of a preset size centered on the first pixel point in the first compressed vector;

The extracted first pixel point and the second pixel point are used as pixel points whose pixel values change dramatically.

In one embodiment, the feature extraction module 306 further includes a second feature point extraction unit, which is used to:

The second one-dimensional vector is slid based on a window of a preset size, and third pixels with drastic pixel value changes in each window are extracted according to the pixel values of the pixels in the second one-dimensional vector; and the second eigenvector is determined based on the third pixel.

In one embodiment, the matching module 308 is specifically configured to:

Determining the matching degree of feature points between the first feature vector and the second feature vector according to a preset matching degree algorithm;

Determine feature points whose matching degree is higher than a preset matching degree threshold as candidate feature point pairs;

The candidate feature point pairs are optimized according to the RANSAC algorithm to obtain the final feature point pairs.

In one embodiment, the image registration apparatus further includes a relationship recording module, which is used to:

Recording a first corresponding relationship between a position of a same pixel point on the reference image and a position on the first one-dimensional vector;

A second corresponding relationship between the position of the same pixel point on the original image and the position on the second one-dimensional vector is recorded.

In one embodiment, the registration module 310 is specifically configured to:

Calculating the position coordinates of each feature point in the feature point pair mapped on the corresponding image according to the first corresponding relationship and the second corresponding relationship;

Calculating a homography matrix of the reference image relative to the original image based on the position coordinates of each feature point in the feature point pair;

Affine transformation is performed on the reference image based on the homography matrix to obtain a target image registered with the original image.

The implementation principle and technical effects of the device provided in this embodiment are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

The exemplary embodiment of the present disclosure also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to cause the electronic device to perform the method according to the embodiment of the present disclosure when executed by the at least one processor.

The exemplary embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, the computer is used to enable the computer to perform the method according to the embodiments of the present disclosure.

With reference to Figure 4, a block diagram of an electronic device 400 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present disclosure described and/or required herein.

As shown in FIG4 , the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to the bus 404.

A plurality of components in the electronic device 400 are connected to the I/O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 may be any type of device capable of inputting information to the electronic device 400, and the input unit 406 may receive input digital or character information, and generate key signal inputs related to user settings and/or function control of the electronic device. The output unit 407 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 404 may include, but is not limited to, a disk, an optical disk. The communication unit 409 allows the electronic device 400 to exchange information/data with other devices via a computer network such as the Internet and/or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and/or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and/or the like.

The computing unit 401 may be a variety of general and/or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above. For example, in some embodiments, the image registration method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and/or installed on the electronic device 400 via the ROM 402 and/or the communication unit 409. In some embodiments, the computing unit 401 may be configured to perform the image registration method in any other appropriate manner (e.g., by means of firmware).

The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions/operations specified in the flow chart and/or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and/or data to a programmable processor.

To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims (12)

1. An image registration method, comprising: Acquire a reference image and an original image to be registered; The reference image and the original image are respectively converted according to the Hilbert curve to obtain a first one-dimensional vector corresponding to the reference image and a second one-dimensional vector corresponding to the original image; wherein the first one-dimensional vector is used to represent the sequence relationship between pixel points in the reference image, and the second one-dimensional vector is used to represent the sequence relationship between pixel points in the original image; Extracting a first feature vector from the first one-dimensional vector, and extracting a second feature vector from the second one-dimensional vector; wherein, extracting the first feature vector from the first one-dimensional vector comprises: sliding a window on the first one-dimensional vector based on a window of a preset size, and extracting a fourth pixel point whose pixel value changes dramatically in each window according to the pixel values of the pixel points in the first one-dimensional vector, and obtaining the first feature vector according to the sequence relationship of the fourth pixel point in the first one-dimensional vector; the fourth pixel point whose pixel value changes dramatically comprises: a first pixel point whose pixel value changes the most between the pixels in the preset first window, and a second pixel point whose pixel value difference with the first pixel point in the preset second window is the largest; extracting the second feature vector from the second one-dimensional vector comprises: sliding a window on the second one-dimensional vector based on a window of a preset size, and extracting a third pixel point whose pixel value changes dramatically in each window according to the pixel values of the pixels in the second one-dimensional vector; determining the second feature vector based on the third pixel point; the extraction method of the third pixel point whose pixel value changes dramatically is the same as the extraction method of the fourth pixel point; According to the matching degree being higher than a preset matching degree threshold, performing feature point matching on the first feature vector and the second feature vector to obtain a feature point pair; According to the feature point pairs, an affine transformation process is performed on the reference image to obtain a target image registered with the original image. 2. The method according to claim 1, characterized in that extracting the first feature vector from the first one-dimensional vector comprises: Inputting the first one-dimensional vector to a preset encoder, compressing the first one-dimensional vector through the encoder, and obtaining a low-dimensional first compressed vector corresponding to the first one-dimensional vector; According to the pixel values of the pixels in the first compression vector, extracting the pixel points whose pixel values change dramatically from the first compression vector, and determining the extracted pixel points as target feature points; A first feature vector is determined based on the target feature point. 3. The method according to claim 2, characterized in that extracting pixels whose pixel values change dramatically from the first compressed vector according to the pixel values of the pixels in the first compressed vector comprises: Performing a differential operation on the first compressed vector to obtain a differential vector; the differential vector is used to represent a change in pixel values between pixel points; Performing sliding window operation on the first compression vector with a preset first window, and extracting first pixel points with the largest pixel value change in each of the first windows according to the differential vector; Sliding a preset second window on the local vector, and extracting the second pixel point having the largest difference in pixel value between the second window and the first pixel point according to the differential vector; wherein the local vector is a vector of a preset size centered on the first pixel point in the first compressed vector; The extracted first pixel point and the second pixel point are used as pixel points whose pixel values change dramatically. 4. The method according to claim 1, characterized in that the step of performing feature point matching on the first feature vector and the second feature vector to obtain a feature point pair according to the matching degree being higher than a preset matching degree threshold comprises: Determining the matching degree of feature points between the first feature vector and the second feature vector according to a preset matching degree algorithm; Determine feature points whose matching degree is higher than a preset matching degree threshold as candidate feature point pairs; The candidate feature point pairs are optimized according to the random sampling RANSAC algorithm to obtain the final feature point pairs. 5. The method according to claim 1, characterized in that the method further comprises: Recording a first corresponding relationship between a position of a same pixel point on the reference image and a position on the first one-dimensional vector; A second corresponding relationship between the position of the same pixel point on the original image and the position on the second one-dimensional vector is recorded. 6. The method according to claim 5, characterized in that the step of performing affine transformation processing on the reference image according to the feature point pairs to obtain a target image registered with the original image comprises: Calculating the position coordinates of each feature point in the feature point pair mapped on the corresponding image according to the first corresponding relationship and the second corresponding relationship; Calculating a homography matrix of the reference image relative to the original image based on the position coordinates of each feature point in the feature point pair; Affine transformation is performed on the reference image based on the homography matrix to obtain a target image registered with the original image. 7. The method according to claim 2 is characterized in that the encoder is obtained after training a preset VAE model, wherein the convolution kernel and deconvolution kernel of the VAE model are both one-dimensional. 8. The method according to claim 1, characterized in that the reference image and the original image are grayscale images respectively. 9. The method according to claim 1 is characterized in that the original image comprises: a text image obtained by photographing a target test question, and the reference image comprises: an image of the target test question in a preset test question library. 10. An image registration device, comprising: An image acquisition module, used to acquire a reference image and an original image to be registered; An image conversion module, used to convert the reference image and the original image respectively according to the Hilbert curve to obtain a first one-dimensional vector corresponding to the reference image and a second one-dimensional vector corresponding to the original image; wherein the first one-dimensional vector is used to represent the sequence relationship between the pixel points in the reference image, and the second one-dimensional vector is used to represent the sequence relationship between the pixel points in the original image; A feature extraction module, used to extract a first feature vector from the first one-dimensional vector, and to extract a second feature vector from the second one-dimensional vector; wherein the feature extraction module is further used to: slide the first one-dimensional vector based on a window of a preset size, and extract fourth pixels whose pixel values vary dramatically in each window according to the pixel values of the pixels in the first one-dimensional vector, and obtain a first feature vector according to the sequence relationship of the fourth pixels in the first one-dimensional vector; the fourth pixels whose pixel values vary dramatically include: a first pixel whose pixel value varies the most between pixels in the preset first window, and a second pixel whose pixel value difference with the first pixel in the preset second window is the largest; and, slide the second one-dimensional vector based on a window of a preset size, and extract third pixels whose pixel values vary dramatically in each window according to the pixel values of the pixels in the second one-dimensional vector; determine a second feature vector based on the third pixel; the extraction method of the third pixel whose pixel value varies dramatically is the same as the extraction method of the fourth pixel; A matching module, configured to match feature points of the first feature vector and the second feature vector according to a matching degree being higher than a preset matching degree threshold, to obtain a feature point pair; The registration module is used to perform affine transformation processing on the reference image according to the feature point pair to obtain a target image registered with the original image. 11. An electronic device, characterized in that the electronic device comprises: Processor; and Memory for storing programs, The program includes instructions, which, when executed by the processor, enable the processor to perform the image registration method according to any one of claims 1 to 9. 12. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the image registration method according to any one of claims 1 to 9.