WO2024130705A1 - 一种图像配准方法、装置、设备和存储介质 - Google Patents

一种图像配准方法、装置、设备和存储介质 Download PDF

Info

Publication number
WO2024130705A1
WO2024130705A1 PCT/CN2022/141486 CN2022141486W WO2024130705A1 WO 2024130705 A1 WO2024130705 A1 WO 2024130705A1 CN 2022141486 W CN2022141486 W CN 2022141486W WO 2024130705 A1 WO2024130705 A1 WO 2024130705A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
sequence
images
registered
registration
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/141486
Other languages
English (en)
French (fr)
Inventor
张博涵
刘超
李美
张勇
黎宇翔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BGI Shenzhen Co Ltd
Original Assignee
BGI Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BGI Shenzhen Co Ltd filed Critical BGI Shenzhen Co Ltd
Priority to PCT/CN2022/141486 priority Critical patent/WO2024130705A1/zh
Priority to CN202280102252.8A priority patent/CN120303692A/zh
Publication of WO2024130705A1 publication Critical patent/WO2024130705A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present application relates to the technical field of image registration, and in particular to an image registration method, device, equipment and storage medium.
  • Image registration is an important technology in the field of image processing research. Its purpose is to align multiple images of the same object taken under different conditions or a sequence of images taken continuously.
  • the properties of registration algorithms include rigidity, affine, elasticity and other directions.
  • An existing image registration method is based on feature point matching. This method mainly finds several pairs of matching feature point pairs in two images that need to be aligned. Each feature point pair includes two feature points, which belong to the two images respectively. Then, the feature point pairs are used to establish a feature description vector. According to the feature description vector, one image is transformed to align it with the other image.
  • the problem with the above method is that for images taken in some special scenarios, such as multiple images of the same sample taken continuously in the medical field, it is difficult to find a sufficient number of feature point pairs, so the above method cannot align this type of images.
  • the present application provides an image registration method, apparatus, device and storage medium for rigid registration of images in the medical field.
  • the first aspect of the present application provides an image registration method, comprising:
  • the image registration model is used to extract the transformation matrix corresponding to each image in the image sequence to be registered; wherein the transformation matrix includes three parameters: vertical offset, horizontal offset and rotation angle;
  • the image is processed using a transformation matrix corresponding to the image to obtain a registration result image corresponding to the image.
  • obtaining the image sequence to be registered includes:
  • the sizes of the multiple images are cropped to a preset target size to obtain a sequence of images to be registered.
  • the step of training an image registration model using the image sequence to be registered includes:
  • the image registration model is obtained by training using the training sequence and the result sequence.
  • determining a training sequence and a result sequence according to the image sequence to be registered includes:
  • Each image except the first image in the sequence of images to be registered, and each image except the first image that is horizontally flipped, are used to form a result sequence.
  • determining a training sequence and a result sequence according to the image sequence to be registered includes:
  • the result sequence is composed of the other one of the two images and the other images, and the image after the other one of the two images and the other images are horizontally flipped.
  • the order of the multiple images in the training sequence is a randomly determined order, or an order set according to a user instruction, or an order in the sequence of images to be registered;
  • the order of the multiple images in the result sequence is a randomly determined order, or an order set according to a user instruction, or an order in the sequence of images to be registered.
  • processing the image using a transformation matrix corresponding to the image to obtain a registration result image corresponding to the image includes:
  • the transformed image or the transformed flipped image is determined as the registration result of the image; wherein the subsequent image refers to the image that is located after the image in the image sequence to be registered.
  • obtaining the image sequence to be registered includes:
  • the sizes of the multiple images are cropped to a preset target size to obtain a sequence of images to be registered.
  • a second aspect of the present application provides an image registration method, comprising:
  • the image is processed using the transformation matrix corresponding to the image to obtain a registration result of the image.
  • the process of training the registration model includes:
  • the image registration model is obtained by training using the training sequence and the result sequence.
  • the process of training the registration model includes:
  • the image registration model is obtained by training using the training sequence and the result sequence.
  • the using a transformation matrix corresponding to the image to process the image to obtain a registration result of the image includes:
  • the transformed image or the transformed flipped image is determined as the registration result of the image; wherein the subsequent image refers to the image that is located after the image in the image sequence to be registered.
  • a third aspect of the present application provides an image registration device, comprising:
  • An acquisition unit used for acquiring a sequence of images to be registered
  • a training unit used for training an image registration model using the image sequence to be registered
  • An extraction unit used to extract the transformation matrix corresponding to each image in the image sequence to be registered by using the image registration model; wherein the transformation matrix includes three parameters: vertical offset, horizontal offset and rotation angle;
  • the processing unit is used to process each image in the sequence of images to be registered by using the transformation matrix corresponding to the image to obtain a registration result image corresponding to the image.
  • the obtaining unit obtains the image sequence to be registered, it is specifically used to:
  • the sizes of the multiple images are cropped to a preset target size to obtain a sequence of images to be registered.
  • the training unit obtains the image registration model by training the image sequence to be registered, it is specifically used to:
  • the image registration model is obtained by training using the training sequence and the result sequence.
  • the training unit determines the training sequence and the result sequence according to the image sequence to be registered, it is specifically used to:
  • Each image except the first image in the sequence of images to be registered, and each image except the first image that is horizontally flipped, are used to form a result sequence.
  • the training unit determines the training sequence and the result sequence according to the image sequence to be registered, it is specifically used to:
  • the result sequence is composed of the other one of the two images and the other images, and the image after the other one of the two images and the other images are horizontally flipped.
  • the order of the multiple images in the training sequence is a randomly determined order, or an order set according to a user instruction, or an order in the sequence of images to be registered;
  • the order of the multiple images in the result sequence is a randomly determined order, or an order set according to a user instruction, or an order in the sequence of images to be registered.
  • the processing unit processes each image in the sequence of images to be registered by using a transformation matrix corresponding to the image to obtain a registration result image corresponding to the image, specifically for:
  • the transformed image or the transformed flipped image is determined as the registration result of the image; wherein the subsequent image refers to the image that is located after the image in the image sequence to be registered.
  • the obtaining unit obtains the image sequence to be registered, it is specifically used to:
  • the sizes of the multiple images are cropped to a preset target size to obtain a sequence of images to be registered.
  • a fourth aspect of the present application provides an electronic device, including a memory and a processor
  • the memory is used to store computer programs
  • the processor is used to execute the computer program, and when the computer program is executed, it is specifically used to implement the image registration method provided in any one of the first aspect or the second aspect of the present application.
  • the fifth aspect of the present application provides a computer storage medium for storing a computer program.
  • the computer program When executed, it is specifically used to implement the image registration method provided in any one of the first aspect or the second aspect of the present application.
  • the present application provides an image registration method, apparatus, device and storage medium, the method comprising: obtaining a sequence of images to be registered; using the sequence of images to be registered to train an image registration model; using the image registration model to extract a transformation matrix corresponding to each image in the sequence of images to be registered; the transformation matrix includes three parameters: vertical offset, horizontal offset and rotation angle; for each image in the sequence of images to be registered, using the transformation matrix corresponding to the image to process the image, and obtaining a registration result image corresponding to the image.
  • This solution does not need to rely on feature points in the image during the registration process, so it can also effectively perform rigid registration for medical images (or other similar images) where it is difficult to extract feature points.
  • FIG1 is a flow chart of a method for training an image registration model provided in an embodiment of the present application
  • FIG2 is an example of image flipping provided by an embodiment of the present application.
  • FIG3 is a flow chart of an image registration method provided in an embodiment of the present application.
  • FIG4 is an exemplary diagram of an image registration process provided by an embodiment of the present application.
  • FIG5 is a schematic diagram of an image registration effect provided by an embodiment of the present application.
  • FIG6 is a schematic diagram of a registration result of an image sequence provided in an embodiment of the present application.
  • FIG7 is a schematic diagram of the structure of an image registration device provided in an embodiment of the present application.
  • FIG8 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
  • the present application provides an imaging method and an imaging system for improving the comprehensiveness of image display.
  • the image registration method provided in the embodiment of the present application can be used to register medical images, and can also be used to register other images with similar features to medical images (such as grayscale images with a mainly black background).
  • the image registration method provided in the embodiment of the present application needs to first train an image registration model using the image sequence to be registered, and then use the image registration model to register each image in the image sequence to be registered.
  • FIG. 1 is a flowchart of a method for training a sequence of images to be registered provided in an embodiment of the present application.
  • the method may include the following steps.
  • the image sequence to be registered can be obtained by first continuously photographing the same object to obtain multiple images, then cropping each image, cropping the size of each image to a preset target size, and the image sequence composed of the cropped multiple images is the image sequence to be registered.
  • the sizes of the multiple images are cropped to a preset target size to obtain a sequence of images to be registered.
  • the image sequence to be registered may be obtained by photographing multiple slices of the same object separately to obtain multiple images, wherein each image shows a slice of the object, and then cropping the multiple images to a preset target size, and the cropped images constitute the image sequence to be registered.
  • the same object here may be a biological tissue sample used for experiments, such as a mouse heart.
  • the size of the multiple images obtained by shooting is the preset target size, cropping may not be performed, and the image sequence formed by the multiple images obtained by shooting may be directly determined as the image sequence to be registered.
  • S102 Decompose the image sequence to be registered into a first image sequence and a second image sequence.
  • step S102 a variety of methods may be used to decompose the first image sequence and the second image sequence. As examples, two optional decomposition methods are described below.
  • the first image sequence includes every image in the image sequence to be registered except the last image
  • the second image sequence includes every image in the image sequence to be registered except the first image
  • the image sequence to be registered includes n images, which are sequentially recorded as image 1 to image n, then the first image sequence includes image 1 to image n-1, and the second image sequence includes image 2 to image n.
  • the second decomposition method is:
  • Two images are randomly selected from the image sequence to be registered, one of the two randomly selected images is placed in the first image sequence, and the other is placed in the second image sequence, and then all remaining unselected images are placed in the first image sequence and the second image sequence at the same time.
  • the image sequence to be registered includes images 1 to 6, and image 1 and image 3 are randomly selected, and image 1 is placed in the first image sequence, image 3 is placed in the second image sequence, and the remaining unselected image 2 and images 4 to 6 are placed in the first image sequence and the second image sequence at the same time.
  • the first image sequence includes image 1, image 2, image 4 to image 6; the second image sequence includes image 3, image 2, image 4 to image 6.
  • the order of images in the first image sequence and the second image sequence may be consistent with the order in the original image sequence to be registered, or may be inconsistent with the order in the original image sequence to be registered and arranged randomly, or may be arranged in a custom order set according to user instructions.
  • step S104 is executed; if there is a flipping in the sequence of images to be registered, step S105 is executed.
  • FIG. 2 the direction of the arrow of image a in FIG. 2 is opposite to the direction of the arrow of image b, so it can be considered that image a is flipped relative to image b.
  • step S103 if at least one image in the image sequence to be registered is flipped, it is determined that the image sequence to be registered is flipped; if no image in the image sequence to be registered is flipped, it is determined that the image sequence to be registered is not flipped.
  • each image in the images to be registered is flipped can be calibrated in advance by manual calibration, or by using a specific recognition algorithm to identify it, and then the calibration or recognition result is marked on each image.
  • S104 determine the first image sequence as a training sequence, and determine the second image sequence as a result sequence.
  • the training sequence can be expressed as: (image 1, image 2, ... image n-1), and the result sequence can be expressed as: (image 2, image 3, ... image n).
  • step S105 L may be added after the image number to indicate the flipped image.
  • the image obtained by horizontally flipping image 2 may be recorded as image 2L.
  • the flipped image sequence may be represented as (image 2L, image 3L . . . image nL).
  • step S106 the copy of the first image sequence may be spliced at the end of the first image sequence, and the flipped image sequence may be spliced at the end of the second image sequence.
  • the training sequence can be represented as (image 1, image 2, ... image n-1, image 1, image 2, ... image n-1), and the result sequence can be represented as (image 2, image 3, ... image n, image 2L, image 3L ... image nL).
  • step S102 to step S106 is equivalent to determining a training sequence and a result sequence according to the image sequence to be registered.
  • determining the training sequence and the result sequence according to the image sequence to be registered may include:
  • Each image except the last image in the image sequence to be registered is used to form a training sequence, and each image except the first image in the image sequence to be registered is used to form a result sequence;
  • Each image except the last image in the image sequence to be registered is used to form a training sequence
  • Each image except the first image in the image sequence to be registered and each image except the first image is horizontally flipped to form a result sequence.
  • determining the training sequence and the result sequence according to the image sequence to be registered may include:
  • One of the two images and the other images are used to form a training sequence; wherein the other image refers to an unselected image in the image sequence to be registered;
  • the resulting sequence is composed of the other of the two images and the other images, and the other of the two images and the other images being horizontally flipped.
  • the image registration model to be trained may be a deep learning model pre-constructed based on any existing deep learning network structure. This embodiment does not limit the specific structure of the image registration model to be trained.
  • the image registration model to be trained can be a convolutional neural network (CNN) model including multiple convolutional layers (for example, three), or a recurrent neural network (RNN attention) model based on an attention mechanism.
  • CNN convolutional neural network
  • RNN attention recurrent neural network
  • step S107 may include:
  • A1 first use the image registration model to be trained to process each image in the training sequence to obtain the transformation matrix corresponding to each image in the training sequence.
  • the above process of obtaining the transformation matrix of the image is determined by the specific structure of the image registration model, and this embodiment does not limit this.
  • the process of obtaining the transformation matrix can be to use multiple convolutional layers in the CNN model to extract features of an input image one by one, and the feature output by the last convolutional layer is the transformation matrix of the input image.
  • the CNN model includes three convolutional layers.
  • the first convolutional layer first extracts features to obtain the first image features
  • the second convolutional layer extracts features from the first image features to obtain the second image features
  • the third convolutional layer extracts features from the second image features to obtain the transformation matrix corresponding to the image.
  • the transformation matrix may be a 2*3 matrix, that is, a matrix with 2 rows and 3 columns, and the value of each element in the matrix depends on the vertical offset y, the horizontal offset x and the rotation angle ⁇ .
  • the transformation matrix can be expressed as:
  • A2 for each image in the training sequence, the image is processed using the transformation matrix corresponding to the image to obtain the training result image corresponding to the image.
  • image 1 is processed with the transformation matrix corresponding to image 1 to obtain the corresponding training result image
  • image 2 is processed with the transformation matrix corresponding to image 2 to obtain the corresponding training result image 2, and so on.
  • the specific process of processing an image with a transformation matrix can be, for each pixel in the image, performing a matrix multiplication operation using the coordinates of the pixel and the transformation matrix corresponding to the image to obtain the transformed coordinates, and then moving the pixel to the transformed coordinates. After performing the above operation on all pixels in the image, these pixels moved to the transformed coordinates constitute the processed training result image.
  • x1 and y1 represent the original coordinates of the pixel
  • x2 and y2 represent the transformed coordinates
  • A3 calculate the deviation between the training result image corresponding to each image in the training sequence and each image in the result sequence to obtain the model loss value.
  • step A3 for each image in the training sequence, the deviation between the training result image corresponding to the image and the image at the same position as the image in the result sequence is calculated, and then the model loss value is determined based on the multiple deviations.
  • the images in the training sequence and the images in the result sequence are located at the same position, which means that the two images are arranged in the same order in the sequences to which they belong.
  • the training sequence is (image 1, image 2, ... image n-1)
  • the result sequence is (image 2, image 3, ... image n).
  • the first image in the training sequence is image 1
  • the first image in the result sequence is image 2. Therefore, it can be considered that image 1 in the training sequence and image 2 in the result sequence are located at the same position, that is, both are the first images in their respective sequences; similarly, image 2 in the training sequence and image 3 in the result sequence are both the second images in their respective sequences, and it can be considered that image 2 in the training sequence and image 3 in the result sequence are located at the same position.
  • the corresponding training result image 1 is obtained after processing in A2.
  • the deviation between the training result image 1 and the first image of the result sequence, that is, image 2 can be calculated.
  • step A3 the deviation between the training result image 1 and the image 2 of the result sequence, the deviation between the training result image 2 and the image 3 of the result sequence, the deviation between the training result image 3 and the image 4 of the result sequence, and so on are calculated in sequence.
  • the deviation between the two images can be represented by the mean square error (MSE), or other indicators, which are not limited in this embodiment.
  • MSE mean square error
  • the specific calculation method of the mean square error of the two images can refer to the relevant prior art literature.
  • each calculated deviation can be determined as the model loss value, the largest deviation among all the calculated deviations can be determined as the model loss value, or the sum of all the calculated deviations can be determined as the model loss value.
  • This embodiment does not limit the specific method of determining the model loss value based on the deviations of multiple images.
  • step S109 is executed; if the model loss value meets the convergence condition, step S110 is executed.
  • the convergence condition may be that the model loss value is less than a preset convergence threshold.
  • the convergence condition may be that each model loss value is less than the convergence threshold.
  • step S109 the process returns to step S107 and uses the registration model with updated parameters to process the images in the training sequence again, thereby obtaining a new model loss value.
  • a back propagation algorithm may be used to determine the update amount corresponding to each parameter in the registration model to be trained according to the model loss value, and then the value of each parameter in the model is updated according to the determined update amount.
  • step S108 If it is determined in step S108 that the model loss value meets the convergence condition, then the registration model at this time is the trained registration model.
  • step S101 all steps after step S101 are equivalent to:
  • the image registration model is trained using the image sequence to be registered.
  • a training sequence and a result sequence are determined according to the image sequence to be registered.
  • the training sequence includes every image in the image sequence to be registered except the last image, and the result sequence includes every image in the image sequence to be registered except the first image;
  • the training sequence includes each image except the last image in the image sequence to be registered and its copy
  • the result sequence includes each image except the first image in the image sequence to be registered and the image after each image except the first image is horizontally flipped.
  • the image registration model is obtained by training the training sequence and the result sequence.
  • FIG3 is a flow chart of the image registration method provided in an embodiment of the present application. The method may include the following steps.
  • step S301 is consistent with step S101 and will not be repeated here.
  • step S302 may refer to step A1 in the embodiment corresponding to FIG1 , and will not be described in detail.
  • step S103 Whether each image in the image sequence to be registered is flipped can be calibrated in advance by manual calibration, or recognized by a specific recognition algorithm, and then the calibration or recognition result is marked on each image. Therefore, in step S303, it can be determined whether each image is flipped based on the calibration result or recognition result of each image in the image sequence to be registered.
  • step S304 is executed to obtain the registration result corresponding to the image.
  • step S305 is performed to obtain a registration result corresponding to the image.
  • step S304 for a flipped image in the sequence of images to be registered, the image may be processed in the manner described in step A2 using the transformation matrix corresponding to the image obtained in S302 to obtain a transformed image.
  • the transformed image is compared with the next image of the flipped image to obtain a first deviation value; and the transformed image is compared with the image after the next image is flipped to obtain a second deviation value.
  • the first deviation value and the second deviation value can be the mean square error of the two compared images, or can also be other indicators that can measure the deviation of the two images, without limitation.
  • the image corresponding to the smaller deviation between the first deviation and the second deviation is selected as the registration result image of the flipped image.
  • image 1 in the image sequence to be registered is flipped.
  • image 2 is image 2.
  • Image 1 is processed with the transformation matrix corresponding to image 1 to obtain the transformed image, which is recorded as transformed image 1.
  • 1L is the flipped image 1.
  • Image 1L is processed with the transformation matrix corresponding to image 1L to obtain the transformed image, which is recorded as transformed image 1L.
  • the transformed image 1 and image 2 are compared to obtain a first deviation value; the transformed image 1L and image 2 are compared to obtain a second deviation value. If the first deviation value is smaller than the second deviation value, the transformed image 1 is determined as the output result corresponding to image 1, that is, as the registration result image.
  • the transformed image 1L is determined as the output result corresponding to the image 1, that is, as the registration result image.
  • step S304 The beneficial effect of step S304 is that, by comparing the image deviations, an image with a smaller deviation is selected as the registration result image of the flipped image, thereby improving the accuracy of the registration of the flipped image.
  • S305 Process the image that has not been flipped using the corresponding transformation matrix to obtain a registration result of the image.
  • step S305 the non-flipped image is processed by the image registration model to obtain a corresponding transformation matrix, and then the non-flipped image is processed using the transformation matrix according to the processing method in step A2 to obtain the highlighted registration result image.
  • step S302 image 5 is processed using the image registration model to obtain the transformation matrix corresponding to image 5, and then image 5 is processed using the transformation matrix corresponding to image 5 in the manner described in step A2 to obtain the registration result image corresponding to image 5.
  • steps S303 to S305 is equivalent to:
  • the image is processed using the transformation matrix corresponding to the image to obtain the registration result image corresponding to the image.
  • step S303 is equivalent to: for each image in the sequence of images to be registered, determining whether the image is flipped.
  • Step S304 is equivalent to:
  • the coordinates of each pixel in the image are adjusted according to the transformation matrix of the image to obtain the registration result image corresponding to the image.
  • Step S305 is equivalent to:
  • the coordinates of each pixel in the image are adjusted according to the image transformation matrix to obtain the transformed image corresponding to the image.
  • the image registration result image is determined based on the deviation between the transformed image, the previous image in the image sequence to be registered, and the image after the previous image is horizontally flipped.
  • FIG. 4 is an example diagram of an image registration process provided in this embodiment.
  • the input image is first input into the trained image registration model to obtain the transformation matrix corresponding to the input image, which is equivalent to step S302 of the embodiment shown in FIG3. Then, according to the transformation matrix corresponding to the input image, the input image is registered to obtain the output image, that is, the registration result image corresponding to the input image, which is equivalent to steps S303 to S305 of the embodiment shown in FIG3.
  • the input image may be each image in the sequence of images to be registered described in step S301 , for example, it may be image 1, image 2, etc. in the sequence of images to be registered.
  • Fig. 5 is a schematic diagram of an image registration effect provided by an embodiment of the present application. As shown in Fig. 5, after the k-th image in the image sequence to be registered is registered using the image registration model, a registration result image corresponding to the k-th image is obtained.
  • Figure 6 is a schematic diagram of a registration result obtained after registering the entire sequence of images to be registered provided in an embodiment of the present application, wherein Figure (1) is a sequence of images to be registered obtained after multiple consecutive shots of a mouse heart, and Figure (2) is a sequence of registration result images obtained after registering the sequence of images to be registered in Figure (1) according to the registration method provided in this embodiment.
  • the beneficial effect of the image registration method provided in the embodiment of the present application is that the image registration model trained by the image sequence to be registered is used to extract the transformation matrix of each image in the image sequence to be registered, and then for each image in the image sequence to be registered, the image is processed using the transformation matrix of the image to obtain a registration result image.
  • This solution does not need to rely on feature points in the image during the registration process, so it can effectively perform rigid registration on medical images (or other similar images) where feature points are difficult to extract.
  • the embodiment of the present application also provides an image registration device, please refer to Figure 7, the device may include the following units.
  • the obtaining unit 701 is used to obtain a sequence of images to be registered.
  • the training unit 702 is used to train the image registration model using the image sequence to be registered.
  • the extraction unit 703 is used to extract the transformation matrix corresponding to each image in the image sequence to be registered by using the image registration model.
  • the processing unit 704 is used to process each image in the sequence of images to be registered using the transformation matrix corresponding to the image to obtain a registration result image corresponding to the image.
  • the obtaining unit 701 obtains the image sequence to be registered, it is specifically used to:
  • the sizes of multiple images are cropped to a preset target size to obtain an image sequence to be registered.
  • the obtaining unit 701 obtains the image sequence to be registered, it is specifically used to:
  • the sizes of multiple images are cropped to a preset target size to obtain an image sequence to be registered.
  • the training unit 702 obtains the image registration model by training the image sequence to be registered, it is specifically used to:
  • the image registration model is obtained by training the training sequence and the result sequence.
  • the training unit 702 determines the training sequence and the result sequence according to the image sequence to be registered, it is specifically used to:
  • Each image except the last image in the image sequence to be registered is used to form a training sequence, and each image except the first image in the image sequence to be registered is used to form a result sequence;
  • Each image except the last image in the image sequence to be registered is used to form a training sequence
  • Each image except the first image in the image sequence to be registered and each image except the first image after being horizontally flipped are used to form a result sequence.
  • the training unit 702 determines the training sequence and the result sequence according to the image sequence to be registered, it is specifically used to:
  • One of the two images and the other images are used to form a training sequence; wherein the other image refers to an unselected image in the image sequence to be registered;
  • the resulting sequence is composed of the other of the two images and the other images, and the other of the two images and the other images being horizontally flipped.
  • the order of the multiple images in the training sequence is a randomly determined order, or an order set according to a user instruction, or an order in the sequence of images to be registered;
  • the order of the multiple images in the result sequence is a randomly determined order, or an order set according to a user instruction, or an order in the sequence of images to be registered.
  • the processing unit 704 processes each image in the sequence of images to be registered using a transformation matrix corresponding to the image to obtain a registration result image corresponding to the image, specifically for:
  • the coordinates of each pixel in the image are adjusted according to the transformation matrix of the image to obtain the registration result image corresponding to the image;
  • the coordinates of each pixel in the image are adjusted according to the transformation matrix of the image to obtain the transformed image corresponding to the image;
  • the image is horizontally flipped to obtain a flipped image, and the coordinates of each pixel in the flipped image are adjusted according to the transformation matrix of the image to obtain a transformed flipped image corresponding to the flipped image;
  • the transformed image or the transformed flipped image is determined as the image registration result; wherein the subsequent image refers to the image located after the image in the image sequence to be registered.
  • the present application provides an image registration device, which includes an acquisition unit 701 that obtains a sequence of images to be registered; a training unit 702 that uses the sequence of images to be registered to obtain an image registration model; an extraction unit 703 that uses the image registration model to extract a transformation matrix corresponding to each image in the sequence of images to be registered; and a processing unit 704 that processes each image in the sequence of images to be registered using the transformation matrix corresponding to the image to obtain a registration result image corresponding to the image.
  • This solution does not need to rely on feature points in the image during the registration process, so it can also effectively perform rigid registration on medical images (or other similar images) for which feature points are difficult to extract.
  • An embodiment of the present application further provides an electronic device, see FIG. 8 , which includes a memory 801 and a processor 802 .
  • the memory 801 is used to store computer programs.
  • the processor 802 is used to execute a computer program. When the computer program is executed, it is specifically used to implement the image registration method provided in any embodiment of the present application.
  • An embodiment of the present application also provides a computer storage medium for storing a computer program.
  • the computer program When executed, it is specifically used to implement the image registration method provided in any embodiment of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本申请提供一种图像配准方法、装置、设备和存储介质，方法包括，获得待配准图像序列；利用待配准图像序列训练得到图像配准模型；利用图像配准模型提取待配准图像序列中每一张图像对应的变换矩阵；变换矩阵包括竖直偏移量，水平偏移量和旋转角度三项参数；针对待配准图像序列中的每一张图像，利用图像对应的变换矩阵处理图像，获得图像对应的配准结果图像。本方案在配准过程中不需要依赖于图像中的特征点，因此对于难以提取特征点的医学图像(或其他类似图像)也能够有效地进行刚性配准。

Description

一种图像配准方法、装置、设备和存储介质 技术领域
本申请涉及图像配准技术领域，特别涉及一种图像配准方法、装置、设备和存储介质。
背景技术
图像配准是图像处理研究领域中的一个重要技术，其目的在于将不同条件下拍摄同一对象得到的多张图像或连续拍摄同一对象获得的图像序列进行对齐。配准算法性质有刚性、仿射、弹性等不同方向。
现有的一种图像配准方法是基于特征点匹配，该方法主要是在两张需要对齐的图像中找到若干对相互匹配的特征点对，每一特征点对均包括两个特征点，分别属于两张图像，然后利用特征点对建立特征描述向量，根据特征描述向量对一张图像进行变换，使其与另一张图像对齐。
上述方法存在的问题是，对于一些特殊场景下拍摄的图像，例如在医学领域对同一样本连续拍摄得到的多张图像，难以找到足够数量的特征点对，因此上述方法无法对这类型的图像进行配准。
发明内容
本申请提供一种图像配准方法、装置、设备和存储介质，用于对医学领域的图像进行刚性配准。
本申请第一方面提供一种图像配准方法，包括：
获得待配准图像序列；
利用所述待配准图像序列训练得到图像配准模型；
利用所述图像配准模型提取所述待配准图像序列中每一张图像对应的变换矩阵；其中，所述变换矩阵包括竖直偏移量，水平偏移量和旋转角度三项参数；
针对所述待配准图像序列中的每一张图像，利用所述图像对应的变换矩阵处理所述图像，获得所述图像对应的配准结果图像。
可选的，所述获得待配准图像序列，包括：
获得对同一物体连续拍摄得到的多张图像；
将所述多张图像的尺寸均裁剪为预设的目标尺寸，得到待配准图像序列。
可选的，所述利用所述待配准图像序列训练得到图像配准模型，包括：
根据所述待配准图像序列确定训练序列和结果序列；
利用所述训练序列和所述结果序列训练得到图像配准模型。
可选的，所述根据所述待配准图像序列确定训练序列和结果序列，包括：
用所述待配准图像序列中除末张图像以外的每一张图像组成训练序列，并用所述待配准图像序列中除首张图像以外的每一张图像组成结果序列；
或者，
用所述待配准图像序列中除末张图像以外的每一张图像组成训练序列；
用所述待配准图像序列中除首张图像以外的每一张图像，以及除首张图像以外的每一张图像被水平翻转后的图像，组成结果序列。
可选的，所述根据所述待配准图像序列确定训练序列和结果序列，包括：
随机选中所述待配准图像序列的两张图像；
用所述两张图像中的一张和其他图像组成训练序列；其中，所述其他图像指代所述待配准图像序列中未选中的图像；
用所述两张图像中的另一张和其他图像组成结果序列；
或者，
用所述两张图像中的另一张和其他图像，以及所述两张图像中的另一张和其他图像被水平翻转后的图像，组成结果序列。
可选的，所述训练序列中多张图像的顺序为随机确定的顺序，或者根据用户指令设定的顺序，或者在所述待配准图像序列中的顺序；
所述结果序列中多张图像的顺序为随机确定的顺序，或者根据用户指令设定的顺序，或者在所述待配准图像序列中的顺序。
可选的，所述针对所述待配准图像序列中的每一张图像，利用所述图像对应的变换矩阵处理所述图像，获得所述图像对应的配准结果图像，包括：
针对所述待配准图像序列的每一张图像，判断所述图像是否发生翻转；
若所述图像未发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的坐标，获得所述图像对应的配准结果图像；
若所述图像发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的坐标，获得所述图像对应的变换后图像；
对所述图像进行水平翻转得到翻转图像，并根据所述图像的变换矩阵调整所述翻转图像中各像素的坐标，获得所述翻转图像对应的变换后翻转图像；
根据所述变换后图像相对于后一张图像的偏差，以及所述变换后翻转图像相对于所述后一张图像的偏差，将所述变换后图像或者所述变换后翻转图像确定为所述图像的配准结果；其中，所述后一张图像指代所述待配准图像序列中位于所述图像后一张的图像。
可选的，所述获得待配准图像序列，包括：
获得分别拍摄同一物体的多个切片而得到的多张图像；其中，每一张所述图像均显示所述物体的一个切片；
将所述多张图像的尺寸均裁剪为预设的目标尺寸，得到待配准图像序列。
本申请第二方面提供一种图像配准方法，包括：
获得待配准图像序列；
利用预先训练好的配准模型逐一处理所述待配准图像序列的每一张图像，获得每一张所述图像对应的变换矩阵；
针对每一张所述图像，利用所述图像对应的变换矩阵处理所述图像，获得所述图像的配准结果。
可选的，训练所述配准模型的过程，包括：
用所述待配准图像序列中除末张图像以外的每一张图像组成训练序列；
用所述待配准图像序列中除首张图像以外的每一张图像组成结果序列；
或者，
用所述待配准图像序列中除首张图像以外的每一张图像，以及除首张图像以外的每一张图像被水平翻转后的图像，组成结果序列；
利用所述训练序列和所述结果序列训练得到图像配准模型。
可选的，训练所述配准模型的过程，包括：
随机选中所述待配准图像序列的两张图像；
用所述两张图像中的一张和其他图像组成训练序列；其中，所述其他图像指代所述待配准图像序列中未选中的图像；
用所述两张图像中的另一张和其他图像组成结果序列；
或者，
用所述两张图像中的另一张和其他图像，以及所述两张图像中的另一张和其他图像被水平翻转后的图像，组成结果序列；
利用所述训练序列和所述结果序列训练得到图像配准模型。
可选的，所述利用所述图像对应的变换矩阵处理所述图像，获得所述图像的配准结果，包括：
若所述图像未发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的坐标，获得所述图像对应的配准结果图像；
若所述图像发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的坐标，获得所述图像对应的变换后图像；
对所述图像进行水平翻转得到翻转图像，并根据所述图像的变换矩阵调整所述翻转图像中各像素的坐标，获得所述翻转图像对应的变换后翻转图像；
根据所述变换后图像相对于后一张图像的偏差，以及所述变换后翻转图像相对于所述后一张图像的偏差，将所述变换后图像或者所述变换后翻转图像确定为所述图像的配准结果；其中，所述后一张图像指代所述待配准图像序列中位于所述图像后一张的图像。
本申请第三方面提供一种图像配准装置，包括：
获得单元，用于获得待配准图像序列；
训练单元，用于利用所述待配准图像序列训练得到图像配准模型；
提取单元，用于利用所述图像配准模型提取所述待配准图像序列中每一张图像对应的变换矩阵；其中，所述变换矩阵包括竖直偏移量，水平偏移量和旋转角度三项参数；
处理单元，用于针对所述待配准图像序列中的每一张图像，利用所述图像对应的变换矩阵处理所述图像，获得所述图像对应的配准结果图像。
可选的，所述获得单元获得待配准图像序列时，具体用于：
获得对同一物体连续拍摄得到的多张图像；
将所述多张图像的尺寸均裁剪为预设的目标尺寸，得到待配准图像序列。
可选的，所述训练单元利用所述待配准图像序列训练得到图像配准模型时，具体用于：
根据所述待配准图像序列确定训练序列和结果序列；
利用所述训练序列和所述结果序列训练得到图像配准模型。
可选的，所述训练单元根据所述待配准图像序列确定训练序列和结果序列时，具体用于：
用所述待配准图像序列中除末张图像以外的每一张图像组成训练序列，并用所述待配准图像序列中除首张图像以外的每一张图像组成结果序列；
或者，
用所述待配准图像序列中除末张图像以外的每一张图像组成训练序列；
用所述待配准图像序列中除首张图像以外的每一张图像，以及除首张图像以外的每一张图像被水平翻转后的图像，组成结果序列。
可选的，所述训练单元根据所述待配准图像序列确定训练序列和结果序列时，具体用于：
随机选中所述待配准图像序列的两张图像；
用所述两张图像中的一张和其他图像组成训练序列；其中，所述其他图像指代所述待配准图像序列中未选中的图像；
用所述两张图像中的另一张和其他图像组成结果序列；
或者，
用所述两张图像中的另一张和其他图像，以及所述两张图像中的另一张和其他图像被水平翻转后的图像，组成结果序列。
可选的，所述训练序列中多张图像的顺序为随机确定的顺序，或者根据用户指令设定的顺序，或者在所述待配准图像序列中的顺序；
所述结果序列中多张图像的顺序为随机确定的顺序，或者根据用户指令设定的顺序，或者在所述待配准图像序列中的顺序。
可选的，所述处理单元针对所述待配准图像序列中的每一张图像，利用所述图像对应的变换矩阵处理所述图像，获得所述图像对应的配准结果图像时，具体用于：
针对所述待配准图像序列的每一张图像，判断所述图像是否发生翻转；
若所述图像未发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的坐标，获得所述图像对应的配准结果图像；
若所述图像发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的坐标，获得所述图像对应的变换后图像；
对所述图像进行水平翻转得到翻转图像，并根据所述图像的变换矩阵调整所述翻转图像中各像素的坐标，获得所述翻转图像对应的变换后翻转图像；
根据所述变换后图像相对于后一张图像的偏差，以及所述变换后翻转图像相对于所述后一张图像的偏差，将所述变换后图像或者所述变换后翻转图像确定为所述图像的配准结果；其中，所述后一张图像指代所述待配准图像序列中位于所述图像后一张的图像。
可选的，所述获得单元获得待配准图像序列时，具体用于：
获得分别拍摄同一物体的多个切片而得到的多张图像；其中，每一张所述图像均显示所述物体的一个切片；
将所述多张图像的尺寸均裁剪为预设的目标尺寸，得到待配准图像序列。
本申请第四方面提供一种电子设备，包括存储器和处理器；
其中，所述存储器用于存储计算机程序；
所述处理器用于执行所述计算机程序，所述计算机程序被执行时，具体用于实现本申请第一方面或第二方面任意一项所提供的图像配准方法。
本申请第五方面提供一种计算机存储介质，用于存储计算机程序，所述计算机程序被执行时，具体用于实现本申请第一方面或第二方面任意一项所提供的图像配准方法。
本申请提供一种图像配准方法、装置、设备和存储介质，方法包括，获得待配准图像序列；利用待配准图像序列训练得到图像配准模型；利用图像配准模型提取待配准图像序列中每一张图像对应的变换矩阵；变换矩阵包括竖直偏移量，水平偏移量和旋转角度三项参数；针对待配准图像序列中的每一张图像，利用图像对应的变换矩阵处理图像，获得图像对应的配准结果图像。本方案在配准过程中不需要依赖于图像中的特征点，因此对于难以提取特征点的医学图像(或其他类似图像)也能够有效地进行刚性配准。
附图说明
图1为本申请实施例提供的一种图像配准模型的训练方法的流程图；
图2为本申请实施例提供的一种图像翻转的示例；
图3为本申请实施例提供的一种图像配准方法的流程图；
图4为本申请实施例提供的一种图像配准过程的示例图；
图5为本申请实施例提供的一种图像配准效果的示意图；
图6为本申请实施例提供的一种图像序列的配准结果示意图；
图7为本申请实施例提供的一种图像配准装置的结构示意图；
图8为本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
本申请提供一种成像方法以及成像系统，用于提高图像显示的全面性。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
本申请实施例提供的图像配准方法，可以用于对医学图像进行配准，也可以用于对其他具有和医学图像相似特征(例如以黑色背景为主的灰度图)的图像进行配准。
本申请实施例提供的图像配准方法，需要先用待配准图像序列训练得到一个图像配准模型，然后利用该图像配准模型对待配准图像序列中的每一张图像进行配准。
请参见图1，为本申请实施例提供的待配准图像序列的训练方法的流程图，该方法可以包括如下步骤。
S101，获得待配准图像序列。
待配准图像序列的获得方式可以是，首先对同一对象连续拍摄得到多张图像，然后对每一张图像进行裁剪处理，将每一张图像的尺寸均裁剪为预设的目标尺寸，裁剪后的多张图像构成的图像序列，就是待配准图像序列。
上述获得待配准图像序列的方式，相当于：
获得对同一物体连续拍摄得到的多张图像；
将所述多张图像的尺寸均裁剪为预设的目标尺寸，得到待配准图像序列。
在一些可选的实施例中，待配准图像序列的获得方式也可以是，对同一物体的多个切片分别进行拍摄，获得多张图像，其中每一张图像都显示该物体的一个切片，然后将多张图像的尺寸均裁剪为预设的目标尺寸，裁剪后的多张图像构成待配准图像序列。
示例性的，这里的同一物体可以是用于试验的生物组织样本，例如可以是小鼠心脏。
可选的，若拍摄得到的多张图像的尺寸就是预设的目标尺寸，也可以不进行裁剪，直接将拍摄得到的多张图像构成的图像序列确定为待配准图像序列。
S102，将待配准图像序列分解为第一图像序列和第二图像序列。
在步骤S102中，可以采用多种方式分解出第一图像序列和第二图像序列，作为示例，下面介绍两种可选的分解方式。
第一种分解方式为，第一图像序列包括待配准图像序列中除末张(即最后一张)图像以外的每一张图像，第二图像序列包括待配准图像序列中除首张(即第一张)图像以外的每一张图像。
也就是说，假设待配准图像序列包括n张图像，依次记为图像1至图像n，则第一图像序列包括图像1至图像n-1，第二图像序列包括图像2至图像n。
第二种分解方式为：
在待配准图像序列中随机选中两张图像，将随机选中的这两张图像一张放在第一图像序列，另一张放在第二图像序列，然后将剩余的未被选中的所有图像，同时放在第一图像序列和第二图像序列。
示例性的，待配准图像序列包括图像1至图像6，随机选中其中的图像1和图像3，将其中图像1放在第一图像序列，图像3放在第二图像序列，剩余未被选中的图像2，以及图像4至图像6同时放入第一图像序列和第二图像序列。由此，第一图像序列包括图像1，图像2，图像4至图像6；第二图像序列包括图像3，图像2，图像4至图像6。
可选的，第一图像序列和第二图像序列中图像的排序，可以和原本待配准图像序列中的排序一致，也可以和原本待配准图像序列中的顺序不一致，而是随机排列，还可以是根据用户指令设定的自定义的顺序排列。
S103，判断待配准图像序列中是否存在翻转情况。
若待配准图像序列中不存在翻转情况，执行步骤S104，若待配准图像序列中存在翻转情况，执行步骤S105。
对于待配准图像序列中的任意一张图像，若这张图像中被拍摄的对象的朝向，和后一张图像中被拍摄的对象的朝向相反，则认为这张图像相对于后一张图像发生翻转。
以图2为例，图2中图像a的箭头的朝向，和图像b的箭头的朝向相反，因此可以认为图像a相对于图像b发生翻转。
在步骤S103中，如果待配准图像序列中有至少一张图像发生翻转，则判断出待配准图像序列存在翻转情况，若待配准图像序列中每一张图像均未发生翻转，则判断出待配准图像序列不存在翻转情况。
待配准图像中每一张图像是否发生翻转，可以预先通过人工标定的方式标定，或者采用特定的识别算法识别，然后将标定或识别的结果标注在每一张图像上。
S104，将第一图像序列确定为训练序列，并将第二图像序列确定为结果序列。
也就是说，当待配准图像序列中不存在翻转情况时，训练序列可以表示为：(图像1，图像2，……图像n-1)，结果序列可以表示为：(图像2，图像3，……图像n)。
S105，复制第一图像序列得到第一图像序列的一个副本，将第二图像序列中每一张图像进行水平翻转，获得翻转图像序列。
在步骤S105中，可以在图像的编号之后增加L表示翻转后的图像，比如将图像2进行水平翻转后得到的图像，可以记为图像2L，由此，翻转图像序列可以表示为(图像2L，图像3L……图像nL)。
S106，拼接第一图像序列和第一图像序列的副本得到训练序列，拼接第二图像序列和翻转图像序列得到结果序列。
在步骤S106中，第一图像序列的副本可以拼接在第一图像序列的末尾，翻转图像序列可以拼接在第二图像序列的末尾，
续接前述示例，在步骤S106中，训练序列可以表示为(图像1，图像2，……图像n-1，图像1，图像2，……图像n-1)，结果序列可以表示为(图像2，图像3，……图像n，图像2L，图像3L……图像nL)。
步骤S102至S106的过程，相当于根据待配准图像序列确定训练序列和结果序列。
当按照第一种分解方式分解出第一图像序列和第二图像序列时，根据待配准图像序列确定训练序列和结果序列，可以包括：
用待配准图像序列中除末张图像以外的每一张图像组成训练序列，并用待配准图像序列中除首张图像以外的每一张图像组成结果序列；
或者，
用待配准图像序列中除末张图像以外的每一张图像组成训练序列；
用待配准图像序列中除首张图像以外的每一张图像，以及除首张图像以外的每一张图像被水平翻转后的图像，组成结果序列。
当按照第二种分解方式分解出第一图像序列和第二图像序列时，根据待配准图像序列确定训练序列和结果序列，可以包括：
随机选中待配准图像序列的两张图像；
用两张图像中的一张和其他图像组成训练序列；其中，其他图像指代待配准图像序列中未选中的图像；
用两张图像中的另一张和其他图像组成结果序列；
或者，
用两张图像中的另一张和其他图像，以及两张图像中的另一张和其他图像被水平翻转后的图像，组成结果序列。
S107，根据训练序列的图像和结果序列的图像确定待训练的图像配准模型的模型损失值。
待训练的图像配准模型，可以是基于任意一种现有的深度学习网络结构预先构建的深度学习模型，本实施例对待训练的图像配准模型的具体结构不做限定。
示例性的，待训练的图像配准模型可以是包括多个卷积层(例如可以是三个)的卷积神经网络(Convolutional Neural Network，CNN)模型，也可以是基于注意力机制的循环神经网络(Recurrent Neural Networks attention，RNN attention)模型。
步骤S107的具体实施方式可以包括：
A1，首先用待训练的图像配准模型处理训练序列中每一张图像，获得训练序列中每一张图像对应的变换矩阵。
上述获取图像的变换矩阵的过程，由图像配准模型的具体结构决定，本实施例对此不作限定。
示例性的，当图像配准模型为CNN模型时，获取变换矩阵的过程可以是，利用CNN模型中的多个卷积层逐一对输入的一张图像进行特征提取，最后一个卷积层输出的特征，就是输入的图像的变换矩阵。
例如，假设CNN模型包括三个卷积层，对于一张训练序列中的图像，第一个卷积层首先对其进行特征提取，获得第一图像特征，然后第二个卷积层对第一图像特征进行特征提取，获得第二图像特征，紧接着第三个卷积层对第二图像特征进行特征提取，获得图像对应的变换矩阵。
在本实施例中，变换矩阵可以是一个2*3，即2行3列的矩阵，矩阵中各元素的值取决于竖直偏移量y，水平偏移量x和旋转角度θ。
具体的，变换矩阵可以表示为：
Figure PCTCN2022141486-appb-000001
A2，针对训练序列中每一张图像，用该图像对应的变换矩阵处理该图像，获得该图像对应的训练结果图像。
示例性的，对于训练序列(图像1，图像2，……图像n-1，图像1，图像2，……图像n-1)，用图像1对应的变换矩阵处理图像1，获得对应的训练结 果图像1，用图像2对应的变换矩阵处理图像2，获得对应的训练结果图像2，以此类推。
步骤A2中，用变换矩阵处理一张图像的具体过程可以是，针对图像中每一个像素，用该像素的坐标和图像对应的变换矩阵做矩阵乘法运算，获得变换后的坐标，然后将该像素移到变换后的坐标处，对图像中全部像素执行完上述操作后，这些移动至变换后的坐标的像素就组成了处理后的训练结果图像。
上述变换后的坐标的具体计算方法可以参见下述公式：
Figure PCTCN2022141486-appb-000002
上述公式中，x1和y1表示像素原本的坐标，x2和y2表示变换后的坐标。
A3，计算训练序列中每一张图像对应的训练结果图像和结果序列中每一张图像的偏差，获得模型损失值。
在步骤A3中，针对训练序列的每一张图像，计算该图像对应的训练结果图像和结果序列中与该图像位于同一位置的图像的偏差，然后根据多个偏差确定模型损失值。
训练序列的图像和结果序列的图像位于同一位置，是指，这两张图像在各自所属的序列中的排序一致。例如，训练序列为(图像1，图像2，……图像n-1)，结果序列为(图像2，图像3，……图像n)，训练序列的第一张图像为图像1，结果序列的第一张图像为图像2，因此可以认为训练序列的图像1和结果序列的图像2位于同一位置，即两者都是各自所属序列的第一张图像；同理，训练序列的图像2和结果序列的图像3都是各自所属序列的第二张图像，可以认为训练序列的图像2和结果序列的图像3位于同一位置。
具体的，针对训练序列的第一张图像，即图像1，经过A2的处理获得对应的训练结果图像1，在A3中，则可以计算训练结果图像1和结果序列的第一张图像，即图像2之间的偏差。
也就是说在步骤A3中，依次计算，训练结果图像1和结果序列的图像2的偏差，训练结果图像2和结果序列的图像3的偏差，训练结果图像3和结果序列的图像4的偏差，以此类推。
两张图像之间的偏差，可以用均方误差(Mean Square Error，MSE)表示， 也可以用其他指标，本实施例不做限定。两张图像的均方误差的具体计算方法可以参考相关的现有技术文献。
在步骤A3中，可以将计算得到的每一个偏差均确定为模型损失值，也可以将计算得到的全部偏差中最大的偏差确定为模型损失值，还可以将计算得到的全部偏差的总和确定为模型损失值，本实施例对根据多张图像的偏差确定模型损失值的具体方式不做限定。
S108，判断模型损失值是否符合预设的收敛条件。
若模型损失值不符合收敛条件，则执行步骤S109，若模型损失值符合收敛条件，则执行步骤S110。
收敛条件的具体内容不做限定。示例性的，收敛条件可以是模型损失值小于预设的收敛阈值。可选的，模型损失值有多个，则收敛条件可以是每一个模型损失值均小于收敛阈值。
S109，根据模型损失值更新待训练的配准模型的参数。
步骤S109执行完之后，返回执行步骤S107，用更新参数后的配准模型再次处理训练序列中的图像，从而获得新的模型损失值。
可选的，在步骤S109中，可以利用反向传播算法，根据模型损失值确定待训练的配准模型中每一个参数对应的更新量，然后按确定的更新量更新模型中的每一个参数的值。
反向传播算法的具体实现可以参考相关的现有技术文献，不再赘述。
S110，输出训练好的配准模型。
若步骤S108中判断出模型损失值符合收敛条件，则此时的配准模型，就是训练好的配准模型。
上述实施例的方法中，步骤S101之后的所有步骤，相当于：
利用待配准图像序列训练得到图像配准模型。
其中，步骤S102至S106所述的过程，相当于：
根据待配准图像序列确定训练序列和结果序列。
其中，训练序列包括待配准图像序列中除末张图像以外的每一张图像，结果序列包括待配准图像序列中除首张图像以外的每一张图像；
或者，
训练序列包括待配准图像序列中除末张图像以外的每一张图像及其副本，结果序列包括待配准图像序列中除首张图像以外的每一张图像以及除首张图像以外的每一张图像被水平翻转后的图像。
步骤S107至S110所述的过程，相当于：
利用训练序列和结果序列训练得到图像配准模型。
根据图1所示的方法完成配准模型的训练后，就可以利用该配准模型对待配准图像图像序列进行配准，请参见图3，为本申请实施例提供的图像配准方法的流程图，该方法可以包括如下步骤。
S301，获得待配准图像序列。
步骤S301的具体实施方式和步骤S101一致，不再赘述。
S302，利用训练好的配准模型逐一处理待配准图像序列的每一张图像，获得每一张图像对应的变换矩阵。
步骤S302的具体实施方式，可以参见图1对应的实施例中的步骤A1，不再赘述。
S303，判断待配准图像序列中每一张图像是否发生翻转。
请参见步骤S103，待配准图像中每一张图像是否发生翻转，可以预先通过人工标定的方式标定，或者采用特定的识别算法识别，然后将标定或识别的结果标注在每一张图像上，因此在步骤S303中，可以根据待配准图像序列中每一张图像的标定结果或识别结果，确定每一张图像是否发生翻转。
对于发生翻转的图像，执行步骤S304，获得该图像对应的配准结果。
对于未发生翻转的图像，执行步骤S305，获得该图像对应的配准结果。
S304，利用对应的变换矩阵处理发生翻转的图像，并根据偏差选择配准结果图像。
在步骤S304中，对于待配准图像序列中一张发生翻转的图像，可以先用S302中获得的该图像对应的变换矩阵，按步骤A2所述的方式处理该图像，获得变换后图像。
然后，比对变换后图像和发生翻转的图像的后一张图像，获得第一偏差值；并且比对变换后图像，和后一张图像翻转后的图像，获得第二偏差值。第一偏差值和第二偏差值可以是被比对的两张图像的均方误差，或者也可以是其他可 以衡量两张图像的偏差的指标，不做限定。
最后选取第一偏差和第二偏差中较小的偏差对应的图像作为发生翻转的图像的配准结果图像。
示例性的，假设待配准图像序列中图像1发生翻转，显然图像1的后一张图像为图像2，用图像1对应的变换矩阵处理图像1，获得变换后图像，记为变换图像1。1L为翻转后的图像1，用图像1L对应的变换矩阵处理图像1L，获得变换后图像，记为变换图像1L。
然后比对变换图像1和图像2，获得第一偏差值；比对变换图像1L和图像2，获得第二偏差值，如果第一偏差值小于第二偏差值，就将变换图像1确定为图像1对应的输出结果，也就是作为配准结果图像。
如果第二偏差值小于第一偏差值，就将变换图像1L确定为图像1对应的输出结果，也就是作为配准结果图像。
步骤S304的有益效果在于，通过比对图像偏差的方式，选择偏差较小的图像作为发生翻转的图像的配准结果图像，从而提高对发生翻转的图像进行配准时的准确度。
S305，利用对应的变换矩阵处理未发生翻转的图像，获得该图像的配准结果。
在步骤S305中，未发生翻转的图像经过图像配准模型处理后，获得对应的变换矩阵，然后用该变换矩阵按照步骤A2中的处理方式处理未发生翻转的图像，就可以获得该凸显的配准结果图像。
示例性的，若待配准图像序列中图像5未发生翻转，在步骤S302中，利用图像配准模型处理图像5，获得图像5对应的变换矩阵，然后用图像5对应的变换矩阵按照步骤A2所述的方式处理图像5，获得图像5对应的配准结果图像。
步骤S303至S305所述的过程，相当于：
针对待配准图像序列中的每一张图像，利用图像对应的变换矩阵处理图像，获得图像对应的配准结果图像。
其中，步骤S303相当于：针对待配准图像序列的每一张图像，判断图像是否发生翻转。
步骤S304相当于：
若图像未发生翻转，根据图像的变换矩阵调整图像中各像素的坐标，获得图像对应的配准结果图像。
步骤S305相当于：
若图像发生翻转，根据图像的变换矩阵调整图像中各像素的坐标，获得图像对应的变换后图像，根据变换后图像，待配准图像序列中图像的前一张图像，以及前一张图像被水平翻转后的图像之间的偏差，确定图像的配准结果图像。
为便于理解本实施例提供的图像配准方法，请参见图4，为本实施例提供的一种图像配准过程的示例图。
如图4所示，根据本实施例提供的图像配准方法，首先将输入图像输入到训练好的图像配准模型，从而获得输入图像对应的变换矩阵，该过程相当于图3所示实施例的步骤S302。然后，根据输入图像对应的变换矩阵，对输入图像进行配准，获得输出图像，也就是输入图像对应的配准结果图像，该过程相当于图3所示实施例的步骤S303至S305。
图4所示的示例中，输入图像可以是步骤S301所述的待配准图像序列中的每一张图像，比如可以是待配准图像序列中的图像1，图像2等。
图5为本申请实施例提供的一种图像配准效果的示意图。如图5所示，利用图像配准模型对待配准图像序列中的第k张图像进行配准后，获得第k张图像对应的配准结果图像。
可以看到，经过配准，第k张图像对应的配准结果图像中，目标物体的位置已经向前一张，也就是第k-1张图像中目标物体的位置对齐。
图6为本申请实施例提供的一种对整个待配准图像序列进行配准后获得的配准结果示意图，其中图(1)为对小鼠心脏连续多次拍摄后获得的待配准图像序列，图(2)为按照本实施例提供的配准方法对图(1)的待配准图像序列配准后获得的配准结果图像的序列。
可以看到，在配准结果图像的序列中，第2张以及之后的每一张图像中的小鼠心脏均向第1张图像中的小鼠心脏对齐，配准效果较好。
本申请实施例所提供的图像配准方法的有益效果在于，利用待配准图像序列训练得到的图像配准模型提取待配准图像序列中每一图像的变换矩阵，然后 针对每一待配准图像序列中每一图像，利用该图像的变换矩阵处理该图像，获得配准结果图像。
本方案在配准过程中不需要依赖于图像中的特征点，因此对于难以提取特征点的医学图像(或其他类似图像)也能够有效地进行刚性配准。
根据本申请实施例提供的图像配准方法，本申请实施例还提供一种图像配准装置，请参见图7，该装置可以包括如下单元。
获得单元701，用于获得待配准图像序列。
训练单元702，用于利用待配准图像序列训练得到图像配准模型。
提取单元703，用于利用图像配准模型提取待配准图像序列中每一张图像对应的变换矩阵。
处理单元704，用于针对待配准图像序列中的每一张图像，利用图像对应的变换矩阵处理图像，获得图像对应的配准结果图像。
可选的，获得单元701获得待配准图像序列时，具体用于：
获得对同一物体连续拍摄得到的多张图像；
将多张图像的尺寸均裁剪为预设的目标尺寸，得到待配准图像序列。
可选的，获得单元701获得待配准图像序列时，具体用于：
获得分别拍摄同一物体的多个切片而得到的多张图像；其中，每一张图像均显示物体的一个切片；
将多张图像的尺寸均裁剪为预设的目标尺寸，得到待配准图像序列。
可选的，训练单元702利用待配准图像序列训练得到图像配准模型时，具体用于：
根据待配准图像序列确定训练序列和结果序列；
利用训练序列和结果序列训练得到图像配准模型。
可选的，训练单元702根据待配准图像序列确定训练序列和结果序列时，具体用于：
用待配准图像序列中除末张图像以外的每一张图像组成训练序列，并用待配准图像序列中除首张图像以外的每一张图像组成结果序列；
或者，
用待配准图像序列中除末张图像以外的每一张图像组成训练序列；
用待配准图像序列中除首张图像以外的每一张图像，以及除首张图像以外的每一张图像被水平翻转后的图像，组成结果序列。
可选的，训练单元702根据待配准图像序列确定训练序列和结果序列时，具体用于：
随机选中待配准图像序列的两张图像；
用两张图像中的一张和其他图像组成训练序列；其中，其他图像指代待配准图像序列中未选中的图像；
用两张图像中的另一张和其他图像组成结果序列；
或者，
用两张图像中的另一张和其他图像，以及两张图像中的另一张和其他图像被水平翻转后的图像，组成结果序列。
可选的，训练序列中多张图像的顺序为随机确定的顺序，或者根据用户指令设定的顺序，或者在待配准图像序列中的顺序；
结果序列中多张图像的顺序为随机确定的顺序，或者根据用户指令设定的顺序，或者在待配准图像序列中的顺序。
可选的，处理单元704针对待配准图像序列中的每一张图像，利用图像对应的变换矩阵处理图像，获得图像对应的配准结果图像时，具体用于：
针对待配准图像序列的每一张图像，判断图像是否发生翻转；
若图像未发生翻转，根据图像的变换矩阵调整图像中各像素的坐标，获得图像对应的配准结果图像；
若图像发生翻转，根据图像的变换矩阵调整图像中各像素的坐标，获得图像对应的变换后图像；
对图像进行水平翻转得到翻转图像，并根据图像的变换矩阵调整翻转图像中各像素的坐标，获得翻转图像对应的变换后翻转图像；
根据变换后图像相对于后一张图像的偏差，以及变换后翻转图像相对于后一张图像的偏差，将变换后图像或者变换后翻转图像确定为图像的配准结果；其中，后一张图像指代待配准图像序列中位于图像后一张的图像。
本实施例提供的图像配准装置，其具体工作原理可以参见本申请任一实施例提供的图像配准方法中的相关步骤，此处不再赘述。
本申请提供一种图像配准装置，装置包括，获得单元701获得待配准图像序列；训练单元702利用待配准图像序列训练得到图像配准模型；提取单元703利用图像配准模型提取待配准图像序列中每一张图像对应的变换矩阵；处理单元704针对待配准图像序列中的每一张图像，利用图像对应的变换矩阵处理图像，获得图像对应的配准结果图像。本方案在配准过程中不需要依赖于图像中的特征点，因此对于难以提取特征点的医学图像(或其他类似图像)也能够有效地进行刚性配准。
本申请实施例还提供一种电子设备，请参见图8，包括存储器801和处理器802。
其中，存储器801用于存储计算机程序。
处理器802用于执行计算机程序，计算机程序被执行时，具体用于实现本申请任一实施例所提供的图像配准方法。
本申请实施例还提供一种计算机存储介质，用于存储计算机程序，计算机程序被执行时，具体用于实现本申请任一实施例所提供的图像配准方法。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (15)

  1. 一种图像配准方法，其特征在于，包括：
    获得待配准图像序列；
    利用所述待配准图像序列训练得到图像配准模型；
    利用所述图像配准模型提取所述待配准图像序列中每一张图像对应的变换矩阵；其中，所述变换矩阵包括竖直偏移量，水平偏移量和旋转角度三项参数；
    针对所述待配准图像序列中的每一张图像，利用所述图像对应的变换矩阵处理所述图像，获得所述图像对应的配准结果图像。
  2. 根据权利要求1所述的方法，其特征在于，所述获得待配准图像序列，包括：
    获得对同一物体连续拍摄得到的多张图像；
    将所述多张图像的尺寸均裁剪为预设的目标尺寸，得到待配准图像序列。
  3. 根据权利要求1所述的方法，其特征在于，所述利用所述待配准图像序列训练得到图像配准模型，包括：
    根据所述待配准图像序列确定训练序列和结果序列；
    利用所述训练序列和所述结果序列训练得到图像配准模型。
  4. 根据权利要求3所述的方法，其特征在于，所述根据所述待配准图像序列确定训练序列和结果序列，包括：
    用所述待配准图像序列中除末张图像以外的每一张图像组成训练序列，并用所述待配准图像序列中除首张图像以外的每一张图像组成结果序列；
    或者，
    用所述待配准图像序列中除末张图像以外的每一张图像组成训练序列；
    用所述待配准图像序列中除首张图像以外的每一张图像，以及除首张图像以外的每一张图像被水平翻转后的图像，组成结果序列。
  5. 根据权利要求3所述的方法，其特征在于，所述根据所述待配准图像序列确定训练序列和结果序列，包括：
    随机选中所述待配准图像序列的两张图像；
    用所述两张图像中的一张和其他图像组成训练序列；其中，所述其他图像 指代所述待配准图像序列中未选中的图像；
    用所述两张图像中的另一张和其他图像组成结果序列；
    或者，
    用所述两张图像中的另一张和其他图像，以及所述两张图像中的另一张和其他图像被水平翻转后的图像，组成结果序列。
  6. 根据权利要求3或4所述的方法，其特征在于，所述训练序列中多张图像的顺序为随机确定的顺序，或者根据用户指令设定的顺序，或者在所述待配准图像序列中的顺序；
    所述结果序列中多张图像的顺序为随机确定的顺序，或者根据用户指令设定的顺序，或者在所述待配准图像序列中的顺序。
  7. 根据权利要求1所述的方法，其特征在于，所述针对所述待配准图像序列中的每一张图像，利用所述图像对应的变换矩阵处理所述图像，获得所述图像对应的配准结果图像，包括：
    针对所述待配准图像序列的每一张图像，判断所述图像是否发生翻转；
    若所述图像未发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的坐标，获得所述图像对应的配准结果图像；
    若所述图像发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的坐标，获得所述图像对应的变换后图像；
    对所述图像进行水平翻转得到翻转图像，并根据所述图像的变换矩阵调整所述翻转图像中各像素的坐标，获得所述翻转图像对应的变换后翻转图像；
    根据所述变换后图像相对于后一张图像的偏差，以及所述变换后翻转图像相对于所述后一张图像的偏差，将所述变换后图像或者所述变换后翻转图像确定为所述图像的配准结果；其中，所述后一张图像指代所述待配准图像序列中位于所述图像后一张的图像。
  8. 根据权利要求1所述的方法，其特征在于，所述获得待配准图像序列，包括：
    获得分别拍摄同一物体的多个切片而得到的多张图像；其中，每一张所述图像均显示所述物体的一个切片；
    将所述多张图像的尺寸均裁剪为预设的目标尺寸，得到待配准图像序列。
  9. 一种图像配准方法，其特征在于，包括：
    获得待配准图像序列；
    利用预先训练好的配准模型逐一处理所述待配准图像序列的每一张图像，获得每一张所述图像对应的变换矩阵；
    针对每一张所述图像，利用所述图像对应的变换矩阵处理所述图像，获得所述图像的配准结果。
  10. 根据权利要求9所述的方法，其特征在于，训练所述配准模型的过程，包括：
    用所述待配准图像序列中除末张图像以外的每一张图像组成训练序列；
    用所述待配准图像序列中除首张图像以外的每一张图像组成结果序列；
    或者，
    用所述待配准图像序列中除首张图像以外的每一张图像，以及除首张图像以外的每一张图像被水平翻转后的图像，组成结果序列；
    利用所述训练序列和所述结果序列训练得到图像配准模型。
  11. 根据权利要求9所述的方法，其特征在于，训练所述配准模型的过程，包括：
    随机选中所述待配准图像序列的两张图像；
    用所述两张图像中的一张和其他图像组成训练序列；其中，所述其他图像指代所述待配准图像序列中未选中的图像；
    用所述两张图像中的另一张和其他图像组成结果序列；
    或者，
    用所述两张图像中的另一张和其他图像，以及所述两张图像中的另一张和其他图像被水平翻转后的图像，组成结果序列；
    利用所述训练序列和所述结果序列训练得到图像配准模型。
  12. 根据权利要求9至11任意一项所述的方法，其特征在于，所述利用所述图像对应的变换矩阵处理所述图像，获得所述图像的配准结果，包括：
    若所述图像未发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的坐标，获得所述图像对应的配准结果图像；
    若所述图像发生翻转，根据所述图像的变换矩阵调整所述图像中各像素的 坐标，获得所述图像对应的变换后图像；
    对所述图像进行水平翻转得到翻转图像，并根据所述图像的变换矩阵调整所述翻转图像中各像素的坐标，获得所述翻转图像对应的变换后翻转图像；
    根据所述变换后图像相对于后一张图像的偏差，以及所述变换后翻转图像相对于所述后一张图像的偏差，将所述变换后图像或者所述变换后翻转图像确定为所述图像的配准结果；其中，所述后一张图像指代所述待配准图像序列中位于所述图像后一张的图像。
  13. 一种图像配准装置，其特征在于，包括：
    获得单元，用于获得待配准图像序列；
    训练单元，用于利用所述待配准图像序列训练得到图像配准模型；
    提取单元，用于利用所述图像配准模型提取所述待配准图像序列中每一张图像对应的变换矩阵；其中，所述变换矩阵包括竖直偏移量，水平偏移量和旋转角度三项参数；
    处理单元，用于针对所述待配准图像序列中的每一张图像，利用所述图像对应的变换矩阵处理所述图像，获得所述图像对应的配准结果图像。
  14. 一种电子设备，其特征在于，包括存储器和处理器；
    其中，所述存储器用于存储计算机程序；
    所述处理器用于执行所述计算机程序，所述计算机程序被执行时，具体用于实现如权利要求1至12任意一项所述的图像配准方法。
  15. 一种计算机存储介质，其特征在于，用于存储计算机程序，所述计算机程序被执行时，具体用于实现如权利要求1至12任意一项所述的图像配准方法。
PCT/CN2022/141486 2022-12-23 2022-12-23 一种图像配准方法、装置、设备和存储介质 Ceased WO2024130705A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2022/141486 WO2024130705A1 (zh) 2022-12-23 2022-12-23 一种图像配准方法、装置、设备和存储介质
CN202280102252.8A CN120303692A (zh) 2022-12-23 2022-12-23 一种图像配准方法、装置、设备和存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/141486 WO2024130705A1 (zh) 2022-12-23 2022-12-23 一种图像配准方法、装置、设备和存储介质

Publications (1)

Publication Number Publication Date
WO2024130705A1 true WO2024130705A1 (zh) 2024-06-27

Family

ID=91587599

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/141486 Ceased WO2024130705A1 (zh) 2022-12-23 2022-12-23 一种图像配准方法、装置、设备和存储介质

Country Status (2)

Country Link
CN (1) CN120303692A (zh)
WO (1) WO2024130705A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109035316A (zh) * 2018-08-28 2018-12-18 北京安德医智科技有限公司 核磁共振图像序列的配准方法及设备
CN109472817A (zh) * 2018-09-27 2019-03-15 浙江工业大学 一种基于循环生成对抗网络的多序列磁共振图像配准方法
CN110853082A (zh) * 2019-10-21 2020-02-28 科大讯飞股份有限公司 医学图像配准方法、装置、电子设备及计算机存储介质
US20200380703A1 (en) * 2019-05-31 2020-12-03 Mitsubishi Electric Research Laboratories, Inc. Robust Image Registration For Multiple Rigid Transformed Images
CN113963037A (zh) * 2021-09-29 2022-01-21 上海联影医疗科技股份有限公司 图像配准方法、装置、计算机设备和存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109035316A (zh) * 2018-08-28 2018-12-18 北京安德医智科技有限公司 核磁共振图像序列的配准方法及设备
CN109472817A (zh) * 2018-09-27 2019-03-15 浙江工业大学 一种基于循环生成对抗网络的多序列磁共振图像配准方法
US20200380703A1 (en) * 2019-05-31 2020-12-03 Mitsubishi Electric Research Laboratories, Inc. Robust Image Registration For Multiple Rigid Transformed Images
CN110853082A (zh) * 2019-10-21 2020-02-28 科大讯飞股份有限公司 医学图像配准方法、装置、电子设备及计算机存储介质
CN113963037A (zh) * 2021-09-29 2022-01-21 上海联影医疗科技股份有限公司 图像配准方法、装置、计算机设备和存储介质

Also Published As

Publication number Publication date
CN120303692A (zh) 2025-07-11

Similar Documents

Publication Publication Date Title
EP3939006B1 (en) Feature point detection
CN112669348B (zh) 一种鱼体姿势估计和鱼体表型数据测量方法及装置
CN111582220B (zh) 一种基于移位图卷积神经网络骨骼点行为识别系统及其识别方法
CN110084236A (zh) 一种图像的矫正方法及装置
CN114972789A (zh) 一种大尺寸导光板缺陷可视化检测方法、系统及存储介质
CN110689501A (zh) 一种畸变校正方法、装置、电子设备及计算机可读存储介质
CN110992411A (zh) 图像配准模型的训练方法和装置
CN109685772B (zh) 一种基于配准失真表示的无参照立体图像质量评估方法
CN104867111A (zh) 一种基于分块模糊核集的非均匀视频盲去模糊方法
CN106204422B (zh) 基于区块子图搜索的超大幅影像快速匹配拼接方法
CN112712518A (zh) 鱼类计数方法、装置、电子设备及存储介质
CN109934765B (zh) 高速相机全景图像拼接方法
CN106296587A (zh) 轮胎模具图像的拼接方法
CN114581340A (zh) 一种图像校正方法及设备
CN115294086A (zh) 医学影像分割方法、分割模型训练方法、介质及电子设备
CN107895145A (zh) 基于卷积神经网络结合超高斯去噪估测手指受力的方法
WO2024065701A1 (zh) 图像修复方法、装置、设备和非瞬态计算机存储介质
CN114092862A (zh) 一种基于最优帧选取的动作评估方法
CN112883983A (zh) 特征提取方法、装置和电子系统
CN111968160A (zh) 图像匹配方法和存储介质
CN116757936B (zh) 图像匹配关系获取方法及其图像拼接方法
CN117422921A (zh) 基于Mask RCNN的花椒锈病图像识别方法及系统
WO2024130705A1 (zh) 一种图像配准方法、装置、设备和存储介质
CN115063822B (zh) 文档检测方法、装置、电子设备及存储介质
CN115359000B (zh) 一种无参考图像质量评价方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22968992

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280102252.8

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 202280102252.8

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22968992

Country of ref document: EP

Kind code of ref document: A1