WO2025001689A1 - 图像配准方法、电子设备以及计算机可读存储介质 - Google Patents

图像配准方法、电子设备以及计算机可读存储介质 Download PDF

Info

Publication number
WO2025001689A1
WO2025001689A1 PCT/CN2024/095576 CN2024095576W WO2025001689A1 WO 2025001689 A1 WO2025001689 A1 WO 2025001689A1 CN 2024095576 W CN2024095576 W CN 2024095576W WO 2025001689 A1 WO2025001689 A1 WO 2025001689A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
images
displacement vector
vector field
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/095576
Other languages
English (en)
French (fr)
Inventor
李孜
田琳
莫志榮
白晓宇
王普阳
吕乐
闫轲
金达开
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Publication of WO2025001689A1 publication Critical patent/WO2025001689A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/269—Analysis of motion using gradient-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/52—Scale-space analysis, e.g. wavelet analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Definitions

  • the present disclosure relates to the field of image registration, and in particular to an image registration method, an electronic device, and a computer-readable storage medium.
  • the embodiments of the present disclosure provide an image registration method, an electronic device, and a computer-readable storage medium to at least solve the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.
  • an image registration method comprising: acquiring at least two images, wherein any one of the images contains shooting results of an object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining a target displacement vector field of the object to be analyzed based on the extracted image features; and performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.
  • an image registration method comprising: acquiring at least two medical images, wherein any one of the medical images contains scanning results of a target part of the same biological object under different conditions by means of computed tomography technology; performing feature extraction on the at least two medical images respectively to obtain image features of any one of the images, wherein the image features include: global features and local features; determining a target displacement vector field of the target part based on the extracted image features; and performing image registration on the at least two medical images based on the target displacement vector field of the target part to obtain an image registration result.
  • an image registration method comprising: in response to an input instruction on an operation interface, displaying at least two images on the operation interface, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; in response to the image registration instruction on the operation interface, displaying the image registration result on the operation interface, wherein the image registration result is based on the target image of the object to be analyzed.
  • the target displacement vector field is obtained by performing image registration on at least two images.
  • the target displacement vector field is determined based on extracted image features.
  • the extracted image features include: global features and local features.
  • the extracted image features are obtained by performing feature extraction on at least two images respectively.
  • an image registration method including: displaying at least two images on a presentation screen of a virtual reality VR device or an augmented reality AR device, wherein any one of the images contains shooting results of an object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining a target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; and driving the VR device or the AR device to render and display the image registration result.
  • an image registration method comprising: acquiring at least two images by calling a first interface, wherein the first interface includes a first parameter, and a parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining a target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; and outputting the image registration result by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the image registration result.
  • an electronic device including: a memory configured to store an executable program; and a processor configured to run the program, wherein the program executes any one of the methods in the above embodiments when it is run.
  • a computer program product including a computer program, which implements the method described in any of the above aspects when executed by a processor.
  • a computer program product including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any of the above aspects is implemented.
  • a computer program is further provided.
  • the computer program is executed by a processor, the method described in any of the above aspects is implemented.
  • a computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the methods in the above embodiments.
  • At least two images are acquired, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain image features of any one of the images; based on the extracted image features, the target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are image registered to obtain image registration results, thereby achieving the purpose of improving the accuracy of image registration. It is easy to notice that feature extraction can be performed on the at least two images respectively to obtain image features containing global features and local features, and the global features and local features can be used to obtain the image features.
  • the local features can expand the receptive field of the image in order to obtain a more accurate target displacement vector field. At least two images can be registered according to the target displacement vector field to obtain an image registration result with higher accuracy. This can be applied to scenes that process complex registration tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.
  • FIG1 is a schematic diagram of a hardware environment of a virtual reality device according to an image registration method according to an embodiment of the present disclosure
  • FIG2 is a structural block diagram of a computing environment of an image registration method according to an embodiment of the present disclosure
  • FIG3 is a flow chart of an image registration method according to Embodiment 1 of the present disclosure.
  • FIG4a is a structural diagram of an image registration process according to an embodiment of the present disclosure.
  • FIG4b is a structural diagram of a convex optimization process according to an embodiment of the present disclosure.
  • FIG5 is a flow chart of an image configuration method according to Embodiment 2 of the present disclosure.
  • FIG6 is a flow chart of an image configuration method according to Embodiment 3 of the present disclosure.
  • FIG7 is a flow chart of an image registration method according to Embodiment 4 of the present disclosure.
  • FIG8 is a flow chart of an image registration method according to Embodiment 5 of the present disclosure.
  • FIG9 is a schematic diagram of an image registration device according to Embodiment 6 of the present disclosure.
  • FIG10 is a schematic diagram of an image registration device according to Embodiment 7 of the present disclosure.
  • FIG11 is a schematic diagram of an image registration device according to Embodiment 8 of the present disclosure.
  • FIG12 is a schematic diagram of an image registration device according to Embodiment 9 of the present disclosure.
  • FIG13 is a schematic diagram of an image registration device according to Embodiment 10 of the present disclosure.
  • FIG14 is a structural block diagram of a computer terminal according to an embodiment of the present disclosure.
  • FIG. 15 is a structural block diagram of a processor according to an embodiment of the present disclosure.
  • Image Registration refers to the matching of geographic coordinates of different image graphics obtained by different imaging methods in the same area.
  • Image registration is a typical problem and technical difficulty in the field of image processing research. Its purpose is to compare or fuse images acquired under different conditions for the same object. For example, images may come from different acquisition devices, taken at different times, different shooting angles, etc. Sometimes, image registration problems for different objects are also required. Specifically, for two images in a set of image data sets, a spatial transformation is found to map one image to another image so that the points corresponding to the same position in the space in the two images correspond one to one.
  • Cost volume It is a common intermediate representation in optical flow estimation or binocular matching, which is used to represent the pixel-level similarity between feature maps to find the corresponding relationship.
  • Convex optimization also called convex optimization, convex minimization, is a subfield of mathematical optimization. It studies the problem of minimizing convex functions defined in convex sets. In a sense, convex optimization is simpler than general mathematical optimization problems. For example, in convex optimization, the local optimal value is the global optimal value.
  • deformable image registration is a basic medical image analysis task, which is traditionally regarded as a continuous update problem of the dense displacement field space between image pairs.
  • image registration is inefficient.
  • a method based on learning deep network prediction of displacement field is proposed to iterate in order to improve the iteration efficiency.
  • image registration methods are difficult to adapt to a variety of application scenarios.
  • Registration has also been formulated as a discrete update problem, which can be handled using a set of dense discrete displacements as the cost volume.
  • the main challenge of this type of approach is the huge size of the search space, since there are millions of voxels in a typical 3D computed tomography (CT) scan, and each voxel in a moving scan can be reasonably paired with thousands of points in other scans, resulting in a high computational burden.
  • CT computed tomography
  • the search space can be pruned by constructing the cost volume in the neighborhood of different voxels.
  • the range of deformation amplitudes that can be solved by this approach is limited by the size of the neighborhood window, and relies on accurate pre-alignment. The implementation process is more difficult.
  • Widely used deformable registration methods with relatively good performance may include: Deeds, ConvexAdam, among which Deeds and ConvexAdam use feature descriptors that provide modality and contrast invariant information, but still represent local information without containing global semantic information. Therefore, it is still difficult to apply in environments with large deformations or complex anatomical differences (for example, the abdomen between patients); in addition, ConvexAdam relies on precise pre-alignment, which is difficult to implement.
  • Handling complex registration tasks relies on unique features that are robust to inter-subject variation, organ deformation, contrast agent injection, and pathological changes, as well as global/contextual information that is beneficial to improving the accuracy of complex deformation registration.
  • the present disclosure uses features embedded in a self-supervised anatomical manner to encode global and local embeddings. In the process of calculating cost loss within a neighborhood, the neighborhood size is limited.
  • the present disclosure proposes to implement the image processing process from small-resolution images to large-resolution images through a pyramid model. Different levels have a smaller search range, rather than a large search range at one resolution. This is beneficial to improving efficiency such as low computational burden and fast running time, expanding the receptive field, and improving registration accuracy.
  • an image registration method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
  • Fig. 1 is a schematic diagram of the hardware environment of a virtual reality device according to an image registration method of an embodiment of the present disclosure.
  • a virtual reality device 104 is connected to a terminal 106, and the terminal 106 is connected to a server 102 through a network.
  • the virtual reality device 104 is not limited to: a virtual reality helmet, virtual reality glasses, a virtual reality all-in-one machine, etc.
  • the terminal 104 is not limited to a PC, a mobile phone, a tablet computer, etc.
  • the server 102 can be a server corresponding to a media file operator, and the network includes but is not limited to: a wide area network, a metropolitan area network or a local area network.
  • the virtual reality device 104 of this embodiment includes: a memory, a processor, and a transmission device.
  • the memory is used to store an application program, which can be used to execute: obtaining at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result, thereby solving the technical problems of low accuracy and limited application scenarios of the image registration method in the related art.
  • the terminal of this embodiment can be used to display the image registration result on the presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, and send the image registration result to the virtual reality device 104. After receiving the image registration result, the virtual reality device 104 displays it at the target delivery position.
  • VR virtual reality
  • AR augmented reality
  • the virtual reality device 104 of this embodiment has an eye-tracking head mounted display (Head Mount Display, referred to as HMD) and an eye tracking module, which have the same functions as those in the above-mentioned embodiment, that is, the screen in the HMD is used to display real-time images, and the eye tracking module in the HMD is used to obtain the real-time movement trajectory of the user's eyeballs.
  • the terminal of this embodiment obtains the user's position information and movement information in the real three-dimensional space through the tracking system, and calculates the three-dimensional coordinates of the user's head in the virtual three-dimensional space, as well as the user's field of view direction in the virtual three-dimensional space.
  • FIG2 shows an embodiment of using the AR/VR device (or mobile device) shown in FIG1 as a computing node in the computing environment 201 in a block diagram.
  • FIG2 is a structural block diagram of a computing environment of an image registration method according to an embodiment of the present disclosure. As shown in FIG2, the computing environment 201 includes multiple (210-1, 210-2, ..., shown in the figure) computing nodes (such as servers) running on a distributed network.
  • Different computing nodes contain local processing and memory resources, and the terminal user 202 can remotely run applications or store data in the computing environment 201.
  • the application can be provided as multiple services 220-1, 220-2, 220-3 and 220-4 in the computing environment 201, representing services "A”, “D”, “E” and "H” respectively.
  • the end user 202 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 202's provision and/or request can be provided to the entry gateway 230.
  • the entry gateway 230 can include a corresponding agent to handle the provision and/or request for the service (one or more services provided in the computing environment 201).
  • Services are provided or deployed based on various virtualization technologies supported by the computing environment 201.
  • services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and/or similar methods.
  • Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine to execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.
  • OS operating system
  • a Pod e.g., a Kubernetes Pod
  • service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods).
  • the Pod may include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers).
  • One or more containers in the Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc.
  • Other services may also be equipped with similar Pods.
  • executing a user request from end user 202 may require invoking one or more services in computing environment 201, and executing one or more functions of one service may require invoking one or more functions of another service.
  • service “A” 220-1 receives a user request from end user 202 from ingress gateway 230
  • Service “A” 220-1 may call service “D” 220-2
  • service “D” 220-2 may request service "E” 220-3 to perform one or more functions.
  • the computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers.
  • the computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.
  • FIG3 is a flow chart of the image registration method according to Embodiment 1 of the present disclosure. As shown in FIG3, the method may include the following steps:
  • Step S302 acquiring at least two images.
  • any image contains the shooting results of the object to be analyzed under different conditions.
  • the at least two images mentioned above may be CT images belonging to the medical field, or images of other fields, and no limitation is imposed on the types of images.
  • the object to be analyzed contained in the above-mentioned images may be the part of the two images that needs to be compared.
  • the above-mentioned at least two images may be abdominal CT images of a patient, and the object to be analyzed may be an abnormal part of the abdomen, for example, a lesion in the patient's abdomen.
  • the object to be analyzed is not limited here, and the object to be analyzed may be determined according to the analysis requirements of the image, and this is only an example.
  • the above-mentioned different conditions may be different time periods, different environments, different types of shooting equipment, and different shooting angles.
  • the different conditions here may be set according to actual scenarios.
  • the object to be analyzed or the area where the object to be analyzed is located can be photographed at different time periods to obtain at least two images; the object to be analyzed or the area where the object to be analyzed is located can also be photographed using different shooting methods to obtain at least two images; the object to be analyzed or the area where the object to be analyzed is located can also be photographed under other different conditions to obtain at least two images.
  • the image registration is performed by acquiring two images as an example, but it is not limited to this.
  • the processing method for acquiring other numbers of images is similar and will not be described in detail here.
  • Step S304 extract features from at least two images respectively to obtain image features of any one of the images.
  • the image features include at least global features and local features.
  • the image feature of any one of the above images may be the image feature of each of the at least two images.
  • feature extraction can be performed on at least two images by a feature extractor to obtain image features of each image.
  • the feature extractor can be a self-supervised anatomical embedding feature extractor (Self-supervised Autoencoder with Mosaicked Training, referred to as SAM), but is not limited to this. It can be other types of feature extractors.
  • SAM is usually used to extract features from images.
  • the algorithm uses a self-supervised learning method to train an anatomical embedding model through self-learning.
  • the model can learn the global and local semantic features of different points on the image, so the image features of any image can be obtained.
  • Image features include at least global features and local features.
  • Step S306 determining the target displacement vector field of the object to be analyzed based on the extracted image features.
  • the above-mentioned target displacement vector field can represent the spatial change of the object to be analyzed in at least two images.
  • the displacement vector field can be estimated by the position of the object to be analyzed.
  • the vectors in the target displacement vector field are used to represent the displacement direction and size of the object to be analyzed in the image.
  • displacement is a vector whose length is the shortest distance from the initial position to the final position of a point undergoing motion, which quantifies the distance and direction of the net or total motion of the point trajectory along a straight line from the initial position to the final position.
  • the displacement vector field can be described as a relative position, that is, as the final position of a point relative to its initial position, and can be defined as the difference between the final position and the initial position.
  • a cost volume can be constructed based on the extracted image features, and the target displacement vector field can be determined based on the cost volume, wherein the cost volume is a common intermediate representation in optical flow estimation or binocular matching, which can be used to represent the pixel-level similarity between features to find the correspondence between pixels, and the target displacement vector field of the object to be analyzed can be determined based on the correspondence.
  • the above-mentioned cost volume may be a 6D cost volume, but is not limited thereto, wherein the 6D cost volume stores a data matching cost for associating a pixel with its corresponding pixel in another image.
  • the cost volume (cost volume or correlation volume) is a common intermediate representation in optical flow estimation or binocular matching, which is used to represent the pixel-level similarity between feature maps, so as to find the correspondence between pixels based on the similarity.
  • the dense cost volume is calculated.
  • its complexity is the square of the number of pixels (H ⁇ W ⁇ H ⁇ W); in the 3D medical image registration problem, its complexity is the square of the number of pixels (H ⁇ W ⁇ D ⁇ H ⁇ W ⁇ D), where the size of the 3D image is H ⁇ W ⁇ D.
  • Constructing 6D cost volume based on SAM features is a method of constructing 6D cost volume by features. It uses the features of two images to construct 6D cost volume to measure the similarity of two 3D objects. By extracting features from the image using SAM, the corresponding 6D cost volume can be obtained.
  • a convex optimization process may be performed based on the cost volume to obtain the target displacement vector field, wherein the convex optimization process refers to a type of optimization problem in which the target function is a convex function.
  • Step S308 performing image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.
  • image registration refers to matching the geographic coordinates of different images acquired by different imaging methods in the same area.
  • image registration is to map one image to another image by finding a spatial transformation so that points at the same spatial position in the two images correspond one to one.
  • any one of the at least two images can be mapped to another image according to the target displacement vector field of the object to be analyzed, so that changes in the object to be analyzed can be observed in one image, thereby facilitating the user to analyze the object to be analyzed and improving the efficiency of the analysis.
  • the object to be analyzed may be the abdominal area of a patient, and the At least two CT images are used to determine the target displacement vector field of the abdominal area.
  • the two CT images can be registered based on the target displacement vector field to obtain an image registration result, which is convenient for doctors to analyze problems in the patient's abdominal area based on the image registration result.
  • any one of the images contains the shooting results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain the image features of any one of the images; based on the extracted image features, the target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are image registered to obtain the image registration result, thereby achieving the purpose of improving the accuracy of image registration.
  • feature extraction can be performed on at least two images respectively to obtain image features containing global features and local features
  • the receptive field of the image can be expanded according to the global features and the local features to obtain a more accurate target displacement vector field
  • at least two images can be registered according to the target displacement vector field to obtain an image registration result with higher accuracy, which can be applied to the scene of processing complex registration tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.
  • feature extraction is performed on at least two images respectively to obtain image features of any one of the images, including: based on the displacement vector field corresponding to the first image of the at least two images, the first image is deformed to obtain a deformed image; feature extraction is performed on the deformed image and the second image of the at least two images respectively to obtain image features of any one of the images, wherein the second image is used to characterize the image other than the first image of the at least two images.
  • the above-mentioned first image and second image may be images of the same area captured at different time points.
  • the above-mentioned first image may be the image captured first, and the second image may be the image captured later.
  • the above-mentioned second image may be the image captured first, and the above-mentioned first image may be the image captured later. No limitation is made to the first image and the second image here.
  • the first image may also be any one of the at least two images, and the second image may be another image of the at least two images except the any one of the images.
  • the first image may be a source image of the two images, and the second image may be a target image.
  • the above displacement vector field can also be called the up-sampled field.
  • the displacement vector field corresponding to the above-mentioned first image can be determined according to the scale information of the first image.
  • the scale information of the first image can be divided into multiple levels. The higher the level, the greater the resolution of the image, and the lower the level, the smaller the resolution of the image. Therefore, if the scale of the first image is the smallest, that is, the lowest level, the displacement vector field can be a preset vector field. If the scale of the first image is not the smallest scale, that is, not the lowest level, the displacement vector field can be a target displacement vector field determined by the image of the previous level.
  • the above content can be described according to three levels, the first level is the level with the smallest image scale, the second level is the level with the second largest image scale, and the third level is the level with the largest image scale.
  • the first image is at the first level, it means that the resolution of the first image is the lowest.
  • the first displacement vector field between the deformed image and the second image of the first level can be directly obtained; when the first image is at the second level, The first image can be deformed based on the displacement vector field obtained after upsampling the first displacement vector field to obtain a deformed image, and the second displacement vector field between the deformed image and the second image of the second level can be obtained; when the first image is at the third level, it means that the resolution of the first image is the largest.
  • the first image can be deformed based on the displacement vector field obtained after upsampling the second displacement vector field to obtain a deformed image, and the third displacement vector field between the deformed image and the second image of the maximum level can be obtained.
  • feature extraction may be performed on the deformed image and the second image obtained at any level to obtain image features of any image, and feature extraction may be performed on the deformed image and the second image obtained at the above three levels to obtain image features of any image.
  • feature extraction is performed on the deformed image and the second image of at least two images respectively to obtain image features of any one of the images, including: feature extraction is performed on the deformed image and the second image to obtain global features and local features; and global features and local features are superimposed to obtain image features.
  • the global features and local features in the deformed image and the second image may be captured simultaneously by a feature extractor, and the global features and local features may be further superimposed to obtain image features of any image.
  • the target displacement vector field of the object to be analyzed is determined, including: obtaining the dot product of the extracted image features to obtain the target cost volume; performing convex optimization processing on the target cost volume to obtain the target displacement vector field.
  • the target cost volume mentioned above is used to find the correspondence between image features according to the similarity of the image features.
  • the dot product of the extracted image features may be obtained to measure the similarity between the first image and the second image, and a target cost volume may be constructed according to the similarity between the two images, and a convex optimization process may be performed on the target cost volume to obtain a target displacement vector field.
  • the target cost volume may be a 6D cost volume.
  • a target cost volume is subjected to convex optimization processing to obtain a target displacement vector field, including: performing parameter minimization processing on the target cost volume to obtain an initial displacement vector field; and performing an average pooling operation on the initial displacement vector field to obtain a target displacement vector field.
  • parameter minimization processing may be performed on the target cost volume, that is, parameter minimization processing may be performed on the target cost volume using Argmin(.) to obtain a variable corresponding to the target cost volume when the output value of Argmin(.) is minimum, and an initial displacement vector field is determined based on the variable.
  • Average pooling operation may be performed on the initial displacement vector field so that the initial displacement vector field may be divided into several regions of the same size, and then the eigenvalues in different regions are averaged to obtain the target displacement vector field.
  • feature extraction is performed on at least two images respectively to obtain image features of any one of the images, including: downsampling the at least two images multiple times respectively to obtain multiple downsampled images of any one of the images, wherein the resolutions of different downsampled images are different; deforming the downsampled image of the first image based on the sampling displacement vector field corresponding to the downsampled image of the first image of the at least two images to obtain a sampled deformed image; and feature extraction is performed on the sampled deformed image and the downsampled image of the second image of the at least two images respectively. Take, and obtain the downsampled image features of the downsampled image of any image; summarize multiple downsampled image features to obtain the image features of any image.
  • At least two images may be downsampled multiple times respectively to obtain representations of the at least two images at different scales, that is, to obtain multiple downsampled images of any image, and the resolutions of different downsampled images are different.
  • the downsampled image of the first image may be deformed according to the sampling displacement vector field corresponding to the downsampled image of the first image to obtain a sampled deformed image.
  • Sampled deformed images of different resolutions may be obtained.
  • Feature extraction may be performed on the sampled deformed image and the downsampled image of the second image respectively to obtain downsampled image features of the downsampled image of any image.
  • Features of multiple downsampled images with different resolutions may be summarized to obtain an image feature pyramid of any image.
  • the downsampled image of the largest scale can be processed to obtain the sampling deformed image of the largest scale.
  • Feature extraction can be performed on the sampling deformed image and the downsampled image of the second image respectively to obtain the downsampled image features of the downsampled image of any image.
  • the sampling displacement vector field required in the image processing process of the next scale can be constructed based on the downsampled image features.
  • a cost volume can be constructed based on the downsampled image features and convex optimization processing can be performed to obtain the sampling displacement vector field required in the image processing process of the next scale.
  • sampling displacement vector fields corresponding to the images of each scale can be obtained cyclically, so as to obtain multiple sampling deformed images in turn according to the multiple sampling displacement vector fields, and multiple downsampled image features can be obtained in turn according to the multiple sampling deformed images.
  • the multiple downsampled image features can be summarized to obtain the image features of any image.
  • the first image is deformed at different scales in a coarse-to-fine manner to obtain a sampled deformed image, and features are extracted from the sampled deformed image and the downsampled image of the second image to obtain features of the downsampled image.
  • the target displacement vector field of the object to be analyzed is determined, including: obtaining the dot product of the downsampled image features of the downsampled image of any image to obtain a sampling cost volume; performing convex optimization processing based on the sampling cost volume to obtain a sampling displacement vector field; and superimposing multiple sampling displacement vector fields to obtain a target displacement vector field.
  • the dot product of the downsampled image features of any image can be obtained to measure the similarity between the downsampled image of the first image and the downsampled image of the second image, and a sampling cost volume can be constructed based on the similarity between the two downsampled images.
  • Convex optimization processing can be performed based on the sampling cost volume to obtain a sampling displacement vector field.
  • the above operation can be performed cyclically on downsampled images of different scales. It should be noted that the sampling displacement vector field obtained at the current scale can be deformed according to the sampling displacement vector field obtained at the previous scale to improve the accuracy of the deformation. After obtaining multiple sampling displacement vector fields at different scales, the multiple sampling displacement vector fields can be superimposed to obtain the target displacement vector field.
  • the method further includes: when the downsampled image of the first image is a preset downsampled image, determining that the sampling displacement vector field corresponding to the downsampled image of the first image is the preset displacement vector field, wherein the preset downsampled image is used to represent the downsampled image of the maximum scale of the first image; in the downsampled image of the first image When the sampled image is not a preset down-sampled image, up-sampling is performed on the sampling displacement vector field corresponding to the down-sampled image of the next scale of the first image to obtain the sampling displacement vector field corresponding to the down-sampled image of the first image.
  • the above-mentioned preset downsampled image can be a downsampled image at the maximum scale. Since the resolution of the downsampled image at the maximum scale is low, the number of features that need to be processed is small. Therefore, the global features of the downsampled image can be efficiently extracted, and the downsampled images can be processed in sequence from large resolution to small resolution. While ensuring that global features and local features can be extracted, the feature extraction efficiency can be further improved.
  • the pre-set preset displacement vector field can be determined as the sampling displacement vector field corresponding to the downsampled image of the first image; after obtaining the downsampled image features according to the downsampled image of the maximum scale, the dot product of the downsampled image features can be obtained to obtain the sampling cost volume, and convex optimization processing can be performed based on the sampling cost volume to obtain the sampling displacement vector field required for the deformation of the downsampled image of the next scale. Based on this, downsampled images of different scales can be processed in sequence to obtain sampling displacement vector fields obtained at different scales.
  • FIG4a is a structural diagram of an image registration process according to an embodiment of the present disclosure.
  • the images to be registered include a first image IS and a second image IT.
  • the first image and the second image can be downsampled respectively to obtain an image pyramid (Pyramid Source) of the first image and an image pyramid (Pyramid Target) of the second image.
  • the top images of the two sampling pyramids are the downsampled images with the largest scale, and the bottom images of the sampling pyramids are the downsampled images with the smallest scale.
  • the processing flow of downsampled images at each scale is similar.
  • the processing flow of downsampled images at the smallest scale is used as an example for explanation: when performing feature processing on the downsampled image, the registration field obtained at the previous scale can be upsampled (Up-Sampled Field), that is, the above-mentioned sampling displacement vector field is used to perform a deformation operation (Warping) on the downsampled image corresponding to the first image to obtain a sampled deformed image, and feature extraction can be performed on the sampled deformed image and the downsampled image of the second image to obtain the downsampled image features, and a convex optimization operation (Convex Optimization) can be performed on the downsampled image features to obtain the sampling displacement vector field, that is, the registration field of the current scale, and finally, multiple sampling displacement vector fields can be superimposed to obtain a target displacement vector field, and the first image and the second image can be image registered according to the target displacement vector field to obtain an image registration result.
  • Up-Sampled Field Up-Sampled Field
  • Figure 4b is a structural diagram of a convex optimization processing process according to an embodiment of the present disclosure.
  • the local features and global features of the first image can be superimposed to obtain image features of the first image and the second image.
  • the image features can be subjected to a dot product operation (Dot Product) to obtain a target cost volume.
  • Dot Product dot product
  • the preset function [Argmin(.)] can be used to determine the value of the minimum independent variable of the function of the target cost volume to obtain an initial displacement vector field, that is, the first registration field u shown in the figure.
  • the initial displacement vector field can be subjected to an average pooling operation (Mean Conv) to obtain a target displacement vector field, that is, the second registration field ⁇ shown in the figure.
  • Mean Conv average pooling operation
  • the image registration method disclosed in the present invention provides a self-supervised anatomical embedding network (SAMConvex), which can be quickly used for a discrete optimization method for image registration, including a decoupled convex optimization program, through which a displacement vector field based on a self-supervised anatomical embedding feature extractor is obtained.
  • the feature extractor can capture global features and local features at the same time.
  • SAMConvex can extract features of different scales and construct a 6D cost volume based on the features.
  • the displacement vector field can be iteratively updated by using a processing process from large scale to small scale.
  • 6D cost volumes of different levels are calculated by taking inputs of different resolutions, and the cost volume of each level is calculated by taking inputs of different resolutions.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware.
  • the technical solution of the present disclosure, or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present disclosure.
  • a storage medium such as ROM/RAM, a disk, or an optical disk
  • a terminal device which can be a mobile phone, a computer, a server, or a network device, etc.
  • an image registration method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
  • FIG5 is a flow chart of an image configuration method according to Embodiment 2 of the present disclosure. As shown in FIG5 , the method includes the following steps:
  • Step S502 acquiring at least two medical images.
  • any medical image contains the scanning results of the target part of the same biological object under different conditions using computer tomography technology.
  • the above-mentioned medical images include but are not limited to CT images and patient skin images, which are not limited here. Set the type of medical image according to actual situation.
  • Step S504 extract features from at least two medical images respectively to obtain image features of any one of the images.
  • image features include: global features and local features.
  • Step S506 determining a target displacement vector field of the target part based on the extracted image features.
  • Step S508 performing image registration on at least two medical images based on the target displacement vector field of the target part to obtain an image registration result.
  • any one of the medical images contains the scanning results of the target part of the same biological object under different conditions by computer tomography technology; feature extraction is performed on the at least two medical images respectively to obtain the image features of any one of the images, wherein the image features include: global features and local features; based on the extracted image features, the target displacement vector field of the target part is determined; based on the target displacement vector field of the target part, at least two medical images are image registered to obtain the image registration result, thereby achieving the purpose of improving the accuracy of image registration.
  • feature extraction can be performed on at least two images respectively to obtain image features containing global features and local features
  • the receptive field of the image can be expanded according to the global features and the local features to obtain a more accurate target displacement vector field
  • at least two images can be registered according to the target displacement vector field to obtain an image registration result with higher accuracy, which can be applied to the scene of processing complex registration tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.
  • an image registration method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
  • FIG6 is a flow chart of an image configuration method according to Embodiment 3 of the present disclosure. As shown in FIG6 , the method includes the following steps:
  • Step S602 In response to an input instruction on the operation interface, at least two images are displayed on the operation interface.
  • any image contains the shooting results of the object to be analyzed under different conditions.
  • the above-mentioned operation interface may be a display interface capable of displaying at least two images, and at least two images may be displayed by performing relevant touch operations on the display interface.
  • Step S604 responding to the image registration instruction on the operation interface, and displaying the image registration result on the operation interface.
  • the image registration result is obtained by performing image registration on at least two images based on the target displacement vector field of the object to be analyzed, wherein the target displacement vector field is determined based on the extracted image features.
  • Features include: global features and local features.
  • the extracted image features are obtained by extracting features from at least two images respectively.
  • the above-mentioned image registration instruction may be an instruction generated by touching a related control of the operation interface when at least two images need to be registered, and the image registration result may be displayed on the operation interface according to the image registration instruction.
  • the image registration result is displayed on the operation interface, wherein the image registration result is obtained by performing image registration on at least two images based on the target displacement vector field of the object to be analyzed, and the target displacement vector field is determined based on the extracted image features, and the extracted image features include: global features and local features, and the extracted image features are obtained by performing feature extraction on at least two images respectively, thereby achieving the purpose of improving the accuracy of image registration.
  • feature extraction can be performed on at least two images respectively to obtain image features containing global features and local features, and the receptive field of the image can be expanded according to the global features and the local features to obtain a more accurate target displacement vector field, and at least two images can be registered according to the target displacement vector field, thereby obtaining an image registration result with higher accuracy, which can be applied to the scene of processing complex registration tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.
  • an image registration method that can be applied to virtual reality scenarios such as virtual reality VR devices and augmented reality AR devices.
  • steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
  • FIG7 is a flow chart of an image registration method according to Embodiment 4 of the present disclosure. As shown in FIG7 , the method may include the following steps:
  • Step S702 display at least two images on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device.
  • VR virtual reality
  • AR augmented reality
  • any image contains the shooting results of the object to be analyzed under different conditions.
  • Step S704 extract features from at least two images respectively to obtain image features of any one of the images.
  • the image features include at least global features and local features.
  • Step S706 determining the target displacement vector field of the object to be analyzed based on the extracted image features.
  • Step S708 performing image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.
  • Step S710 drive the VR device or AR device to render and display the image registration result.
  • At least two images are displayed on the presentation screen of a virtual reality VR device or an augmented reality AR device, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain the image features of any one of the images; based on the extracted image features, the target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are image registered to obtain the image registration result; the VR device or AR device is driven to render and display the image registration result, thereby achieving the purpose of improving the accuracy of image registration.
  • feature extraction can be performed on at least two images respectively to obtain image features containing global features and local features
  • the receptive field of the image can be expanded according to the global features and the local features to obtain a more accurate target displacement vector field
  • at least two images can be registered according to the target displacement vector field to obtain an image registration result with higher accuracy, which can be applied to the scene of processing complex registration tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.
  • the above image registration method can be applied to a hardware environment composed of a server and a virtual reality device.
  • the image registration result is displayed on the presentation screen of the virtual reality VR device or the augmented reality AR device.
  • the server can be a server corresponding to the media file operator.
  • the above network includes but is not limited to: a wide area network, a metropolitan area network or a local area network.
  • the above virtual reality device is not limited to: a virtual reality helmet, virtual reality glasses, a virtual reality all-in-one machine, etc.
  • the virtual reality device includes: a memory, a processor, and a transmission device.
  • the memory is used to store an application, which can be used to execute: displaying at least two images on a presentation screen of a virtual reality VR device or an augmented reality AR device, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images, wherein the image features at least include: global features and local features; determining the target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; and driving the VR device or AR device to render and display the image registration result.
  • the above-mentioned image registration method applied in a VR device or an AR device of this embodiment may include the method of the embodiment shown in FIG. 7 , so as to achieve the purpose of driving the VR device or the AR device and displaying the image registration result.
  • the processor of this embodiment can call the application stored in the memory to execute the above steps through a transmission device.
  • the transmission device can receive media files sent by the server through the network, and can also be used for data transmission between the processor and the memory.
  • a target tracking method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown can be executed in an order different from that shown here. or described steps.
  • FIG8 is a flow chart of an image registration method according to Embodiment 5 of the present disclosure. As shown in FIG8 , the method includes the following steps:
  • Step S802 acquiring at least two images by calling the first interface.
  • the first interface includes a first parameter, the parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions.
  • the first interface in the above steps may be an interface for data interaction between the cloud server and the client.
  • the client may pass at least two images into the interface function as the first parameter of the interface function to achieve the purpose of uploading at least two images to the cloud server.
  • Step S804 extract features from at least two images respectively to obtain image features of any one of the images.
  • the image features include at least global features and local features.
  • Step S806 determining the target displacement vector field of the object to be analyzed based on the extracted image features.
  • Step S808 performing image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.
  • Step S810 output the image registration result by calling the second interface.
  • the second interface includes a second parameter, and a parameter value of the second parameter is an image registration result.
  • the above-mentioned second interface can be an interface for data interaction between the cloud server and the client.
  • the cloud server can pass the image registration result to the interface function as the second parameter of the interface function to achieve the purpose of sending the image registration result to the client.
  • At least two images are acquired by calling the first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter is at least two images, any one of which contains the shooting results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain image features of any one of the images; based on the extracted image features, the target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are image registered to obtain image registration results; by calling the second interface, the image registration results are output, wherein the second interface includes a second parameter, the parameter value of the second parameter is the image registration result, and the purpose of improving the accuracy of image registration is achieved.
  • feature extraction can be performed on at least two images respectively to obtain image features containing global features and local features
  • the receptive field of the image can be expanded according to the global features and the local features, so as to obtain a more accurate target displacement vector field
  • at least two images can be registered according to the target displacement vector field, so as to obtain an image registration result with higher accuracy, which can be applied to the scene of processing complex registration tasks, thereby solving the technical problems of low accuracy and limited application scenarios of image registration methods in related technologies.
  • FIG9 is a schematic diagram of an image registration device according to Embodiment 6 of the present disclosure.
  • the device 900 includes: an acquisition module 902 , an extraction module 904 , a determination module 906 , and a registration module 908 .
  • the acquisition module is configured to acquire at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions;
  • the extraction module is configured to perform feature extraction on at least two images respectively to obtain the image features of any one of the images;
  • the determination module is configured to determine the target displacement vector field of the object to be analyzed based on the extracted image features;
  • the registration module is configured to perform image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain the image registration result.
  • the acquisition module 902, extraction module 904, determination module 906, and registration module 908 correspond to steps S302 to S308 in Example 1, and the four modules and corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1.
  • the above-mentioned modules or units may be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n), and the above-mentioned modules may also be part of the device and may be run in the computer terminal 1400A provided in Example 1.
  • the extraction module is also configured to deform the first image based on the displacement vector field corresponding to the first image of at least two images to obtain a deformed image; and perform feature extraction on the deformed image and the second image of at least two images respectively to obtain image features of any one of the images, wherein the second image is used to characterize the image other than the first image of at least two images.
  • the extraction module is further configured to extract features from the deformed image and the second image to obtain global features and local features; and to superimpose the global features and the local features to obtain image features.
  • the determination module is further configured to obtain the dot product of the extracted image features to obtain the target cost volume; and perform convex optimization processing on the target cost volume to obtain the target displacement vector field.
  • the determination module is further configured to perform parameter minimization processing on the target cost volume to obtain an initial displacement vector field; and perform an average pooling operation on the initial displacement vector field to obtain a target displacement vector field.
  • the extraction module is also configured to perform multiple downsampling on at least two images respectively to obtain multiple downsampled images of any one image, wherein the resolutions of different downsampled images are different; based on the sampling displacement vector field corresponding to the downsampled image of the first image among the at least two images, the downsampled image of the first image is deformed to obtain a sampled deformed image; feature extraction is performed on the sampled deformed image and the downsampled image of the second image among the at least two images respectively to obtain downsampled image features of the downsampled image of any one image; and multiple downsampled image features are summarized to obtain image features of any one image.
  • the determination module is also configured to obtain the dot product of the downsampled image features of the downsampled image of any image to obtain a sampling cost volume; perform convex optimization processing based on the sampling cost volume to obtain a sampling displacement vector field; and superimpose multiple sampling displacement vector fields to obtain a target displacement vector field.
  • the device is also configured to determine that the sampling displacement vector field corresponding to the downsampled image of the first image is the preset displacement vector field when the downsampled image of the first image is the preset downsampled image, wherein the preset downsampled image is used to represent the downsampled image of the maximum scale of the first image;
  • the sampled image is not a preset down-sampled image
  • up-sampling is performed on the sampling displacement vector field corresponding to the down-sampled image of the next scale of the first image to obtain the sampling displacement vector field corresponding to the down-sampled image of the first image.
  • FIG. 10 is a schematic diagram of an image registration device according to Example 7 of the present disclosure.
  • the device 1000 includes: an acquisition module 1002, an extraction module 1004, a determination module 1006, and a registration module 1008.
  • the acquisition module is configured to acquire at least two medical images, wherein any one of the medical images contains the scan results of the target part of the same biological object under different conditions by means of computer tomography technology;
  • the extraction module is configured to perform feature extraction on the at least two medical images respectively to obtain image features of any one of the images, wherein the image features include: global features and local features;
  • the determination module is configured to determine the target displacement vector field of the target part based on the extracted image features; and the registration module is configured to determine the target displacement vector field of the target part.
  • Image registration is performed on the at least two medical images to obtain image registration results.
  • the acquisition module 1002, extraction module 1004, determination module 1006, and registration module 1008 correspond to steps S502 to S508 in Example 2, and the four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1.
  • the above-mentioned modules or units may be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n), and the above-mentioned modules may also be part of the device and may be run in the computer terminal 1400A provided in Example 1.
  • FIG 11 is a schematic diagram of an image registration device according to embodiment 8 of the present disclosure. As shown in Figure 11, the device 1100 includes: a first display module 1102 and a second display module 1104.
  • the first display module is configured to respond to an input instruction acting on an operation interface, and display at least two images on the operation interface, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions;
  • the second display module is configured to respond to an image registration instruction acting on the operation interface, and display the image registration result on the operation interface, wherein the image registration result is obtained by performing image registration on at least two images based on a target displacement vector field of the object to be analyzed, and the target displacement vector field is determined based on extracted image features, and the extracted image features include: global features and local features, and the extracted image features are obtained by performing feature extraction on at least two images respectively.
  • first display module 1102 and the second display module 1104 correspond to steps S602 to S604 in Embodiment 3, and the examples and application scenarios implemented by the two modules and the corresponding steps are the same. However, it is not limited to the contents disclosed in the above-mentioned embodiment 1.
  • the above-mentioned modules or units may be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n), and the above-mentioned modules may also be part of the device and may be run in the computer terminal 1400A provided in embodiment 1.
  • FIG. 12 is a schematic diagram of an image registration device according to Example 9 of the present disclosure.
  • the device 1200 includes: a display module 1202, an extraction module 1204, a determination module 1206, a registration module 1208, and a driving module 1210.
  • the display module is configured to display at least two images on a presentation screen of a virtual reality VR device or an augmented reality AR device, wherein any one of the images contains shooting results of the object to be analyzed under different conditions;
  • the extraction module is configured to perform feature extraction on at least two images respectively to obtain image features of any one of the images;
  • the determination module is configured to determine the target displacement vector field of the object to be analyzed based on the extracted image features;
  • the registration module is configured to perform image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result;
  • the driving module is used to drive the VR device or AR device to render and display the image registration result.
  • the above-mentioned display module 1202, extraction module 1204, determination module 1206, registration module 1208, and driving module 1210 correspond to steps S702 to S710 in Example 4, and the five modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1.
  • the above-mentioned modules or units may be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n), and the above-mentioned modules may also be part of the device and may be run in the computer terminal 1400A provided in Example 1.
  • FIG. 13 is a schematic diagram of an image registration device according to embodiment 10 of the present disclosure. As shown in Figure 13, the device 1300 includes: an acquisition module 1302, an extraction module 1304, a determination module 1306, a registration module 1308, and a calling module 1310.
  • the acquisition module is configured to acquire at least two images by calling a first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions;
  • the extraction module is configured to perform feature extraction on the at least two images respectively to obtain the image features of any one of the images;
  • the determination module is configured to determine the image features of the object to be analyzed based on the extracted image features.
  • the target displacement vector field of the object to be analyzed is determined; the registration module is used to perform image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; the calling module is configured to output the image registration result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the image registration result.
  • the acquisition module 1302, extraction module 1304, determination module 1306, registration module 1308, and calling module 1310 correspond to steps S802 to S810 in Example 5, and the five modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1.
  • the above-mentioned modules or units may be hardware components or software components stored in a memory (e.g., memory 1402) and processed by one or more processors (e.g., processors 1401a, 1401b, ..., 1401n), and the above-mentioned modules may also be part of the device and may be run in the computer terminal 10 provided in Example 1.
  • the embodiments of the present disclosure may provide an electronic device, which may be an AR/VR device, and the AR/VR device may be any AR/VR device in an AR/VR device group.
  • the AR/VR device may also be replaced by a terminal device such as a mobile terminal.
  • the AR/VR device may be located in at least one network device among a plurality of network devices of a computer network.
  • the above-mentioned AR/VR device can execute the program code of the following steps in the image registration method: obtaining at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain the image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; and performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.
  • Figure 14 is a block diagram of a computer terminal according to an embodiment of the present disclosure.
  • the computer terminal 1400A may include: one or more (only one is shown in the figure) processors 1401, a memory 1402, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.
  • the memory can be used to store software programs and modules, such as the program instructions/modules corresponding to the image registration method and device in the embodiment of the present disclosure.
  • the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image registration method.
  • the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • FIG. 15 shows a structural block diagram of a processor for implementing the embodiment of the present disclosure.
  • the processor 1500 is configured to run a program, wherein the program executes the method in the above embodiment when the processor runs it.
  • the processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; perform feature extraction on the at least two images respectively to obtain the image features of any one of the images; determine the target displacement vector field of the object to be analyzed based on the extracted image features; and perform image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.
  • the processor may also execute program code of the following steps: deforming the first image based on the displacement vector field corresponding to the first image of at least two images to obtain a deformed image; performing feature extraction on the deformed image and the second image of at least two images respectively to obtain image features of any one of the images, wherein the second image is used to characterize the image other than the first image of at least two images.
  • the processor may also execute program codes of the following steps: extracting features from the deformed image and the second image to obtain global features and local features; and superimposing the global features and the local features to obtain image features.
  • the processor may also execute program codes of the following steps: obtaining the dot product of the extracted image features to obtain a target cost volume; and performing convex optimization processing on the target cost volume to obtain a target displacement vector field.
  • the processor may also execute program codes of the following steps: performing parameter minimization processing on the target cost volume to obtain an initial displacement vector field; performing an average pooling operation on the initial displacement vector field to obtain a target displacement vector field.
  • the processor may also execute program codes of the following steps: downsampling at least two images multiple times respectively to obtain multiple downsampled images of any one image, wherein different downsampled images have different resolutions; deforming the downsampled image of the first image based on the sampling displacement vector field corresponding to the downsampled image of the first image of at least two images to obtain a sampled deformed image; extracting features from the sampled deformed image and the downsampled image of the second image of at least two images respectively to obtain downsampled image features of the downsampled image of any one image; and summarizing multiple downsampled image features to obtain image features of any one image.
  • the processor may also execute the program code of the following steps: obtaining the dot product of the downsampled image features of any downsampled image to obtain a sampling cost volume; performing convex optimization processing based on the sampling cost volume to obtain a sampling displacement vector field; and superimposing multiple sampling displacement vector fields to obtain a target displacement vector field.
  • the processor may also execute the program code of the following steps: when the downsampled image of the first image is a preset downsampled image, determining that the sampling displacement vector field corresponding to the downsampled image of the first image is a preset displacement vector field, wherein the preset downsampled image is used to represent the downsampled image of the maximum scale of the first image; when the downsampled image of the first image is not the preset downsampled image, upsampling the sampling displacement vector field corresponding to the downsampled image of the next scale of the first image to obtain the sampling displacement vector field corresponding to the downsampled image of the first image.
  • the processor can call the information and application program stored in the memory through the transmission device to execute the following steps: obtaining at least two medical images, wherein any one of the medical images contains the scanning results of the target part of the same biological object under different conditions through the computer tomography technology; performing feature extraction on the at least two medical images respectively to obtain the image features of any one of the images, wherein the image features include: global features and local features; based on the extracted image features, determining the target displacement vector field of the target part; based on the target displacement vector field of the target part, performing image registration on the at least two medical images to obtain the image registration result.
  • the processor can call the information and application stored in the memory through the transmission device to execute the following steps: responding to the input instruction on the operation interface, displaying at least two images on the operation interface, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; responding to the image registration instruction on the operation interface, displaying the image registration result on the operation interface, wherein the image registration result is obtained by performing image registration on at least two images based on the target displacement vector field of the object to be analyzed, and the target displacement vector field is determined based on the extracted image features, and the extracted image features include: global features and local features, and the extracted image features are obtained by performing feature extraction on at least two images respectively.
  • the processor can call the information and application stored in the memory through the transmission device to execute the following steps: display at least two images on the presentation screen of the virtual reality VR device or the augmented reality AR device, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; perform feature extraction on the at least two images respectively to obtain the image features of any one of the images; determine the target displacement vector field of the object to be analyzed based on the extracted image features; perform image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain the image registration result; drive the VR device or the AR device to render and display the image registration result.
  • the processor can call the information and application stored in the memory through the transmission device to execute the following steps: obtain at least two images by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions; perform feature extraction on the at least two images respectively to obtain the image features of any one of the images; determine the target displacement vector field of the object to be analyzed based on the extracted image features; perform image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain the image registration result; output the image registration result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the image registration result.
  • At least two images are obtained, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain the image features of any one of the images; based on the extracted image features, the target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are image registered to obtain the image registration result, thereby achieving the purpose of improving the accuracy of image registration.
  • feature extraction can be performed on at least two images respectively to obtain image features containing global features and local features
  • the receptive field of the image can be expanded according to the global features and the local features to obtain a more accurate target displacement vector field
  • at least two images can be registered according to the target displacement vector field to obtain an image registration result with higher accuracy, thereby solving the technical problem of low accuracy in processing the registration task in the related art.
  • the structure shown in FIG. 14 is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, and other terminal devices.
  • FIG. 14 does not limit the structure of the above-mentioned electronic device.
  • the computer terminal A may also include more or fewer components (such as a network interface, a display device, etc.) than those shown in FIG. 14, or may have a configuration different from that shown in FIG. 14.
  • a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
  • the embodiment of the present disclosure further provides a computer-readable storage medium.
  • the computer-readable storage medium can be used to store the program code executed by the image registration method provided in the above embodiment 1.
  • Computer storage medium may also be referred to as computer storage medium. It may include a data signal propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable storage media may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device.
  • the program code contained in the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, radio frequency, etc., or any suitable combination of the foregoing.
  • the above-mentioned computer-readable storage medium can be located in any computer terminal in the AR/VR device terminal group in the AR/VR device network, or in any mobile terminal in the mobile terminal group.
  • the computer-readable storage medium is configured to store program codes for executing the following steps: acquiring at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; and performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result.
  • the above-mentioned storage medium is also configured to store program codes for executing the following steps: deforming the first image based on the displacement vector field corresponding to the first image of at least two images to obtain a deformed image; performing feature extraction on the deformed image and the second image of at least two images respectively to obtain image features of any one of the images, wherein the second image is used to characterize the image other than the first image of at least two images.
  • the storage medium is further configured to store program codes for executing the following steps: extracting features from the deformed image and the second image to obtain global features and local features; and superimposing the global features and the local features to obtain image features.
  • the storage medium is further configured to store program codes for executing the following steps: obtaining the dot product of the extracted image features to obtain a target cost volume; and performing convex optimization processing on the target cost volume to obtain a target displacement vector field.
  • the storage medium is further configured to store program codes for executing the following steps: performing parameter minimization processing on the target cost volume to obtain an initial displacement vector field; performing average pooling operation on the initial displacement vector field to obtain a target displacement vector field.
  • the storage medium is also configured to store program codes for executing the following steps: downsampling at least two images multiple times respectively to obtain multiple downsampled images of any one image, wherein different downsampled images have different resolutions; deforming the downsampled image of the first image based on a sampling displacement vector field corresponding to the downsampled image of the first image of at least two images to obtain a sampled deformed image; extracting features from the sampled deformed image and the downsampled image of the second image of at least two images respectively to obtain downsampled image features of the downsampled image of any one image; and summarizing multiple downsampled image features to obtain image features of any one image.
  • the storage medium is also configured to store program codes for executing the following steps: obtaining the dot product of the downsampled image features of any image to obtain a sampling cost volume; performing convex optimization processing based on the sampling cost volume to obtain a sampling displacement vector field; and superimposing multiple sampling displacement vector fields to obtain a target displacement vector field.
  • the storage medium is also configured to store program codes for executing the following steps: when a downsampled image of the first image is a preset downsampled image, determining that a sampling displacement vector field corresponding to the downsampled image of the first image is a preset displacement vector field, wherein the preset downsampled image is used to characterize a downsampled image of the maximum scale of the first image; when the downsampled image of the first image is not a preset downsampled image, upsampling the sampling displacement vector field corresponding to a downsampled image of the next scale of the first image to obtain a sampling displacement vector field corresponding to the downsampled image of the first image.
  • the computer-readable storage medium is configured to store program codes for executing the following steps: acquiring at least two medical images, wherein any one of the medical images contains scanning results of a target part of the same biological object under different conditions by means of computed tomography; performing feature extraction on the at least two medical images respectively to obtain image features of any one of the images, wherein the image features include: global features and local features; determining a target displacement vector field of the target part based on the extracted image features; and performing image registration on the at least two medical images based on the target displacement vector field of the target part to obtain an image registration result.
  • the computer-readable storage medium is configured to store program codes for executing the following steps: in response to an input instruction acting on an operation interface, displaying at least two images on the operation interface, wherein any one of the images contains shooting results of the object to be analyzed under different conditions; in response to an image registration instruction acting on the operation interface, displaying an image registration result on the operation interface, wherein the image registration result is obtained by performing image registration on at least two images based on a target displacement vector field of the object to be analyzed, wherein the target displacement vector field is determined based on extracted image features, and the extracted image features include: global features and local features.
  • the extracted image features are obtained by extracting features from at least two images respectively.
  • the computer-readable storage medium is configured to store program codes for executing the following steps: displaying at least two images on a presentation screen of a virtual reality VR device or an augmented reality AR device, wherein any one of the images contains shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining a target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; and driving the VR device or AR device to render and display the image registration result.
  • the computer-readable storage medium is configured to store program codes for executing the following steps: acquiring at least two images by calling a first interface, wherein the first interface includes a first parameter, and a parameter value of the first parameter is at least two images, and any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain image features of any one of the images; determining a target displacement vector field of the object to be analyzed based on the extracted image features; performing image registration on at least two images based on the target displacement vector field of the object to be analyzed to obtain an image registration result; and outputting the image registration result by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the image registration result.
  • At least two images are obtained, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; feature extraction is performed on the at least two images respectively to obtain the image features of any one of the images; based on the extracted image features, the target displacement vector field of the object to be analyzed is determined; based on the target displacement vector field of the object to be analyzed, at least two images are image registered to obtain the image registration result, thereby achieving the purpose of improving the accuracy of image registration.
  • feature extraction can be performed on at least two images respectively to obtain image features containing global features and local features
  • the receptive field of the image can be expanded according to the global features and the local features to obtain a more accurate target displacement vector field
  • at least two images can be registered according to the target displacement vector field to obtain an image registration result with higher accuracy, thereby solving the technical problem of low accuracy in processing the registration task in the related art.
  • the disclosed technical content can be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a logical function division.
  • multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
  • the units described as separate components may or may not be physically separate.
  • the components shown may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure.
  • the aforementioned storage medium includes: U disk, read-only memory (ROM, referred to as Read-Only Memory), random access memory (RAM, referred to as Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
  • the solution provided by the embodiments of the present disclosure can be applied to the process of image registration, obtaining at least two images, wherein any one of the images contains the shooting results of the object to be analyzed under different conditions; performing feature extraction on the at least two images respectively to obtain the image features of any one of the images; determining the target displacement vector field of the object to be analyzed based on the extracted image features; and performing image registration on the at least two images based on the target displacement vector field of the object to be analyzed to obtain the image registration result, thereby solving the technical problems of low accuracy and limited application scenarios of the image registration method in the related art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Human Computer Interaction (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Ophthalmology & Optometry (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Image Processing (AREA)

Abstract

本公开公开了一种图像配准方法、电子设备以及计算机可读存储介质。其中，该方法包括：获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果。本公开解决了相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。

Description

图像配准方法、电子设备以及计算机可读存储介质
交叉援引
本公开要求于2023年06月26日提交中国专利局、申请号为2023107674466、发明名称为“图像配准方法、电子设备以及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及图像配准领域，具体而言，涉及一种图像配准方法、电子设备以及计算机可读存储介质。
背景技术
目前的图像配准方法一般使用提供具有模态和对比度不变性信息的特征描述符，但是由于上述特征描述符表示的信息有限，导致在具有大形变或者复杂解剖学差异(例如，患者间腹部)的环境中面临挑战，因此，现有的图像配准方法对配准任务进行处理的准确度较低，难以应用于处理复杂的配准任务的场景。
针对上述的问题，目前尚未提出有效的解决方案。
发明内容
本公开实施例提供了一种图像配准方法、电子设备以及计算机可读存储介质，以至少解决相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。
根据本公开实施例的一个方面，提供了一种图像配准方法，包括：获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果。
根据本公开实施例的一个方面，还提供了一种图像配准方法，包括：获取至少两张医学图像，其中，任意一张医学图像中包含了通过计算机断层扫描技术对同一个生物对象的目标部位在不同条件下的扫描结果；对至少两张医学图像分别进行特征提取，得到任意一张图像的图像特征，其中，图像特征包括：全局特征和局部特征；基于提取到的图像特征，确定目标部位的目标位移矢量场；基于目标部位的目标位移矢量场，将至少两张医学图像进行图像配准，得到图像配准结果。
根据本公开实施例的一个方面，还提供了一种图像配准方法，包括：响应作用于操作界面上的输入指令，在操作界面上显示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；响应作用于操作界面上的图像配准指令，在操作界面上显示图像配准结果，其中，图像配准结果是基于待分析的对象的目 标位移矢量场，将至少两张图像进行图像配准得到的，目标位移矢量场是基于提取到的图像特征确定的，提取到的图像特征包括：全局特征和局部特征，提取到的图像特征是对至少两张图像分别进行特征提取得到的。
根据本公开实施例的一个方面，还提供了一种图像配准方法，包括：在虚拟现实VR设备或增强现实AR设备的呈现画面上展示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；驱动VR设备或AR设备，渲染展示图像配准结果。
根据本公开实施例的一个方面，还提供了一种图像配准方法，包括：通过调用第一接口，获取至少两张图像，其中，第一接口包括第一参数，第一参数的参数值为至少两张图像，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；通过调用第二接口，输出图像配准结果，其中，第二接口包括第二参数，第二参数的参数值为图像配准结果。
根据本公开实施例的一个方面，还提供了一种电子设备，包括：存储器，被设置为存储有可执行程序；处理器，被设置为运行程序，其中，程序运行时执行上述实施例中任意一项的方法。
根据本公开实施例的一个方面，还提供了一种计算机程序产品，包括计算机程序，该计算机程序在被处理器执行时实现如前述任一方面所述的方法。
根据本公开实施例的一个方面，还提供了一种计算机程序产品，包括非易失性计算机可读存储介质，该非易失性计算机可读存储介质存储计算机程序，该计算机程序被处理器执行时实现如前述任一方面所述的方法。
根据本公开实施例的一个方面，还提供了一种计算机程序，该计算机程序被处理器执行时实现如前述任一方面所述的方法。
根据本公开实施例的一个方面，还提供了一种计算机可读存储介质，计算机可读存储介质包括存储的可执行程序，其中，在可执行程序运行时控制计算机可读存储介质所在设备执行上述实施例中任意一项的方法。
在本公开实施例中，获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果，实现了提高图像配准准确度的目的。容易注意到的是，可以对至少两张图像分别进行特征提取，得到包含有全局特征和局部特征的图像特征，可以根据全局特征和局 部特征扩大图像的感受野，以便得到更为准确的目标位移矢量场，可以根据目标位移矢量场将至少两张图像进行配准，从而得到准确度较高的图像配准结果，可以应用于处理复杂的配准任务的场景，进而解决了相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。
容易注意到的是，上面的通用描述和后面的详细描述仅仅是为了对本公开进行举例和解释，并不构成对本公开的限定。
附图说明
此处所说明的附图用来提供对本公开的进一步理解，构成本公开的一部分，本公开的示意性实施例及其说明用于解释本公开，并不构成对本公开的不当限定。在附图中：
图1是根据本公开实施例的一种图像配准方法的虚拟现实设备的硬件环境的示意图；
图2是根据本公开实施例的一种图像配准方法的计算环境的结构框图；
图3是根据本公开实施例1的图像配准方法的流程图；
图4a是根据本公开实施例的一种图像配准过程的结构图；
图4b是根据本公开实施例的一种凸优化处理过程的结构图；
图5是根据本公开实施例2的一种图像配置方法的流程图；
图6是根据本公开实施例3的一种图像配置方法的流程图；
图7是根据本公开实施例4的一种图像配准方法的流程图；
图8是根据本公开实施例5的一种图像配准方法的流程图；
图9是根据本公开实施例6的一种图像配准装置的示意图；
图10是根据本公开实施例7的一种图像配准装置的示意图；
图11是根据本公开实施例8的一种图像配准装置的示意图；
图12是根据本公开实施例9的一种图像配准装置的示意图；
图13是根据本公开实施例10的一种图像配准装置的示意图；
图14是根据本公开实施例的一种计算机终端的结构框图；
图15是本公开实施例的一种处理器的结构框图。
具体实施方式
为了使本技术领域的人员更好地理解本公开方案，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分的实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本公开保护的范围。
需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二” 等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本公开的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
首先，在对本公开实施例进行描述的过程中出现的部分名词或术语适用于如下解释：
配准(registration)：是指同一区域内以不同成像手段所获得的不同图像图形的地理坐标的匹配。图像配准是图像处理研究领域中的一个典型问题和技术难点，其目的在于比较或融合针对同一对象在不同条件下获取的图像，例如图像会来自不同的采集设备，取自不同的时间，不同的拍摄视角等等，有时也需要用到针对不同对象的图像配准问题。具体地说，对于一组图像数据集中的两幅图像，通过寻找一种空间变换把一幅图像映射到另一幅图像，使得两图中对应于空间同一位置的点一一对应起来，
代价体积(cost volume or correlation volume)：是光流估计或者双目匹配中一种常见的中间表示，用来表示特征图之间像素级别的相似度从而找到对应关系。
凸优化：或叫做凸最优化，凸最小化，是数学最优化的一个子领域，研究定义于凸集中的凸函数最小化的问题，凸优化在某种意义上说较一般情形的数学最优化问题要简单，譬如在凸优化中局部最优值是全局最优值。
目前，可变形图像配准是一项基本的医学图像分析任务，传统上被视为图像对之间密集位移场空间的连续更新问题，存在图像配准的迭代过程效率较低的问题。为了解决该问题，提出了一种基于学习的深度网络预测位移场的方法进行迭代，以便提高迭代效率，但是，由于不同类型的注册任务都需要培训，因此图像配准方法难以适应于多种应用场景。
此外，收集足够的训练数据需要花费大量的时间，迭代更新和基于学习的方法都依赖于在强度上计算的相似性度量，难以应用在解剖对应关系上，一些研究使用提供模态和对比度不变信息的特征描述符进行处理，但是一般只能表示局部信息而不包含全局语义信息，因此，图像配准方法在具有大变形或复杂解剖学差异(例如，患者间腹部)的环境中面临挑战。
配准也被表述为一个离散更新问题，可以采用一组密集的离散位移作为成本量进行处理，这类方法的主要挑战是搜索空间的巨大规模，因为在典型的三维电子计算机断层扫描(Computed Tomography，简称为CT)中存在数百万体素，并且移动扫描中的每个体素都可以与其他扫描中的数千个点合理配对，从而导致高计算负担。为了通过离散更新获得快速配准，可以通过在不同体素的邻域内构建成本量来修剪搜索空间，然而，该方式可以解决的变形幅度范围受到邻域窗口大小的限制，依赖于精确的预对 齐，实施过程较为困难。
广泛的使用的性能比较好的可变形配准方法可以包括：Deeds、ConvexAdam，其中，Deeds和ConvexAdam使用提供模态和对比度不变信息的特征描述符，但是仍然表示的是局部信息而不包含全局语义信息，因此，还是难以应用在具有大变形或复杂解剖学差异(例如，患者间腹部)的环境中；此外ConvexAdam依赖于精确的预对齐，该实施过程较为困难。
处理复杂的配准任务依赖于对主体间变异、器官变形、造影剂注射和病理变化等具有鲁棒性的独特特征，以及有利于提高复杂变形配准准确性的全局/上下文信息，为了实现这些目标，本公开采用基于自我监督解剖的方式嵌入的特征，对全局和局部嵌入进行编码；在邻域内计算代价损失过程中，会受到邻域大小的限制，本公开为了解决该问题，提出了通过金字塔模式实现从小分辨率图像到大分辨率图像的图像处理过程，不同的级别都有一个较小的搜索范围，而不是一个分辨率下存在一个大搜索范围，这有利于提高计算负担低、运行时间快等效率，扩大感受野，提高配准准确率。
实施例1
根据本公开实施例，提供了一种图像配准方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图1是根据本公开实施例的一种图像配准方法的虚拟现实设备的硬件环境的示意图。如图1所示，虚拟现实设备104与终端106相连接，终端106与服务器102通过网络进行连接，上述虚拟现实设备104并不限定于：虚拟现实头盔、虚拟现实眼镜、虚拟现实一体机等，上述终端104并不限定于PC、手机、平板电脑等，服务器102可以为媒体文件运营商对应的服务器，上述网络包括但不限于：广域网、城域网或局域网。
可选地，该实施例的虚拟现实设备104包括：存储器、处理器和传输装置。存储器用于存储应用程序，该应用程序可以用于执行：获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场将至少两张图像进行图像配准，得到图像配准结果，从而解决了相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。
该实施例的终端可以用于执行在虚拟现实(Virtual Reality，简称为VR)设备或增强现实(Augmented Reality，简称为AR)设备的呈现画面上展示图像配准结果，并向虚拟现实设备104发送图像配准结果，虚拟现实设备104在接收到图像配准结果之后在目标投放位置显示出来。
可选地，该实施例的虚拟现实设备104带有的眼球追踪的头戴式显示器(Head Mount Display，简称为HMD)头显与眼球追踪模块与上述实施例中的作用相同，也即，HMD头显中的屏幕，用于显示实时的画面，HMD中的眼球追踪模块，用于获取用户眼球的实时运动轨迹。该实施例的终端通过跟踪系统获取用户在真实三维空间的位置信息与运动信息，并计算出用户头部在虚拟三维空间中的三维坐标，以及用户在虚拟三维空间中的视野朝向。
图1示出的硬件结构框图，不仅可以作为上述AR/VR设备(或移动设备)的示例性框图，还可以作为上述服务器的示例性框图，一种可选实施例中，图2以框图示出了使用上述图1所示的AR/VR设备(或移动设备)作为计算环境201中计算节点的一种实施例。图2是根据本公开实施例的一种图像配准方法的计算环境的结构框图，如图2所示，计算环境201包括运行在分布式网络上的多个(图中采用210-1，210-2，…,来示出)计算节点(如服务器)。不同计算节点都包含本地处理和内存资源，终端用户202可以在计算环境201中远程运行应用程序或存储数据。应用程序可以作为计算环境201中的多个服务220-1，220-2，220-3和220-4进行提供，分别代表服务“A”，“D”，“E”和“H”。
终端用户202可以通过客户端上的web浏览器或其他软件应用程序提供和访问服务，在一些实施例中，可以将终端用户202的供应和/或请求提供给入口网关230。入口网关230可以包括一个相应的代理来处理针对服务(计算环境201中提供的一个或多个服务)的供应和/或请求。
服务是根据计算环境201支持的各种虚拟化技术来提供或部署的。在一些实施例中，可以根据基于虚拟机(Virtual Machine，简称为VM)的虚拟化、基于容器的虚拟化和/或类似的方式提供服务。基于虚拟机的虚拟化可以是通过初始化虚拟机来模拟真实的计算机，在不直接接触任何实际硬件资源的情况下执行程序和应用程序。在虚拟机虚拟化机器的同时，根据基于容器的虚拟化，可以启动容器来虚拟化整个操作系统(Operating System，简称为OS)，以便多个工作负载可以在单个操作系统实例上运行。
在基于容器虚拟化的一个实施例中，服务的若干容器可以被组装成一个Pod(例如，Kubernetes Pod)。举例来说，如图2所示，服务220-2可以配备一个或多个Pod240-1，240-2，…，240-N(统称为Pod)。Pod可以包括代理245和一个或多个容器242-1，242-2，…，242-M(统称为容器)。Pod中一个或多个容器处理与服务的一个或多个相应功能相关的请求，代理245通常控制与服务相关的网络功能，如路由、负载均衡等。其他服务也可以配备类似的Pod。
在操作过程中，执行来自终端用户202的用户请求可能需要调用计算环境201中的一个或多个服务，执行一个服务的一个或多个功能需要调用另一个服务的一个或多个功能。如图2所示，服务“A”220-1从入口网关230接收终端用户202的用户请求， 服务“A”220-1可以调用服务“D”220-2，服务“D”220-2可以请求服务“E”220-3执行一个或多个功能。
上述的计算环境可以是云计算环境，资源的分配由云服务提供上管理，允许功能的开发无需考虑实现、调整或扩展服务器。该计算环境允许开发人员在不构建或维护复杂基础设施的情况下执行响应事件的代码。服务可以被分割完成一组可以自动独立伸缩的功能，而不是扩展单个硬件设备来处理潜在的负载。
在上述运行环境下，本公开提供了如图3所示的图像配准方法。需要说明的是，该实施例的图像配准方法可以由图1所示实施例的移动终端执行。图3是根据本公开实施例1的图像配准方法的流程图。如图3所示，该方法可以包括如下步骤：
步骤S302，获取至少两张图像。
其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果。
上述的至少两张图像可以为属于医学领域的CT，也可以是其他领域的图像，此处对图像的类型不做任何限定。
上述的图像中包含的待分析的对象可以为两张图像中需要进行比对的部分。在医学领域中，上述的至少两张图像可以为患者的腹部CT图像，待分析的对象可以为腹部中出现异常的部位，例如，患者腹部的病灶。需要说明的是，此处对待分析的对象并不做任何限定，可以根据对图像的分析需求确定待分析的对象，此处仅作实例说明。
上述的不同条件可以为不同时间段、不同环境、不同类型的拍摄设备、不同拍摄视角，此处的不同条件可以根据实际的场景进行设置。
在一种可选的实施例中，可以在不同时间段对待分析的对象或者待分析的对象所在的区域进行拍摄，得到至少两张图像；还可以使用不同的拍摄方式对待分析的对象或者待分析的对象所在的区域进行拍摄，得到至少两张图像，还可以在其他不同条件下对待分析对象或者待分析的对象所在的区域进行拍摄，得到至少两张图像。
需要说明的是，在本公开实施例中，以获取两张图像进行图像配准为例进行说明，但不仅限于此，获取其他数量的图像的处理方式类似，在此不作赘述。
步骤S304，对至少两张图像分别进行特征提取，得到任意一张图像的图像特征。
其中，图像特征至少包括：全局特征和局部特征。
上述的任意一张图像的图像特征可以为至少两张图像中每张图像的图像特征。
在一种可选的实施例中，可以通过特征提取器对至少两张图像进行特征提取，得到每张图像的图像特征，该特征提取器可以为自监督解剖嵌入特征提取器(Self-supervised Autoencoder with Mosaicked Training，简称为SAM)，但不限于此，该可以是其他类型的特征提取器。
需要说明的是，SAM通常用于对图像进行特征提取，该算法使用了一种自监督学习方法，通过自我学习的方式训练一个解剖嵌入模型，该模型可以学习图像上不同点的全局性语义特征和局部性语义特征，因此可以得到任意一张图像的图像特征，且该 图像特征至少包括全局特征和局部特征。
步骤S306，基于提取到的图像特征，确定待分析的对象的目标位移矢量场。
上述的目标位移矢量场可以表示待分析的对象在至少两张图像中的空间变化，可以通过对待分析的对象的位置进行位移矢量场估计，目标位移矢量场中的矢量用于表示待分析的对象在图像中的位移方向和大小。
上述的目标位移矢量场在几何学和力学中，位移是一个向量，其长度是从初始位置到经历运动的点的最终位置的最短距离，它量化了点轨迹从初始位置到最终位置沿直线的净运动或总运动的距离和方向。位移矢量场可以描述为相对位置，即，作为点相对于其初始位置的最终位置，可以定义为最终位置和初始位置之间的差异。
在一种可选的实施例中，可以根据提取到的图像特征构建代价体积，根据该代价体积可以确定出目标位移矢量场，其中，代价体积是光流估计或者双目匹配中一种常见的中间表示，可以用于表示特征之间像素级别的相似度从而找到像素之间的对应关系，可以根据该对应关系确定出待分析对象的目标位移矢量场。
上述的代价体积可以为6D代价体积，但不限于此，其中，6D代价体积存储用于将像素与其在另一张图像上对应像素相关联的数据匹配成本。
需要说明的是，代价体积(cost volume or correlation volume)是光流估计或者双目匹配中一种常见的中间表示，用来表示特征图之间像素级别的相似度，以便于根据该相似度找到像素之间的对应关系。一般计算的是稠密的代价体积(dense cost volume)，在光流问题中，它的复杂度为像素个数的平方(H×W×H×W)；在3D医学图像配准问题中，它的复杂度为像素个数的平方(H×W×D×H×W×D)，其中3D图像的大小为H×W×D。
基于SAM特征构建6D代价体积是一种通过特征构建6D代价体积的方法，它使用两张图像的特征来构建6D代价体积，用来衡量两个3D物体的相似程度。通过将SAM提取图像中的特征，可以得到对应的6D代价体积。
可选地，可以基于上述代价体积进行凸优化处理得到目标位移矢量场。其中，凸优化处理是指求取目标函数为凸函数的一类优化问题。
步骤S308，基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果。
上述的图像配准是指对同一区域内不同成像手段获取的不同图像的地理坐标进行匹配，可选地，图像配准就是通过寻找一种空间变换将一幅图像映射到另一幅图像中，使得两幅图像中对于空间同一位置的点一一对应。
在一种可选的实施例中，可以根据待分析的对象的目标位移矢量场将至少两张图像中的任意一张图像映射到另一张图像中，以便在一张图像中可以观察到待分析的对象的变化情况，从而便于用户对待分析的对象进行分析，以便提高分析的效率。
在医学领域中，待分析的对象可以为患者腹部区域，可以根据患者腹部区域的至 少两张CT确定腹部区域的目标位移矢量场，可以根据该目标位移矢量场将两张CT进行配准，从而得到图像配准结果，便于医生根据图像配准结果分析患者腹部区域的问题。
通过上述步骤，获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果，实现了提高图像配准准确度的目的。容易注意到的是，可以对至少两张图像分别进行特征提取，得到包含有全局特征和局部特征的图像特征，可以根据全局特征和局部特征扩大图像的感受野，以便得到更为准确的目标位移矢量场，可以根据目标位移矢量场将至少两张图像进行配准，从而得到准确度较高的图像配准结果，可以应用于处理复杂的配准任务的场景，进而解决了相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。
本公开上述实施例中，对至少两张图像分别进行特征提取，得到任意一张图像的图像特征，包括：基于至少两张图像中的第一图像对应的位移矢量场，对第一图像进行变形，得到变形图像；对变形图像和至少两张图像中的第二图像分别进行特征提取，得到任意一张图像的图像特征，其中，第二图像用于表征至少两张图像中除第一图像之外的图像。
上述的第一图像和第二图像可以为不同时间点采集到的同一区域的图像，上述的第一图像可以为先采集到的图像，第二图像可以为后采集的图像；上述的第二图像可以为先采集到的图像，上述的第一图像可以为后采集到的图像，此处对第一图像和第二图像不做任何限定。
上述的第一图像也可以为至少两张图像中的任意一张图像，上述的第二图像可以为至少两张图像中除该任意一张图像中的其他图像。可选地，在至少两张图像仅包含两张图像的情况下，第一图像可以是两张图像中的源图像，第二图像可以是目标图像。
上述的位移矢量场也可以成为配准场(up-sampled field)。
上述的第一图像对应的位移矢量场可以根据第一图像的尺度信息确定，第一图像的尺度信息可以分为多个层级，层级越高，则说明图像的分辨率越大，层级越低，则说明图像的分辨率越小，因此，如果第一图像的尺度最小，也即层级最低，则该位移矢量场可以是预设矢量场，如果第一图像的尺度不是最小尺度，也即不是最低层级，则该位移矢量场可以是通过上一个层级的图像所确定出的目标位移矢量场。
在一种可选的实施例中，可以按照三个层级对上述内容进行描述，第一层级为图像尺度最小的层级、第二层级为图像尺度次大的层级、第三层级为图像尺度最大的层级，当第一图像处于第一层级时，说明第一图像的分辨率最低，此时可以直接获取该变形图像与第一层级的第二图像之间的第一位移矢量场；当第一图像处于第二层级时， 可以基于对第一位移矢量场进行上采样后所得到的位移矢量场对第一图像进行变形，得到变形图像，并获取该变形图像与第二层级的第二图像之间的第二位移矢量场；当第一图像处于第三层级时，说明第一图像的分辨率最大，此时可以基于对第二位移矢量场进行上采样后所得到的位移矢量场对第一图像进行变形，得到变形图像，并获取该变形图像与最大层级的第二图像之间的第三位移矢量场。
在另一种可选的实施例中，可以对任意一个层级得到的变形图像和第二图像分别进行特征提取，得到任意一张图像的图像特征，还可以对上述三个层级得到的变形图像和第二图像分别进行特征提取，得到任意一张图像的图像特征。
本公开上述实施例中，对变形图像和至少两张图像中的第二图像分别进行特征提取，得到任意一张图像的图像特征，包括：对变形图像和第二图像进行特征提取，得到全局特征和局部特征；对全局特征和局部特征进行叠加，得到图像特征。
在一种可选的实施例中，可以通过特征提取器同时捕获变形图像和第二图像中的全局特征和局部特征，进一步可以对全局特征和局部特征进行叠加，从而得到任意一张图像的图像特征。
本公开上述实施例中，基于提取到的图像特征，确定待分析的对象的目标位移矢量场，包括：获取提取到的图像特征的点积，得到目标代价体积；对目标代价体积进行凸优化处理，得到目标位移矢量场。
上述的目标代价体积用于根据图像特征的相似度找到图像特征之间的对应关系。
在一种可选的实施例中，可以获取提取到的图像特征的点积，来衡量第一图像和第二图像之间的相似度，并根据两个图像之间的相似度构建目标代价体积，可以对目标代价体积进行凸优化处理(convex optimization)，得到目标位移矢量场。其中，目标代价体积可以为6D代价体积。
本公开上述实施例中，对目标代价体积进行凸优化处理得到目标位移矢量场，包括：对目标代价体积进行参数最小化处理，得到初始位移矢量场；对初始位移矢量场进行平均池化操作，得到目标位移矢量场。
在一种可选的实施例中，可以为目标代价体积进行参数最小化处理，也即，利用Argmin(.)对目标代价体积进行参数最小化处理，以便得到目标代价体积在Argmin(.)的输出值最小时对应的变量，根据该变量确定出初始位移矢量场，可以对初始位移矢量场进行平均池化操作，以便可以将初始位移矢量场分成若干个大小相同的区域，然后对不同区域内的特征值取平均值，从而得到目标位移矢量场。
本公开上述实施例中，对至少两张图像分别进行特征提取，得到任意一张图像的图像特征，包括：分别对至少两张图像进行多次下采样，得到任意一张图像的多个下采样图像，其中，不同下采样图像的分辨率不同；基于至少两张图像中的第一图像的下采样图像对应的采样位移矢量场，对第一图像的下采样图像进行变形，得到采样变形图像；对采样变形图像和至少两张图像中的第二图像的下采样图像分别进行特征提 取，得到任意一张图像的下采样图像的下采样图像特征；对多个下采样图像特征进行汇总，得到任意一张图像的图像特征。
在一种可选的实施例中，可以分别对至少两张图像进行多次下采样，以便得到至少两张图像在不同尺度上的表示，即得到任意一张图像的多个下采样图像，不同下采样图像的分辨率不同，可以根据第一图像的下采样图像对应的采样位移矢量场对第一图像的下采样图像进行变形，得到采样变形图像，可以得到不同分辨率的采样变形图像，可以对采样变形图像和第二图像的下采样图像分别进行特征提取，得到任意一张图像的下采样图像的下采样图像特征，可以对多个不同分辨率的下采样图像特征进行汇总，得到任意一张图像的图像特征金字塔。
在另一种可选的实施例中，可以从最大的尺度的下采样图像进行处理，得到最大尺度的采样变形图像，可以对采样变形图像和第二图像的下采样图像分别进行特征提取，得到任意一张图像的下采样图像的下采样图像特征，可以根据该下采样图像特征构建下一尺度图像处理过程中所需要的采样位移矢量场，可选地，可以根据下采样图像特征构建代价体积，进行凸优化处理，可以得到下一尺度图像处理过程中所需要的采样位移矢量场，循环得到各个尺度的图像对应的采样位移矢量场，从而根据多个采样位移矢量场依次得到多个采样变形图像，根据多个采样变形图像依次得到多个下采样图像特征，可以对多个下采样图像特征进行汇总，得到任意一张图像的图像特征。
通过由粗到精的方式在不同尺度的情况下对第一图像进行变形得到采样变形图像，并对采样变形图像和第二图像的下采样图像分别进行特征提取，得到下采样图像特征。
本公开上述实施例中，基于提取到的图像特征，确定待分析的对象的目标位移矢量场，包括：获取任意一张图像的下采样图像的下采样图像特征的点积，得到采样代价体积；基于采样代价体积进行凸优化处理，得到采样位移矢量场；对多个采样位移矢量场进行叠加，得到目标位移矢量场。
在一种可选的实施例中，可以获取任意一张图像的下采样图像特征的点积，来衡量第一图像的下采样图像和第二图像的下采样图像之间的相似度，并根据两个下采样图像之间的相似度构建采样代价体积，可以基于采样代价体积进行凸优化处理，得到采样位移矢量场。
可以对不同尺度的下采样图像循环进行上述操作，需要说明的是，可以根据上一尺度得到的采样位移矢量场对当前尺度得到的采样位移矢量场进行变形，以便提高变形的准确度，在得到不同尺度下的多个采样位移矢量场之后，可以对多个采样位移矢量场进行叠加，得到目标位移矢量场。
本公开上述实施例中，该方法还包括：在第一图像的下采样图像是预设下采样图像的情况下，确定第一图像的下采样图像对应的采样位移矢量场为预设位移矢量场，其中，预设下采样图像用于表征第一图像的最大尺度的下采样图像；在第一图像的下 采样图像不是预设下采样图像的情况下，对第一图像的下一尺度的下采样图像对应的采样位移矢量场进行上采样处理，得到第一图像的下采样图像对应的采样位移矢量场。
上述的预设下采样图像可以为最大尺度下的下采样图像，由于最大尺度下的下采样图像的分辨率较低，因此，需要处理的特征数量较少，因此，可以高效的提取到下采样图像的全局特征，可以从大分辨率到小分辨率对下采样图像依次进行处理，在保证能提取到全局特征和局部特征的情况下，可以进一步提高特征的提取效率。
在一种可选的实施例中，可以在第一图像的下采样图像是最大尺度的下采样图像时，由于是第一次进行变形，因此还未得到位移矢量场，此时可以将预先设置的预设位移矢量场确定为第一图像的下采样图像对应的采样位移矢量场；在根据最大尺度的下采样图像得到下采样图像特征之后，可以获取下采样图像特征的点积，得到采样代价体积，可以基于采样代价体积进行凸优化处理，得到下一尺度的下采样图像进行变形时所需要的采样位移矢量场，基于此可以对不同尺度的下采样图像进行依次处理，得到不同尺度下得到的采样位移矢量场。
下面以两张图像配准的场景为例，对本公开的优选实施例进行详细说明。图4a是根据本公开实施例的一种图像配准过程的结构图,如图4a所示，需要进行图像配准的图像包含第一图像IS和第二图像IT，可以分别对第一图像和第二图像进行下采样，得到第一图像的图像金字塔(Pyramid Source)和第二图像的图像金字塔(Pyramid Target)，两个采样金字塔的最上层图像为尺度最大的下采样图像，采样金字塔最下层图像为尺度最小的下采样图像。每个尺度的下采样图像处理流程类似，此处，以尺度最小的下采样图像处理流程为例进行说明：在对下采样图像进行特征处理时，可以对上一尺度得到的配准场进行上采样(Up-Sampled Field)，也即上述的采样位移矢量场对第一图像对应的下采样图像进行变形操作(Warping)，得到采样变形图像，可以对采样变形图像和第二图像的下采样图像分别进行特征提取，得到下采样图像特征，可以对下采样图像特征进行凸优化操作(Convex Optimization)，得到采样位移矢量场，也即得到当前尺度的配准场，最后可以对多个采样位移矢量场进行叠加，得到目标位移矢量场，可以根据目标位移矢量场将第一图像和第二图像进行图像配准，得到图像配准结果。需要说明的是，对于尺度最小的下采样图像的处理流程，可以根据预先设定的配准场对第一图像对应的下采样图像进行变形操作(Warping)，得到采样变形图像。
图4b是根据本公开实施例的一种凸优化处理过程的结构图，如图4b所示，可以对第一图像的局部特征和全局特征进行叠加，得到第一图像和第二图像的图像特征，可以对图像特征进行点积操作(Dot Product)，得到目标代价体积，可以利用预设函数[Argmin(.)]确定目标代价体积的函数最小自变量的取值，得到初始位移矢量场，也即图中显示的第一配准场u，可以对初始位移矢量场进行平均池化操作(Mean Conv)，得到目标位移矢量场，也即图中显示的第二配准场ψ。
本公开的图像配准方法提供了一种自监督解剖嵌入网络(SAMConvex)，可以快速的用于图像配准的离散优化方法，包含一个解耦的凸优化程序，通过该凸优化程序来获取基于自监督解剖嵌入特征提取器的位移矢量场，该特征提取器同时可以捕获全局特征和局部特征，可选地，SAMConvex可以提取不同尺度的特征，并基于该特征可以构建6D代价体积，通过使用由大尺度到小尺度的处理过程可以迭代更新位移矢量场。
需要注意的是，不同级别的6D代价体积通过采用不同分辨率的输入来进行计算，每个级别的成本量都是通过采用不同分辨率的输入来计算的。
需要说明的是，本公开所涉及的用户信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)，均为经用户授权或者经过各方充分授权的信息和数据，并且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准，并提供有相应的操作入口，供用户选择授权或者拒绝。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本公开并不受所描述的动作顺序的限制，因为依据本公开，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，本公开中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本公开所必须的。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本公开各个实施例的方法。
实施例2
根据本公开实施例，还提供了一种图像配准方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图5是根据本公开实施例2的一种图像配置方法的流程图，如图5所示，该方法包括如下步骤：
步骤S502，获取至少两张医学图像。
其中，任意一张医学图像中包含了通过计算机断层扫描技术对同一个生物对象的目标部位在不同条件下的扫描结果。
上述的医学图像包括但不限于CT图像、患者的皮肤图像，此处不做限定，可以 根据实际情况设置医学图像的类型。
步骤S504，对至少两张医学图像分别进行特征提取，得到任意一张图像的图像特征。
其中，图像特征包括：全局特征和局部特征。
步骤S506，基于提取到的图像特征，确定目标部位的目标位移矢量场。
步骤S508，基于目标部位的目标位移矢量场，将至少两张医学图像进行图像配准，得到图像配准结果。
通过上述步骤，获取至少两张医学图像，其中，任意一张医学图像中包含了通过计算机断层扫描技术对同一个生物对象的目标部位在不同条件下的扫描结果；对至少两张医学图像分别进行特征提取，得到任意一张图像的图像特征，其中，图像特征包括：全局特征和局部特征；基于提取到的图像特征，确定目标部位的目标位移矢量场；基于目标部位的目标位移矢量场，将至少两张医学图像进行图像配准，得到图像配准结果，实现了提高图像配准准确度的目的。容易注意到的是，可以对至少两张图像分别进行特征提取，得到包含有全局特征和局部特征的图像特征，可以根据全局特征和局部特征扩大图像的感受野，以便得到更为准确的目标位移矢量场，可以根据目标位移矢量场将至少两张图像进行配准，从而得到准确度较高的图像配准结果，可以应用于处理复杂的配准任务的场景，进而解决了相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。
需要说明的是，本公开上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例3
根据本公开实施例，还提供了一种图像配准方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图6是根据本公开实施例3的一种图像配置方法的流程图，如图6所示，该方法包括如下步骤：
步骤S602，响应作用于操作界面上的输入指令，在操作界面上显示至少两张图像。
其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果。
上述的操作界面可以为能够显示至少两张图像的显示界面，通过在显示界面上进行相关触控操作可以显示出至少两张图像。
步骤S604，响应作用于操作界面上的图像配准指令，在操作界面上显示图像配准结果。
其中，图像配准结果是基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准得到的，目标位移矢量场是基于提取到的图像特征确定的，提取到的图像 特征包括：全局特征和局部特征，提取到的图像特征是对至少两张图像分别进行特征提取得到的。
上述的图像配准指令可以为在需要对至少两张图像进行配准时通过对操作界面的相关控件进行触控生成的指令，根据该图像配准指令可以在操作界面上显示图像配准结果。
通过上述步骤，响应作用于操作界面上的输入指令，在操作界面上显示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；响应作用于操作界面上的图像配准指令，在操作界面上显示图像配准结果，其中，图像配准结果是基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准得到的，目标位移矢量场是基于提取到的图像特征确定的，提取到的图像特征包括：全局特征和局部特征，提取到的图像特征是对至少两张图像分别进行特征提取得到的，实现了提高图像配准准确度的目的。容易注意到的是，可以对至少两张图像分别进行特征提取，得到包含有全局特征和局部特征的图像特征，可以根据全局特征和局部特征扩大图像的感受野，以便得到更为准确的目标位移矢量场，可以根据目标位移矢量场将至少两张图像进行配准，从而得到准确度较高的图像配准结果，可以应用于处理复杂的配准任务的场景，进而解决了相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。
需要说明的是，本公开上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例4
根据本公开实施例，还提供了一种可以应用于虚拟现实VR设备、增强现实AR设备等虚拟现实场景下的图像配准方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图7是根据本公开实施例4的一种图像配准方法的流程图。如图7所示，该方法可以包括如下步骤：
步骤S702，在虚拟现实VR设备或增强现实AR设备的呈现画面上展示至少两张图像。
其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果。
步骤S704，对至少两张图像分别进行特征提取，得到任意一张图像的图像特征。
其中，图像特征至少包括：全局特征和局部特征。
步骤S706，基于提取到的图像特征，确定待分析的对象的目标位移矢量场。
步骤S708，基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果。
步骤S710，驱动VR设备或AR设备，渲染展示图像配准结果。
通过上述步骤，在虚拟现实VR设备或增强现实AR设备的呈现画面上展示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；驱动VR设备或AR设备，渲染展示图像配准结果，实现了提高图像配准准确度的目的。容易注意到的是，可以对至少两张图像分别进行特征提取，得到包含有全局特征和局部特征的图像特征，可以根据全局特征和局部特征扩大图像的感受野，以便得到更为准确的目标位移矢量场，可以根据目标位移矢量场将至少两张图像进行配准，从而得到准确度较高的图像配准结果，可以应用于处理复杂的配准任务的场景，进而解决了相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。
可选地，在本实施例中，上述图像配准方法可以应用于由服务器、虚拟现实设备所构成的硬件环境中。在虚拟现实VR设备或增强现实AR设备的呈现画面上展示图像配准结果，服务器可以为媒体文件运营商对应的服务器，上述网络包括但不限于：广域网、城域网或局域网，上述虚拟现实设备并不限定于：虚拟现实头盔、虚拟现实眼镜、虚拟现实一体机等。
可选地，虚拟现实设备包括：存储器、处理器和传输装置。存储器用于存储应用程序，该应用程序可以用于执行：在虚拟现实VR设备或增强现实AR设备的呈现画面上展示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征，其中，图像特征至少包括：全局特征和局部特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；驱动VR设备或AR设备，渲染展示图像配准结果。
需要说明的是，该实施例的上述应用在VR设备或AR设备中的图像配准方法可以包括图7所示实施例的方法，以实现驱动VR设备或AR设备，展示图像配准结果的目的。
可选地，该实施例的处理器可以通过传输装置调用上述存储器存储的应用程序以执行上述步骤。传输装置可以通过网络接收服务器发送的媒体文件，也可以用于上述处理器与存储器之间的数据传输。
需要说明的是，本公开上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例5
根据本公开实施例，还提供了一种目标跟踪方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出 或描述的步骤。
图8是根据本公开实施例5的一种图像配准方法的流程图，如图8所示，该方法包括如下步骤：
步骤S802，通过调用第一接口，获取至少两张图像。
其中，第一接口包括第一参数，第一参数的参数值为至少两张图像，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果。
上述步骤中的第一接口可以是云服务器与客户端之间进行数据交互的接口，客户端可以将至少两张图像传入接口函数，作为接口函数的第一参数，实现将至少两张图像上传到云服务器的目的。
步骤S804，对至少两张图像分别进行特征提取，得到任意一张图像的图像特征。
其中，图像特征至少包括：全局特征和局部特征。
步骤S806，基于提取到的图像特征，确定待分析的对象的目标位移矢量场。
步骤S808，基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果。
步骤S810，通过调用第二接口，输出图像配准结果。
其中，第二接口包括第二参数，第二参数的参数值为图像配准结果。
上述的第二接口可以是云服务器与客户端之间进行数据交互的接口，云服务器可以将图像配准结果传入接口函数，作为接口函数的第二参数，实现将图像配准结果下发至客户端的目的。
通过上述步骤，通过调用第一接口，获取至少两张图像，其中，第一接口包括第一参数，第一参数的参数值为至少两张图像，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；通过调用第二接口，输出图像配准结果，其中，第二接口包括第二参数，第二参数的参数值为图像配准结果，实现了提高图像配准准确度的目的。容易注意到的是，可以对至少两张图像分别进行特征提取，得到包含有全局特征和局部特征的图像特征，可以根据全局特征和局部特征扩大图像的感受野，以便得到更为准确的目标位移矢量场，可以根据目标位移矢量场将至少两张图像进行配准，从而得到准确度较高的图像配准结果，可以应用于处理复杂的配准任务的场景，进而解决了相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。
需要说明的是，本公开上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例6
根据本公开实施例，还提供了一种用于实施上述图像配置方法的图像配准装置， 图9是根据本公开实施例6的一种图像配准装置的示意图，如图9所示，该装置900包括：获取模块902、提取模块904、确定模块906、配准模块908。
其中，获取模块被设置为获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；提取模块被设置为对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；确定模块被设置为基于提取到的图像特征，确定待分析的对象的目标位移矢量场；配准模块被设置为基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果。
此处需要说明的是，上述获取模块902、提取模块904、确定模块906、配准模块908对应于实施例1中的步骤S302至步骤S308，四个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器(例如，存储器1402)中并由一个或多个处理器(例如，处理器1401a，1401b，……，1401n)处理的硬件组件或软件组件，上述模块也可以作为装置的一部分可以运行在实施例1提供的计算机终端1400A中。
本公开上述实施例中，提取模块还被设置为基于至少两张图像中的第一图像对应的位移矢量场对第一图像进行变形，得到变形图像；对变形图像和至少两张图像中的第二图像分别进行特征提取，得到任意一张图像的图像特征，其中，第二图像用于表征至少两张图像中除第一图像之外的图像。
本公开上述实施例中，提取模块还被设置为对变形图像和第二图像进行特征提取，得到全局特征和局部特征；对全局特征和局部特征进行叠加，得到图像特征。
本公开上述实施例中，确定模块还被设置为获取提取到的图像特征的点积，得到目标代价体积；对目标代价体积进行凸优化处理，得到目标位移矢量场。
本公开上述实施例中，确定模块还被设置为对目标代价体积进行参数最小化处理，得到初始位移矢量场；对初始位移矢量场进行平均池化操作，得到目标位移矢量场。
本公开上述实施例中，提取模块还被设置为分别对至少两张图像进行多次下采样，得到任意一张图像的多个下采样图像，其中，不同下采样图像的分辨率不同；基于至少两张图像中的第一图像的下采样图像对应的采样位移矢量场，对第一图像的下采样图像进行变形，得到采样变形图像；对采样变形图像和至少两张图像中的第二图像的下采样图像分别进行特征提取，得到任意一张图像的下采样图像的下采样图像特征；对多个下采样图像特征进行汇总，得到任意一张图像的图像特征。
本公开上述实施例中，确定模块还被设置为获取任意一张图像的下采样图像的下采样图像特征的点积，得到采样代价体积；基于采样代价体积进行凸优化处理，得到采样位移矢量场；对多个采样位移矢量场进行叠加，得到目标位移矢量场。
本公开上述实施例中，该装置还被设置为在第一图像的下采样图像是预设下采样图像的情况下，确定第一图像的下采样图像对应的采样位移矢量场为预设位移矢量场，其中，预设下采样图像用于表征第一图像的最大尺度的下采样图像；在第一图像的下 采样图像不是预设下采样图像的情况下，对第一图像的下一尺度的下采样图像对应的采样位移矢量场进行上采样处理，得到第一图像的下采样图像对应的采样位移矢量场。
需要说明的是，本公开上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例7
根据本公开实施例，还提供了一种用于实施上述目标跟踪方法的目标跟踪装置，图10是根据本公开实施例7的一种图像配准装置的示意图，如图10所示，该装置1000包括：获取模块1002、提取模块1004、确定模块1006、配准模块1008。
其中，获取模块被设置为获取至少两张医学图像，其中，任意一张医学图像中包含了通过计算机断层扫描技术对同一个生物对象的目标部位在不同条件下的扫描结果；提取模块被设置为对至少两张医学图像分别进行特征提取，得到任意一张图像的图像特征，其中，图像特征包括：全局特征和局部特征；确定模块被设置为基于提取到的图像特征，确定目标部位的目标位移矢量场；配准模块被设置为基于目标部位的目标位移矢量场。将至少两张医学图像进行图像配准，得到图像配准结果。
此处需要说明的是，上述获取模块1002、提取模块1004、确定模块1006、配准模块1008对应于实施例2中的步骤S502至步骤S508，四个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器(例如，存储器1402)中并由一个或多个处理器(例如，处理器1401a，1401b，……，1401n)处理的硬件组件或软件组件，上述模块也可以作为装置的一部分可以运行在实施例1提供的计算机终端1400A中。
需要说明的是，本公开上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例8
根据本公开实施例，还提供了一种用于实施上述目标跟踪方法的目标跟踪装置，图11是根据本公开实施例8的一种图像配准装置的示意图，如图11所示，该装置1100包括：第一显示模块1102、第二显示模块1104。
其中，第一显示模块被设置为响应作用于操作界面上的输入指令，在操作界面上显示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；第二显示模块被设置为响应作用于操作界面上的图像配准指令，在操作界面上显示图像配准结果，其中，图像配准结果是基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准得到的，目标位移矢量场是基于提取到的图像特征确定的，提取到的图像特征包括：全局特征和局部特征，提取到的图像特征是对至少两张图像分别进行特征提取得到的。
此处需要说明的是，上述第一显示模块1102、第二显示模块1104对应于实施例3中的步骤S602至步骤S604，两个模块与对应的步骤所实现的实例和应用场景相同， 但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器(例如，存储器1402)中并由一个或多个处理器(例如，处理器1401a，1401b，……，1401n)处理的硬件组件或软件组件，上述模块也可以作为装置的一部分可以运行在实施例1提供的计算机终端1400A中。
需要说明的是，本公开上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例9
根据本公开实施例，还提供了一种用于实施上述目标跟踪方法的目标跟踪装置，图12是根据本公开实施例9的一种图像配准装置的示意图，如图12所示，该装置1200包括：展示模块1202、提取模块1204、确定模块1206、配准模块1208、驱动模块1210。
其中，展示模块被设置为在虚拟现实VR设备或增强现实AR设备的呈现画面上展示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；提取模块被设置为对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；确定模块被设置为基于提取到的图像特征，确定待分析的对象的目标位移矢量场；配准模块被设置为基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；驱动模块用于驱动VR设备或AR设备，渲染展示图像配准结果。
此处需要说明的是，上述展示模块1202、提取模块1204、确定模块1206、配准模块1208、驱动模块1210对应于实施例4中的步骤S702至步骤S710，五个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器(例如，存储器1402)中并由一个或多个处理器(例如，处理器1401a，1401b，……，1401n)处理的硬件组件或软件组件，上述模块也可以作为装置的一部分可以运行在实施例1提供的计算机终端1400A中。
需要说明的是，本公开上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例10
根据本公开实施例，还提供了一种用于实施上述目标跟踪方法的目标跟踪装置，图13是根据本公开实施例10的一种图像配准装置的示意图，如图13所示，该装置1300包括：获取模块1302、提取模块1304、确定模块1306、配准模块1308、调用模块1310。
其中，获取模块被设置为通过调用第一接口，获取至少两张图像，其中，第一接口包括第一参数，第一参数的参数值为至少两张图像，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；提取模块被设置为对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；确定模块被设置为基于提取到的图像特征，确 定待分析的对象的目标位移矢量场；配准模块用于基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；调用模块被设置为通过调用第二接口，输出图像配准结果，其中，第二接口包括第二参数，第二参数的参数值为图像配准结果。
此处需要说明的是，上述获取模块1302、提取模块1304、确定模块1306、配准模块1308、调用模块1310对应于实施例5中的步骤S802至步骤S810，五个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器(例如，存储器1402)中并由一个或多个处理器(例如，处理器1401a，1401b，……，1401n)处理的硬件组件或软件组件，上述模块也可以作为装置的一部分可以运行在实施例1提供的计算机终端10中。
需要说明的是，本公开上述实施例中涉及到的优选实施方案与实施例1提供的方案以及应用场景、实施过程相同，但不仅限于实施例1所提供的方案。
实施例11
本公开的实施例可以提供一种电子设备，该电子设备可以为AR/VR设备，该AR/VR设备可以是AR/VR设备群中的任意一个AR/VR设备。可选地，在本实施例中，上述AR/VR设备也可以替换为移动终端等终端设备。
可选地，在本实施例中，上述AR/VR设备可以位于计算机网络的多个网络设备中的至少一个网络设备。
在本实施例中，上述AR/VR设备可以执行图像配准方法中以下步骤的程序代码：获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果。
可选地，图14是根据本公开实施例的一种计算机终端的结构框图。如图14所示，该计算机终端1400A可以包括：一个或多个(图中仅示出一个)处理器1401、存储器1402、存储控制器、以及外设接口，其中，外设接口与射频模块、音频模块和显示器连接。
其中，存储器可用于存储软件程序以及模块，如本公开实施例中的图像配准方法和装置对应的程序指令/模块，处理器通过运行存储在存储器内的软件程序以及模块，从而执行各种功能应用以及数据处理，即实现上述的图像配准方法。存储器可包括高速随机存储器，还可以包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器可进一步包括相对于处理器远程设置的存储器，这些远程存储器可以通过网络连接至终端A。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
本公开实施例，还提供了一种处理器。图15示出了用来实现本公开实施例的处理器的结构框图。如图15所示，该处理器1500被设置为运行程序，其中，程序被该处理器运行时执行上述实施例中的方法。
可选地，处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果。
可选地，上述处理器还可以执行如下步骤的程序代码：基于至少两张图像中的第一图像对应的位移矢量场对第一图像进行变形，得到变形图像；对变形图像和至少两张图像中的第二图像分别进行特征提取，得到任意一张图像的图像特征，其中，第二图像用于表征至少两张图像中除第一图像之外的图像。
可选地，上述处理器还可以执行如下步骤的程序代码：对变形图像和第二图像进行特征提取，得到全局特征和局部特征；对全局特征和局部特征进行叠加，得到图像特征。
可选地，上述处理器还可以执行如下步骤的程序代码：获取提取到的图像特征的点积，得到目标代价体积；对目标代价体积进行凸优化处理，得到目标位移矢量场。
可选地，上述处理器还可以执行如下步骤的程序代码：对目标代价体积进行参数最小化处理，得到初始位移矢量场；对初始位移矢量场进行平均池化操作，得到目标位移矢量场。
可选地，上述处理器还可以执行如下步骤的程序代码：分别对至少两张图像进行多次下采样，得到任意一张图像的多个下采样图像，其中，不同下采样图像的分辨率不同；基于至少两张图像中的第一图像的下采样图像对应的采样位移矢量场对第一图像的下采样图像进行变形，得到采样变形图像；对采样变形图像和至少两张图像中的第二图像的下采样图像分别进行特征提取，得到任意一张图像的下采样图像的下采样图像特征；对多个下采样图像特征进行汇总，得到任意一张图像的图像特征。
可选地，上述处理器还可以执行如下步骤的程序代码：获取任意一张图像的下采样图像的下采样图像特征的点积，得到采样代价体积；基于采样代价体积进行凸优化处理，得到采样位移矢量场；对多个采样位移矢量场进行叠加，得到目标位移矢量场。
可选地，上述处理器还可以执行如下步骤的程序代码：在第一图像的下采样图像是预设下采样图像的情况下，确定第一图像的下采样图像对应的采样位移矢量场为预设位移矢量场，其中，预设下采样图像用于表征第一图像的最大尺度的下采样图像；在第一图像的下采样图像不是预设下采样图像的情况下，对第一图像的下一尺度的下采样图像对应的采样位移矢量场进行上采样处理，得到第一图像的下采样图像对应的采样位移矢量场。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：获取至少两张医学图像，其中，任意一张医学图像中包含了通过计算机断层扫描技术对同一个生物对象的目标部位在不同条件下的扫描结果；对至少两张医学图像分别进行特征提取，得到任意一张图像的图像特征，其中，图像特征包括：全局特征和局部特征；基于提取到的图像特征，确定目标部位的目标位移矢量场；基于目标部位的目标位移矢量场，将至少两张医学图像进行图像配准，得到图像配准结果。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：响应作用于操作界面上的输入指令，在操作界面上显示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；响应作用于操作界面上的图像配准指令，在操作界面上显示图像配准结果，其中，图像配准结果是基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准得到的，目标位移矢量场是基于提取到的图像特征确定的，提取到的图像特征包括：全局特征和局部特征，提取到的图像特征是对至少两张图像分别进行特征提取得到的。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：在虚拟现实VR设备或增强现实AR设备的呈现画面上展示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；驱动VR设备或AR设备，渲染展示图像配准结果。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：通过调用第一接口，获取至少两张图像，其中，第一接口包括第一参数，第一参数的参数值为至少两张图像，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；通过调用第二接口，输出图像配准结果，其中，第二接口包括第二参数，第二参数的参数值为图像配准结果。
采用本公开实施例，获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果，实现了提高图像配准准确度的目的。容易注意到的是，可以对至少两张图像分别进行特征提取，得到包含有全局特征和局部特征的图像特征，可以根据全局特征和局部特征扩大图像的感受野，以便得到更为准确的目标位移矢量场，可以根据目标位移矢量场将至少两张图像进行配准，从而得到准确度较高的图像配准结果，进而解决了相关技术中对配准任务进行处理的准确度较低的技术问题。
本领域普通技术人员可以理解，图14所示的结构仅为示意，计算机终端也可以是智能手机(如Android手机、iOS手机等)、平板电脑、掌上电脑以及移动互联网设备(MobileInternetDevices，MID)、PAD等终端设备。图14并不对上述电子装置的结构造成限定。例如，计算机终端A还可包括比图14中所示更多或者更少的组件(如网络接口、显示装置等)，或者具有与图14所示不同的配置。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令终端设备相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：闪存盘、只读存储器(Read-Only Memory，ROM)、随机存取器(Random Access Memory，RAM)、磁盘或光盘等。
实施例12
本公开的实施例还提供了一种计算机可读存储介质。可选地，在本实施例中，上述计算机可读存储介质可以用于保存上述实施例1所提供的图像配准方法所执行的程序代码。
计算机可读存储介质计算机存储介质也可以称作计算机存储介质。其可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了可读程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读存储介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
计算机可读存储介质中包含的程序代码可以用任何适当的介质传输，包括但不限于无线、有线、光缆、射频等等，或者上述的任意合适的组合。
可选地，在本实施例中，上述计算机可读存储介质可以位于AR/VR设备网络中AR/VR设备终端群中的任意一个计算机终端中，或者位于移动终端群中的任意一个移动终端中。
可选地，在本实施例中，计算机可读存储介质被设置为存储用于执行以下步骤的程序代码：获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果。
可选地，上述存储介质还被设置为存储用于执行以下步骤的程序代码：基于至少两张图像中的第一图像对应的位移矢量场对第一图像进行变形，得到变形图像；对变形图像和至少两张图像中的第二图像分别进行特征提取，得到任意一张图像的图像特征，其中，第二图像用于表征至少两张图像中除第一图像之外的图像。
可选地，上述存储介质还被设置为存储用于执行以下步骤的程序代码：对变形图像和第二图像进行特征提取，得到全局特征和局部特征；对全局特征和局部特征进行叠加，得到图像特征。
可选地，上述存储介质还被设置为存储用于执行以下步骤的程序代码：获取提取到的图像特征的点积，得到目标代价体积；对目标代价体积进行凸优化处理，得到目标位移矢量场。
可选地，上述存储介质还被设置为存储用于执行以下步骤的程序代码：对目标代价体积进行参数最小化处理，得到初始位移矢量场；对初始位移矢量场进行平均池化操作，得到目标位移矢量场。
可选地，上述存储介质还被设置为存储用于执行以下步骤的程序代码：分别对至少两张图像进行多次下采样，得到任意一张图像的多个下采样图像，其中，不同下采样图像的分辨率不同；基于至少两张图像中的第一图像的下采样图像对应的采样位移矢量场对第一图像的下采样图像进行变形，得到采样变形图像；对采样变形图像和至少两张图像中的第二图像的下采样图像分别进行特征提取，得到任意一张图像的下采样图像的下采样图像特征；对多个下采样图像特征进行汇总，得到任意一张图像的图像特征。
可选地，上述存储介质还被设置为存储用于执行以下步骤的程序代码：获取任意一张图像的下采样图像的下采样图像特征的点积，得到采样代价体积；基于采样代价体积进行凸优化处理，得到采样位移矢量场；对多个采样位移矢量场进行叠加，得到目标位移矢量场。
可选地，上述存储介质还被设置为存储用于执行以下步骤的程序代码：在第一图像的下采样图像是预设下采样图像的情况下，确定第一图像的下采样图像对应的采样位移矢量场为预设位移矢量场，其中，预设下采样图像用于表征第一图像的最大尺度的下采样图像；在第一图像的下采样图像不是预设下采样图像的情况下，对第一图像的下一尺度的下采样图像对应的采样位移矢量场进行上采样处理，得到第一图像的下采样图像对应的采样位移矢量场。
可选地，在本实施例中，计算机可读存储介质被设置为存储用于执行以下步骤的程序代码：获取至少两张医学图像，其中，任意一张医学图像中包含了通过计算机断层扫描技术对同一个生物对象的目标部位在不同条件下的扫描结果；对至少两张医学图像分别进行特征提取，得到任意一张图像的图像特征，其中，图像特征包括：全局特征和局部特征；基于提取到的图像特征，确定目标部位的目标位移矢量场；基于目标部位的目标位移矢量场，将至少两张医学图像进行图像配准，得到图像配准结果。
可选地，在本实施例中，计算机可读存储介质被设置为存储用于执行以下步骤的程序代码：响应作用于操作界面上的输入指令，在操作界面上显示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；响应作用于操作界面上的图像配准指令，在操作界面上显示图像配准结果，其中，图像配准结果是基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准得到的，目标位移矢量场是基于提取到的图像特征确定的，提取到的图像特征包括：全局特征和局部 特征，提取到的图像特征是对至少两张图像分别进行特征提取得到的。
可选地，在本实施例中，计算机可读存储介质被设置为存储用于执行以下步骤的程序代码：在虚拟现实VR设备或增强现实AR设备的呈现画面上展示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；驱动VR设备或AR设备，渲染展示图像配准结果。
可选地，在本实施例中，计算机可读存储介质被设置为存储用于执行以下步骤的程序代码：通过调用第一接口，获取至少两张图像，其中，第一接口包括第一参数，第一参数的参数值为至少两张图像，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果；通过调用第二接口，输出图像配准结果，其中，第二接口包括第二参数，第二参数的参数值为图像配准结果。
采用本公开实施例，获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果，实现了提高图像配准准确度的目的。容易注意到的是，可以对至少两张图像分别进行特征提取，得到包含有全局特征和局部特征的图像特征，可以根据全局特征和局部特征扩大图像的感受野，以便得到更为准确的目标位移矢量场，可以根据目标位移矢量场将至少两张图像进行配准，从而得到准确度较高的图像配准结果，进而解决了相关技术中对配准任务进行处理的准确度较低的技术问题。
上述本公开实施例序号仅仅为了描述，不代表实施例的优劣。
在本公开的上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
在本公开所提供的几个实施例中，应该理解到，所揭露的技术内容，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显 示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM，简称为Read-Only Memory)、随机存取存储器(RAM，简称为Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅是本公开的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本公开原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本公开的保护范围。
工业实用性
本公开实施例提供的方案可以应用于图像配准的过程中，获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；对至少两张图像分别进行特征提取，得到任意一张图像的图像特征；基于提取到的图像特征，确定待分析的对象的目标位移矢量场；基于待分析的对象的目标位移矢量场，将至少两张图像进行图像配准，得到图像配准结果，进而解决了相关技术中图像配准方法的准确度较低，应用场景受限的技术问题。

Claims (20)

  1. 一种图像配准方法，其特征在于，包括：
    获取至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；
    对所述至少两张图像分别进行特征提取，得到所述任意一张图像的图像特征；
    基于提取到的图像特征，确定所述待分析的对象的目标位移矢量场；
    基于所述待分析的对象的目标位移矢量场，将所述至少两张图像进行图像配准，得到图像配准结果。
  2. 根据权利要求1所述的方法，其特征在于，对所述至少两张图像分别进行特征提取，得到所述任意一张图像的图像特征，包括：
    基于所述至少两张图像中的第一图像对应的位移矢量场，对所述第一图像进行变形，得到变形图像；
    对所述变形图像和所述至少两张图像中的第二图像分别进行特征提取，得到所述任意一张图像的图像特征，其中，所述第二图像用于表征所述至少两张图像中除所述第一图像之外的图像。
  3. 根据权利要求2所述的方法，其特征在于，对所述变形图像和所述至少两张图像中的第二图像分别进行特征提取，得到所述任意一张图像的图像特征，包括：
    对所述变形图像和所述第二图像进行特征提取，得到全局特征和局部特征；
    对所述全局特征和所述局部特征进行叠加，得到所述图像特征。
  4. 根据权利要求1所述的方法，其特征在于，基于提取到的图像特征，确定所述待分析的对象的目标位移矢量场，包括：
    获取所述提取到的图像特征的点积，得到目标代价体积；
    对所述目标代价体积进行凸优化处理，得到所述目标位移矢量场。
  5. 根据权利要求4所述的方法，其特征在于，对所述目标代价体积进行凸优化处理，得到目标位移矢量场，包括：
    对所述目标代价体积进行参数最小化处理，得到初始位移矢量场；
    对所述初始位移矢量场进行平均池化操作，得到所述目标位移矢量场。
  6. 根据权利要求1至5中任意一项所述的方法，其特征在于，对所述至少两张图像分别进行特征提取，得到所述任意一张图像的图像特征，包括：
    分别对所述至少两张图像进行多次下采样，得到所述任意一张图像的多个下采样图像，其中，不同所述下采样图像的分辨率不同；
    基于所述至少两张图像中的第一图像的下采样图像对应的采样位移矢量场，对所述第一图像的下采样图像进行变形，得到采样变形图像；
    对所述采样变形图像和所述至少两张图像中的第二图像的下采样图像分别进行特征提取，得到所述任意一张图像的下采样图像的下采样图像特征；
    对多个所述下采样图像特征进行汇总，得到所述任意一张图像的图像特征。
  7. 根据权利要求6所述的方法，其特征在于，基于提取到的图像特征，确定所述待分析的对象的目标位移矢量场，包括：
    获取所述任意一张图像的下采样图像的下采样图像特征的点积，得到采样代价体积；
    对所述采样代价体积进行凸优化处理，得到采样位移矢量场；
    对多个所述采样位移矢量场进行叠加，得到所述目标位移矢量场。
  8. 根据权利要求6所述的方法，其特征在于，所述方法还包括：
    在所述第一图像的下采样图像是预设下采样图像的情况下，确定所述第一图像的下采样图像对应的采样位移矢量场为预设位移矢量场，其中，所述预设下采样图像用于表征所述第一图像的最大尺度的下采样图像；
    在所述第一图像的下采样图像不是所述预设下采样图像的情况下，对所述第一图像的下一尺度的下采样图像对应的采样位移矢量场进行上采样处理，得到所述第一图像的下采样图像对应的采样位移矢量场。
  9. 一种图像配准方法，其特征在于，包括：
    获取至少两张医学图像，其中，任意一张医学图像中包含了通过计算机断层扫描技术对同一个生物对象的目标部位在不同条件下的扫描结果；
    对所述至少两张医学图像分别进行特征提取，得到所述任意一张图像的图像特征；
    基于提取到的图像特征，确定所述目标部位的目标位移矢量场；
    基于所述目标部位的目标位移矢量场，将所述至少两张医学图像进行图像配准，得到图像配准结果。
  10. 一种图像配准方法，其特征在于，包括：
    响应作用于操作界面上的输入指令，在所述操作界面上显示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；
    响应作用于所述操作界面上的图像配准指令，在所述操作界面上显示图像配准结果，其中，所述图像配准结果是基于所述待分析的对象的目标位移矢量场，将所述至少两张图像进行图像配准得到的，所述目标位移矢量场是基于提取到的图像特征确定的，所述提取到的图像特征是对所述至少两张图像分别进行特征提取得到的。
  11. 一种图像配准方法，其特征在于，包括：
    在虚拟现实VR设备或增强现实AR设备的呈现画面上展示至少两张图像，其中，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；
    对所述至少两张图像分别进行特征提取，得到所述任意一张图像的图像特征；
    基于提取到的图像特征，确定所述待分析的对象的目标位移矢量场；
    基于所述待分析的对象的目标位移矢量场，将所述至少两张图像进行图像配准，得到图像配准结果；
    驱动所述VR设备或所述AR设备，渲染展示所述图像配准结果。
  12. 根据权利要求11所述的方法，其特征在于，对所述至少两张图像分别进行特征提取，得到所述任意一张图像的图像特征，包括：
    基于所述至少两张图像中的第一图像对应的位移矢量场，对所述第一图像进行变形，得到变形图像；
    对所述变形图像和所述至少两张图像中的第二图像分别进行特征提取，得到所述任意一张图像的图像特征，其中，所述第二图像用于表征所述至少两张图像中除所述第一图像之外的图像。
  13. 根据权利要求12所述的方法，其特征在于，对所述变形图像和所述至少两张图像中的第二图像分别进行特征提取，得到所述任意一张图像的图像特征，包括：
    对所述变形图像和所述第二图像进行特征提取，得到全局特征和局部特征；
    对所述全局特征和所述局部特征进行叠加，得到所述图像特征。
  14. 根据权利要求11所述的方法，其特征在于，基于提取到的图像特征，确定所述待分析的对象的目标位移矢量场，包括：
    获取所述提取到的图像特征的点积，得到目标代价体积；
    对所述目标代价体积进行凸优化处理，得到所述目标位移矢量场。
  15. 一种图像配准方法，其特征在于，包括：
    通过调用第一接口，获取至少两张图像，其中，所述第一接口包括第一参数，所述第一参数的参数值为所述至少两张图像，任意一张图像中包含了对待分析的对象在不同条件下的拍摄结果；
    对所述至少两张图像分别进行特征提取，得到所述任意一张图像的图像特征；
    基于提取到的图像特征，确定所述待分析的对象的目标位移矢量场；
    基于所述待分析的对象的目标位移矢量场，将所述至少两张图像进行图像配准，得到图像配准结果；
    通过调用第二接口，输出所述图像配准结果，其中，所述第二接口包括第二参数，所述第二参数的参数值为所述图像配准结果。
  16. 一种电子设备，其特征在于，包括：
    存储器，被设置为存储有可执行程序；
    处理器，被设置为运行所述程序，其中，所述程序运行时执行权利要求1 至15中任意一项所述的方法。
  17. 一种计算机程序产品，其中，包括计算机程序，所述计算机程序在被处理器执行时实现权利要求1至15任意一项所述的方法。
  18. 一种计算机程序产品，其中，包括非易失性计算机可读存储介质，所述非易失性计算机可读存储介质存储计算机程序，所述计算机程序被处理器执行时实现权利要求1至15任意一项所述的方法。
  19. 一种计算机程序，其中，所述计算机程序被处理器执行时实现权利要求1至15任意一项所述的方法。
  20. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质包括存储的可执行程序，其中，在所述可执行程序运行时控制所述计算机可读存储介质所在设备执行权利要求1至15中任意一项所述的方法。
PCT/CN2024/095576 2023-06-26 2024-05-27 图像配准方法、电子设备以及计算机可读存储介质 Ceased WO2025001689A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310767446.6A CN116823905B (zh) 2023-06-26 2023-06-26 图像配准方法、电子设备以及计算机可读存储介质
CN202310767446.6 2023-06-26

Publications (1)

Publication Number Publication Date
WO2025001689A1 true WO2025001689A1 (zh) 2025-01-02

Family

ID=88142419

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/095576 Ceased WO2025001689A1 (zh) 2023-06-26 2024-05-27 图像配准方法、电子设备以及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN116823905B (zh)
WO (1) WO2025001689A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116823905B (zh) * 2023-06-26 2024-06-28 阿里巴巴达摩院(杭州)科技有限公司 图像配准方法、电子设备以及计算机可读存储介质
CN117354471B (zh) * 2023-12-05 2024-04-09 深圳市微浦技术有限公司 多摄像头协同监控方法、装置、设备及存储介质
CN120782830B (zh) * 2025-09-05 2025-11-07 湖南大学 一种基于大模型鲁棒特征的医学图像配准方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113724307A (zh) * 2021-09-02 2021-11-30 深圳大学 基于特征自校准网络的图像配准方法、装置及相关组件
CN115761594A (zh) * 2022-11-28 2023-03-07 南昌航空大学 一种基于全局与局部耦合的光流计算方法
CN116823905A (zh) * 2023-06-26 2023-09-29 阿里巴巴达摩院(杭州)科技有限公司 图像配准方法、电子设备以及计算机可读存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0913930D0 (en) * 2009-08-07 2009-09-16 Ucl Business Plc Apparatus and method for registering two medical images
KR102233427B1 (ko) * 2014-04-14 2021-03-29 삼성전자주식회사 의료 영상 정합 방법 및 장치
US10304220B2 (en) * 2016-08-31 2019-05-28 International Business Machines Corporation Anatomy segmentation through low-resolution multi-atlas label fusion and corrective learning
CN110517309A (zh) * 2019-07-19 2019-11-29 沈阳工业大学 一种基于卷积神经网络的单目深度信息获取方法
CN111681205B (zh) * 2020-05-08 2023-04-07 上海联影智能医疗科技有限公司 图像分析方法、计算机设备和存储介质
CN116310618B (zh) * 2022-09-08 2025-10-03 华中科技大学 多模图像的配准网络训练装置及其训练方法以及配准方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113724307A (zh) * 2021-09-02 2021-11-30 深圳大学 基于特征自校准网络的图像配准方法、装置及相关组件
CN115761594A (zh) * 2022-11-28 2023-03-07 南昌航空大学 一种基于全局与局部耦合的光流计算方法
CN116823905A (zh) * 2023-06-26 2023-09-29 阿里巴巴达摩院(杭州)科技有限公司 图像配准方法、电子设备以及计算机可读存储介质

Also Published As

Publication number Publication date
CN116823905A (zh) 2023-09-29
CN116823905B (zh) 2024-06-28

Similar Documents

Publication Publication Date Title
EP3786890B1 (en) Method and apparatus for determining pose of image capture device, and storage medium therefor
WO2025001689A1 (zh) 图像配准方法、电子设备以及计算机可读存储介质
US10210613B2 (en) Multiple landmark detection in medical images based on hierarchical feature learning and end-to-end training
CN111091521B (zh) 图像处理方法及装置、电子设备和计算机可读存储介质
CN110766735B (zh) 一种图像的匹配方法、装置、设备及存储介质
KR102108418B1 (ko) 재구성된 영상군에 기초한 영상 제공 방법 및 이를 이용한 장치
TW202203854A (zh) 圖像處理方法及電子設備、電腦可讀儲存介質
WO2020048484A1 (zh) 超分辨图像重建方法、装置、终端和存储介质
CN113256529A (zh) 图像处理方法、装置、计算机设备及存储介质
WO2024146649A1 (zh) 图像处理方法、存储介质及计算机终端
CN113887319A (zh) 三维姿态的确定方法、装置、电子设备及存储介质
CN116188689A (zh) 辐射场的处理方法、存储介质以及计算机终端
CN114998935A (zh) 图像处理方法、装置、计算机设备及存储介质
CN115376198A (zh) 视线方向估计方法、装置、电子设备、介质及程序产品
CN113470112A (zh) 图像处理方法、装置、存储介质以及终端
CN115115724A (zh) 图像处理方法、装置、计算机设备及存储介质
US12361645B2 (en) Mesh transformation with efficient depth reconstruction and filtering in passthrough augmented reality (AR) systems
CN112190331A (zh) 手术导航信息的确定方法、装置及系统、电子装置
CN115731326B (zh) 虚拟角色生成方法及装置、计算机可读介质和电子设备
CN115527166A (zh) 图像处理方法、计算机可读存储介质以及电子设备
JP2019025240A (ja) 画像処理装置、画像処理方法、画像処理システム及びプログラム
CN117876808A (zh) 一种模型训练方法及装置
CN112991179B (zh) 用于输出信息的方法、装置、设备以及存储介质
CN110415239B (zh) 图像处理方法、装置、设备、医疗电子设备以及介质
US12456267B2 (en) Automatic registration of landmarks for augmented reality assisted surgery

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24830337

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE