CN113689477A - Multi-modality medical image registration method, system, readable storage medium and device - Google Patents
Multi-modality medical image registration method, system, readable storage medium and device Download PDFInfo
- Publication number
- CN113689477A CN113689477A CN202111056802.0A CN202111056802A CN113689477A CN 113689477 A CN113689477 A CN 113689477A CN 202111056802 A CN202111056802 A CN 202111056802A CN 113689477 A CN113689477 A CN 113689477A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- image
- registration
- medical image
- modal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Abstract
The application relates to a multi-modality medical image registration method, system, readable storage medium and device, acquiring a multi-modality medical image and a target coordinate system, wherein the target coordinate system comprises an anatomical coordinate system and/or an image coordinate system; the target coordinate system is an image coordinate system where an anatomical coordinate system or a modal image of a subject is located, and is not a fixed world coordinate system which is generally adopted, multi-modal medical images are registered under the target coordinate system, cross-plane three-dimensional registration can be achieved in the target coordinate system which is different from the world coordinate system, and the problem that the related technology is difficult to achieve three-dimensional registration and point registration and automatic registration are difficult to distinguish scanning directions is solved, so that the accuracy of medical image registration is improved.
Description
Technical Field
The present application relates to the field of medical imaging technologies, and in particular, to a method, a system, a readable storage medium, and a device for multi-modality medical image registration.
Background
Medical imaging devices are now widely used in clinical diagnosis and therapy. Due to different principles of various device modalities, the image modalities can be divided into an anatomical structure image and a functional image, the anatomical structure image includes, for example, a CT (Computed Tomography) image, an MR (Magnetic Resonance) image, an ultrasound image, and the like, the functional image includes a PET (Positron Emission Tomography) image, a SPECT (Single-Photon Emission Tomography) image, and the like, and the images of the plurality of modalities are fused into one image, so that information from various aspects of a human body can be intuitively expressed in the same image, conditions of the anatomical structure, the functional metabolism, and the like of the human body can be intuitively provided, and the purposes of accurate diagnosis, accurate positioning of a lesion, improvement of a radiotherapy plan, and the like can be achieved. One of the key factors affecting the accurate fusion of multi-modal medical images is the image registration technique.
At present, no effective solution is provided for the problems of low accuracy of medical image registration and difficulty in directly realizing cross-plane registration in the related technology.
Disclosure of Invention
Based on this, it is necessary to provide a multi-modality medical image registration method, system, readable storage medium and device for solving the problems of low accuracy of medical image registration and difficulty in directly realizing cross-plane registration in the related art.
In a first aspect, the present application provides a multi-modality medical image registration method, comprising the steps of:
acquiring a multi-modal medical image and a target coordinate system, wherein the target coordinate system comprises an anatomical coordinate system and/or an image coordinate system;
and registering the multi-modal medical image under a target coordinate system to obtain a registered modal image.
In one embodiment, registering the multi-modality medical images in the target coordinate system comprises the steps of:
identifying a target part according to the multi-modal medical image;
if the identified target part belongs to a preset target type, registering the multi-modal medical image by adopting an anatomical coordinate system;
conversely, the multi-modality medical images are registered using either the anatomical coordinate system or the image coordinate system.
In one embodiment, the method further comprises the following steps when registering the multi-modal medical images by using the anatomical coordinate system:
the orientations of the anatomical coordinate systems are respectively labeled on the multi-modal medical images.
In one embodiment, when registering the multi-modal medical image in the target coordinate system, the registering operation includes at least one of rigid transformation, nonlinear transformation, projective transformation and affine transformation, and each registering operation has a preset registering range corresponding to the target part.
In one embodiment, after obtaining the registered modality image, the method further includes the following steps:
and carrying out secondary registration on the registered modal image in a target coordinate system or a world coordinate system.
In one embodiment, the multi-modality medical image registration method further comprises the steps of:
and fusing the registered modal images to obtain a multi-modal registered fused image.
In a second aspect, the present application provides a multi-modality medical image registration system, comprising:
a data acquisition unit for acquiring a multi-modality medical image and a target coordinate system, wherein the target coordinate system comprises an anatomical coordinate system and/or an image coordinate system;
and the image registration unit is used for registering the multi-modal medical image under the target coordinate system to obtain a registered modal image.
In one embodiment, the image registration unit is further configured to perform target site recognition based on the multi-modality medical image; if the identified target part belongs to a preset target type, registering the multi-modal medical image by adopting an anatomical coordinate system; conversely, the multi-modality medical images are registered using either the anatomical coordinate system or the image coordinate system.
In one embodiment, the image registration unit is further configured to label the positions of the anatomical coordinate systems on the multi-modal medical image, respectively.
In one embodiment, when the image registration unit registers the multi-modal medical image in the target coordinate system, the registration operation includes at least one of rigid transformation, nonlinear transformation, projective transformation and affine transformation, and each registration operation has a registration preset range corresponding to the target part.
In one embodiment, the image registration unit is further configured to perform secondary registration on the registered modality image in the target coordinate system or the world coordinate system after obtaining the registered modality image.
In one embodiment, the multi-modal medical image registration system further includes an image fusion unit, configured to fuse the registered modality images to obtain a multi-modal registered fusion image.
In a third aspect, the present application provides a readable storage medium having stored thereon an executable program, which when executed by a processor, performs the steps of any of the above-described multi-modality medical image registration methods.
In a fourth aspect, the present application provides a multi-modality medical image registration apparatus, which includes a memory and a processor, wherein the memory stores an executable program, and the processor implements the steps of the multi-modality medical image registration method when executing the executable program.
Compared with the related art, according to the multi-modality medical image registration method, system, readable storage medium and device provided by the application, the target coordinate system is an image coordinate system where an anatomical coordinate system or a modal image of a subject is located, and a fixed world coordinate system which is not generally adopted is adopted, the multi-modality medical image is registered under the target coordinate system, cross-plane three-dimensional registration can be achieved in the target coordinate system which is different from the world coordinate system, the problems that the three-dimensional registration is difficult to achieve in the related art, and the scanning position is difficult to distinguish between point registration and automatic registration are solved, and therefore the accuracy of medical image registration is improved.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic diagram of an exemplary medical device 100 in one embodiment;
FIG. 2 is a schematic diagram of exemplary hardware and/or software components of an exemplary computing device 200 on which processing engine 140 is implemented, in one embodiment;
FIG. 3 is a diagram of exemplary hardware and/or software components of an exemplary mobile device 300 on which terminal 130 may be implemented, in one embodiment;
FIG. 4 is a flow diagram of a multi-modality medical image registration method in one embodiment;
FIG. 5 is a schematic structural diagram of a multi-modality medical image registration system in one embodiment;
fig. 6 is a schematic structural diagram of a multi-modality medical image registration system in another embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference herein to "a plurality" means greater than or equal to two. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. Reference herein to the terms "first," "second," and the like, are merely distinguishing between similar items and not necessarily referring to a particular ordering for the items.
Fig. 1 is a schematic diagram of an exemplary medical device 100 for multi-modality medical image registration, according to an embodiment. Referring to fig. 1, a medical device 100 may include a scanner 110, a network 120, one or more terminals 130, a processing engine 140, and a memory 150. All components in the medical device 100 may be interconnected by a network 120.
The scanner 110 may scan an object and generate scanned image data related to the scanned object. In some embodiments, the scanner 110 may be a medical imaging device, such as a CT device, a PET device, a SPECT device, an MRI device, and the like, or any combination thereof (e.g., a PET-CT device or a CT-MRI device).
Reference to "image" in this application may refer to a 2D image, a 3D image, a 4D image, and/or any related data, which is not intended to limit the scope of this application. Various modifications and alterations will occur to those skilled in the art in light of the present disclosure.
The scanner 110 may include a support assembly 111, a detector assembly 112, a scanning bed 114, an electronics module 115, and a cooling assembly 116.
The support assembly 111 may support one or more components of the scanner 110, such as the detector assembly 112, the electronics module 115, the cooling assembly 116, and the like. In some embodiments, the support assembly 111 may include a main frame, a frame base, a front cover, and a rear cover (not shown). The front cover plate may be coupled to the frame base. The front cover plate may be perpendicular to the chassis base. The main frame may be mounted to a side of the front cover. The mainframe may include one or more support brackets to house the detector assembly 112 and/or the electronics module 115. The mainframe may include a circular opening (e.g., detection region 113) to accommodate the subject. In some embodiments, the opening of the main chassis may be other shapes, including, for example, oval. The rear cover may be mounted to a side of the main frame opposite the front cover. The frame base may support a front cover plate, a main frame, and/or a rear cover plate. In some embodiments, the scanner 110 may include a housing to cover and protect the mainframe.
The detector assembly 112 may detect radiation events (e.g., X-ray signals, etc.) emitted from the detection region 113. In some embodiments, the detector assembly 112 may receive radiation (e.g., X-ray signals, etc.) and generate electrical signals. The detector assembly 112 may include one or more detector cells. One or more detector units may be packaged to form a detector block. One or more detector blocks may be packaged to form a detector box. One or more detector cassettes may be mounted to form a detection ring. One or more detection rings may be mounted to form a detector module.
The scanning bed 114 may support and position the subject at a desired location in the examination region 113. In some embodiments, the subject may be on a scanning couch 114. The scanning bed 114 may be moved and brought to a desired position in the detection region 113. In some embodiments, the scanner 110 may have a relatively long axial field of view, such as a 2 meter long axial field of view. Accordingly, the scanning bed 114 may be movable along the axial direction over a wide range (e.g., greater than 2 meters).
The electronic module 115 may collect and/or process the electrical signals generated by the detector assembly 112. The electronic module 115 may include one or a combination of an adder, a multiplier, a subtractor, an amplifier, a driver circuit, a differential circuit, an integrating circuit, a counter, a filter, an analog-to-digital converter, a lower limit detection circuit, a constant coefficient discriminator circuit, a time-to-digital converter, a coincidence circuit, and the like. The electronics module 115 may convert analog signals related to the energy of the radiation received by the detector assembly 112 into digital signals. The electronics module 115 may compare the plurality of digital signals, analyze the plurality of digital signals, and determine image data from the energy of the radiation received in the detector assembly 112. In some embodiments, if the detector assembly 112 has a large axial field of view (e.g., 0.75 meters to 2 meters), the electronics module 115 may have a high data input rate from multiple detector channels. For example, the electronic module 115 may process billions of events per second. In some embodiments, the data input rate may be related to the number of detector cells in the detector assembly 112.
The cooling assembly 116 may generate, transfer, transport, conduct, or circulate a cooling medium through the scanner 110 to absorb heat generated by the scanner 110 during imaging. In some embodiments, the cooling assembly 116 may be fully integrated into the scanner 110 and become part of the scanner 110. In some embodiments, the cooling assembly 116 may be partially integrated into the scanner 110 and associated with the scanner 110. The cooling assembly 116 may allow the scanner 110 to maintain a suitable and stable operating temperature (e.g., 25 ℃, 30 ℃, 35 ℃, etc.). In some embodiments, the cooling assembly 116 may control the temperature of one or more target components of the scanner 110. The target components may include the detector assembly 112, the electronics module 115, and/or any other components that generate heat during operation. The cooling medium may be in a gaseous state, a liquid state (e.g., water), or a combination of one or more thereof. In some embodiments, the gaseous cooling medium may be air.
The scanner 110 may scan a subject located within its examination region and generate a plurality of imaging data relating to the subject. In the present application, "subject" and "object" are used interchangeably. By way of example only, the subject may include a scan target, an artificial object, and the like. In another embodiment, the subject may include scanning a particular portion, organ, and/or tissue of the target. For example, the subject may include the head, brain, neck, body, shoulders, arms, chest, heart, stomach, blood vessels, soft tissue, knees, feet, or other parts, or the like, or any combination thereof. In the present application, the subject is primarily the heart.
The network 120 may include any suitable network that can assist the medical devices 100 in exchanging information and/or data. In some embodiments, one or more components of the medical device 100 (e.g., the scanner 110, the terminal 130, the processing engine 140, the memory 150, etc.) may communicate information and/or data with one or more other components of the medical device 100 via the network 120. For example, the processing engine 140 may obtain image data from the scanner 110 via the network 120. As another example, processing engine 140 may obtain user instructions from terminal 130 via network 120. The one or more terminals 130 include a mobile device 131, a tablet computer 132, a laptop computer 133, the like, or any combination thereof. In some embodiments, mobile device 131 may include a smart home device, a wearable device, a mobile device, a virtual reality device, an augmented reality device, and the like, or any combination thereof.
The processing engine 140 may process data and/or information obtained from the scanner 110, the terminal 130, and/or the memory 150. In some embodiments, processing engine 140 may be a single server or a group of servers. The server groups may be centralized or distributed. In some embodiments, the processing engine 140 may be local or remote. For example, the processing engine 140 may access information and/or data stored in the scanner 110, the terminal 130, and/or the memory 150 through the network 120. As another example, the processing engine 140 may be directly connected to the scanner 110, the terminal 130, and/or the memory 150 to access stored information and/or data. In some embodiments, processing engine 140 may be implemented on a cloud platform. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an interconnected cloud, a multi-cloud, and the like, or any combination thereof. In some embodiments, processing engine 140 may be implemented by computing device 200 having one or more components shown in FIG. 2.
In some embodiments, the memory 150 may be connected to the network 120 for communication with one or more other components in the medical device 100 (e.g., the processing engine 140, the terminal 130, etc.). One or more components in the medical device 100 may access data or instructions stored in the memory 150 through the network 120. In some embodiments, the memory 150 may be directly connected to or in communication with one or more other components in the medical device 100 (e.g., the processing engine 140, the terminal 130, etc.). In some embodiments, memory 150 may be part of processing engine 140.
FIG. 2 is a schematic diagram of exemplary hardware and/or software components of an exemplary computing device 200 on which processing engine 140 may be implemented, for one embodiment. As shown in FIG. 2, computing device 200 may include an internal communication bus 210, a processor (processor)220, a Read Only Memory (ROM)230, a Random Access Memory (RAM)240, a communication port 250, input/output components 260, a hard disk 270, and a user interface device 280.
Fig. 3 is a diagram of exemplary hardware and/or software components of an exemplary mobile device 300 on which terminal 130 may be implemented, for one embodiment. As shown in fig. 3, mobile device 300 may include antenna 310, display 320, Graphics Processing Unit (GPU)330, Central Processing Unit (CPU)340, input output unit (I/O)350, memory 360, and storage 390. In some embodiments, any other suitable component may also be included in mobile device 300, including but not limited to a system bus or a controller (not shown). In some embodiments, a mobile operating system 370 (e.g., iOS, Android, Windows Phone, etc.) and one or more application programs 380 may be loaded from storage 390 into memory 360 for execution by CPU 340. The application 380 may include a browser or any other suitable mobile application for receiving and rendering information related to image processing or other information from the processing engine 140. User interaction with the information flow may be enabled through the I/O350 and provided to the processing engine 140 and/or other components of the medical device 100 via the network 120.
To implement the various modules, units and their functionality described in this application, a computer hardware platform may be used as the hardware platform(s) for one or more of the elements described herein. A computer with user interface elements may be used as a Personal Computer (PC) or any other type of workstation or terminal device. The computer may also act as a server if suitably programmed. A multi-modality medical image registration method, system, etc. may be implemented in the medical device 100.
Flow charts are used herein to illustrate operations performed by methods according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in the exact order in which they are performed. Rather, various steps may be processed in reverse order or simultaneously. Meanwhile, other operations are added to or removed from these processes.
Referring to fig. 4, a flowchart of a multi-modality medical image registration method according to an embodiment of the present application is shown. The multi-modality medical image registration method in this embodiment comprises the steps of:
step S410: acquiring a multi-modal medical image and a target coordinate system; the target coordinate system comprises an anatomical coordinate system and/or an image coordinate system;
in this step, acquiring the multi-modality medical image specifically includes: acquiring a first modality image of a scanned object, the first modality image including an anatomical image of the scanned object; acquiring a second modality image of the scanned object, wherein the second modality image comprises a functional image of the scanned object, the first modality image comprises a CT image, an MR image, an ultrasonic image and the like, and the second modality image comprises a PET image, a SPECT image and the like;
the anatomical image is an image that can acquire an anatomical structure of a scanning object, and the functional image is an image that can reflect functional metabolic information about physiology, biochemistry and the like of the scanning object.
In this step, the multi-modal medical images are different medical images obtained under different imaging mechanisms, and can provide different information, for example, the multi-modal medical images can be images obtained by scanning the same part of the subject with medical scanning devices of different modalities, such as CT scan images, PET scan images, MRI scan images, and the like; the target coordinate system may be an anatomical coordinate system for the subject and/or an image coordinate system in which the modality image is located; the anatomical coordinate system may comprise three-dimensional orthogonal coordinate axes when scanning the subject on the medical scanning device: the head-foot direction (H, F), the left-right direction (R, L), and the anterior-posterior direction (A, P) are different image coordinate systems from the anatomical coordinate system in which the modality images are located, since the orientations used in acquiring data in a medical scan are not generally true, and are generally oblique to the subject.
The multi-modal medical images can be obtained from the memory 150, the memory 150 can be provided with a database for storing various modal images, and the data of the modal images can also be obtained from the electronic module 115 after scanning, and the specific process is as follows: the subject may be placed on a bed 114 of the medical device scanner 110, enter a detection region 113 of the scanner 110 and perform a scan shot, acquire data directly from the electronic module 115, and acquire images of various modalities through image algorithms.
Specifically, the medical device may obtain scan image data during scanning, which includes multiple types, such as computed tomography image data, X-ray image data, magnetic resonance image data, positron emission tomography image data, multi-modality fused image data, and the like, and may obtain scan image data of a corresponding modality through a scan imaging modality device, such as a CT device, an MR device, or a PET device, and the scan image data may also be read from a memory of the corresponding device.
Further, after obtaining the images of various modalities, the images may be preprocessed, for example, the target region in which registration is required is segmented to facilitate subsequent processing, and in the preprocessing, techniques such as an image segmentation algorithm and deep learning may be used.
Step S420: registering the multi-modal medical image under a target coordinate system to obtain a registered modal image;
in this step, the multi-modal medical image may be subjected to various registration operations under the target coordinate system, and the adjusted orientation and direction of the modal image are referred to the target coordinate system during the registration process.
In this embodiment, the target coordinate system is an image coordinate system where an anatomical coordinate system or a modal image of a subject is located, and is not a fixed world coordinate system which is generally adopted, and the multi-modal medical image is registered under the target coordinate system, so that cross-plane three-dimensional registration can be realized in the target coordinate system different from the world coordinate system, and the problems that the three-dimensional registration is difficult to realize in the related art and the scanning orientation is difficult to distinguish in point registration and automatic registration are solved, thereby improving the accuracy of medical image registration.
It should be noted that the above multi-modality medical image registration method may be executed on a console of the medical device, or may be executed on a post-processing workstation of the medical device, or may be executed on a terminal device capable of communicating with the medical device, and is not limited to this, and may be modified according to the needs of the actual application.
In one embodiment, registering the multi-modality medical images in the target coordinate system comprises the steps of:
identifying a target part according to the multi-modal medical image;
if the identified target part belongs to a preset target type, registering the multi-modal medical image by adopting an anatomical coordinate system;
conversely, the multi-modality medical images are registered using either the anatomical coordinate system or the image coordinate system.
In this embodiment, before the registration, the target region recognition may be performed on the multi-modal medical image, and since the multi-modal medical image includes a plurality of modality images, the target region recognition may be performed using any one or more modality images of the multi-modal medical image. Judging whether the identified target part accords with a preset target type, if so, registering the multi-modal medical image by adopting an anatomical coordinate system, and enhancing the accuracy of registering the medical image containing the target part by identifying the target part; if the identified target part does not belong to the preset target type, the multi-modal medical image can be registered by selectively adopting an anatomical coordinate system or an image coordinate system, and the target coordinate system can be switched as required at any time in the registration process, so that the registration can be completed more quickly, accurately and conveniently.
Specifically, the target site may include various organs, soft tissues, blood vessels, bones, and the like.
Further, taking the target part as an organ as an example, when organ recognition is performed, various different image recognition algorithms for the organ can be adopted; the preset organ types can be heart, knee joint, shoulder joint, etc., and the organ types are generally acquired by oblique scanning, so that the anatomical coordinate system is suitable for registration. Meanwhile, if the identified organ does not belong to the preset organ type, the anatomical coordinate system or the image coordinate system in which the modal image is located can be freely selected for registration.
It should be noted that the type of organ is not limited to a specific organ, and may include a scanned part, such as the head or the whole body.
In one embodiment, when the anatomical coordinate system is used for registration, the method further comprises the following steps:
the orientations of the anatomical coordinate systems are respectively labeled on the multi-modal medical images.
In this embodiment, the multi-modal medical images are obtained according to the acquired scan data at respective scan orientations, the orientations of the anatomical coordinate system are different from the orientation of the image coordinate system where the modal image is located, and the orientations of the anatomical coordinate system are respectively labeled on the multi-modal medical images, so that the multi-modal medical images can be adjusted in various orientations on the anatomical coordinate system, and image registration is realized.
The orientations of the anatomical coordinate system include a head-foot orientation (H, F, respectively), an anterior-posterior orientation (A, P, respectively), and a left-right orientation (L, R, respectively) of the subject. The anatomical coordinate system is a spatial coordinate system in which the subject is scanned on the medical scanning device, that is, a spatial coordinate system in which the subject is placed on a scanning bed of the medical scanning device, and three dimensions of the spatial coordinate system may be a head and foot orientation, a front-back orientation, and a left-right orientation of the subject, and the front-back orientation and the left-right orientation are relative to the front of the subject. The image coordinate system of the modality image is related to the scanning position, the image coordinate system of the modality image is generally different from the anatomical coordinate system, but when some more common parts (such as organs or tissues which can be scanned in an orthostatic way) are scanned, the image coordinate system of the modality image may be the same as the anatomical coordinate system.
In one embodiment, when registering the multi-modality medical image in the target coordinate system, in addition to selecting the appropriate target coordinate system, manual registration or semi-automatic registration or automatic registration in the target coordinate system may be selected, that is: receiving a manual operation instruction, and registering the multi-modal medical image according to the manual operation instruction; or, manually extracting features in a man-machine interaction mode, and self-using the computer by using a related algorithm for registration; or, the multimodality medical images are automatically registered directly by the computer according to an anatomical part recognition algorithm of the target part.
In the embodiment, in a target image coordinate system, through a manual operation instruction, human-computer interaction is performed to extract features, and a correlation algorithm or an anatomical part recognition algorithm of a target part is combined to register a multi-modal medical image, so that the multi-modal medical image can be registered in the target coordinate system; the accuracy of the modal image registration can be improved by flexibly selecting and applying various modes.
In one embodiment, when registering the multi-modal medical image in the target coordinate system, the registering operation includes at least one of a rigid transformation, a non-linear transformation, a projective transformation, and an affine transformation, and each registering operation has a registration preset range corresponding to the target portion.
In the embodiment, the multi-modal medical image can be registered by using various modes such as rigid transformation, nonlinear transformation, projection transformation, affine transformation and the like, and a registration preset range corresponding to a target part is set for the registration operation in the image registration process, so that mismatch caused by excessive registration adjustment can be prevented, and the accuracy of image registration is improved.
By way of example of rigid transformation, the multi-modality medical images can be aligned in the target coordinate system by performing operations of registration such as translation and rotation to achieve registration. Because the target coordinate system is different from the world coordinate system, the modal image can realize the cross-plane three-dimensional registration through operations such as translation, rotation and the like, thereby improving the accuracy of medical image registration.
In one embodiment, after obtaining the registered modality images, the method further comprises the following steps:
and carrying out secondary registration on the registered modal image in a target coordinate system or a world coordinate system.
In this embodiment, after the multi-modal medical images are registered in the target coordinate system, the registered multi-modal medical images can be secondarily registered in the target coordinate system or the world coordinate system, so as to further improve the registration accuracy.
Specifically, multiple registration modes such as manual registration, point registration and automatic registration can be adopted during secondary registration. During manual registration, the mode image can be translated (in four directions of up, down, left and right) and rotated (including clockwise direction and anticlockwise direction) under a target coordinate system or a world coordinate system through a manual operation instruction; during point registration, points needing registration (namely the centers of the cross lines) are determined by respectively arranging the cross lines on the multi-modal medical image and dragging the cross lines, point pairs are formed, more than 3 groups of point pairs are obtained, and then the multi-modal medical image is registered by using more than 3 groups of points; in automatic registration, an automatic registration matrix calculation, which may be based on an anatomical location identification algorithm, and a registration operation may be employed.
In one embodiment, the multi-modality medical image registration method further comprises the steps of:
and fusing the registered modal images to obtain a multi-modal registered fused image.
In this embodiment, after the registration operation is completed, the registered modality images may be fused, and information of a plurality of modality images is integrated to obtain a multi-modality registration fused image with an integral feature, so that multiple features are displayed in the same image.
Specifically, parameters such as a fusion ratio, an upper layer and a lower layer can be flexibly set during fusion operation, so that different fusion display effects are realized.
Furthermore, after the multi-modal registered fused image is obtained, the multi-modal registered fused image can be displayed on a display device, specific image features are shown on the display device, and the multi-modal medical image before registration and the multi-modal registered fused image can be correspondingly stored in a memory for reading.
In practical applications, taking the heart as an example of a target organ, through extensive analysis of PET/MR examination image data by clinical personnel, it is found that the registration problem of the cardiac PET/MR image is mainly caused by respiratory motion of a subject and has a weak relation with heartbeat motion due to the way of PET data and MR data. The displacement direction generated by the human body breathing motion is mainly focused on the head and foot directions of the testee. The normal acquisition orientation of the heart is not a normal position, and is a special oblique position, the normal acquisition also comprises a plurality of complex conditions such as a long axis, a short axis, 2ch, 3ch, 4ch and the like of the heart, so that the problem cannot be solved by using a translation and rotation orientation in a plane during image registration. According to research, the matching of the upper layer and the lower layer of the heart PET/MR image is adjusted in the head and foot direction of a human body, so that a better fusion effect can be obtained in certain scanning layers, and the requirement of clinical examination can be met.
In clinical imaging of knee joints, shoulder joints, etc., in addition to the heart, the image scanning and viewing orientation is also not usually true, but is often a special oblique position relative to the anatomical location. In the image registration process, the coordinate system condition and the scanning orientation condition of the subject can be combined at the same time, and the method for adjusting the registration degree can be flexibly selected.
The scheme of the application can realize the self-adaptive application of the scanning position of the subject in the system, an in-plane registration scheme is applied in a part of scenes, and the scanning position information of the subject is called to perform registration in another part of special scenes. For cardiac PET/MR imaging, the scheme of the application greatly reduces the difficulty of image registration of post-clinical treatment, is convenient for technicians and doctors to operate and use, and improves the image accuracy and the working efficiency.
According to the multi-modality medical image registration method, an embodiment of the application further provides a multi-modality medical image registration system, and the following describes an embodiment of the multi-modality medical image registration system in detail.
Referring to fig. 5, a schematic structural diagram of a multi-modality medical image registration system according to an embodiment is shown. The multi-modality medical image registration system in this embodiment comprises:
a data acquisition unit 510 for acquiring a multi-modal medical image and a target coordinate system, wherein the target coordinate system comprises an anatomical coordinate system and/or an image coordinate system;
an image registration unit 520, configured to register the multi-modal medical image in the target coordinate system, so as to obtain a registered modal image.
In this embodiment, the target coordinate system is an image coordinate system where an anatomical coordinate system or a modal image of a subject is located, and is not a fixed world coordinate system which is generally adopted, and the multi-modal medical image is registered under the target coordinate system, so that cross-plane three-dimensional registration can be realized in the target coordinate system different from the world coordinate system, and the problems that manual registration is difficult to realize three-dimensional registration and point registration and automatic registration are difficult to distinguish a scanning position are solved, and the accuracy of medical image registration is improved.
In one embodiment, the image registration unit 520 is further configured to perform target portion identification according to the multi-modal medical image, and if the identified target portion belongs to a preset target type, perform registration on the multi-modal medical image by using an anatomical coordinate system; conversely, the multi-modality medical images are registered using either the anatomical coordinate system or the image coordinate system.
In an embodiment, the image registration unit 520 is further configured to label the position of the anatomical coordinate system on the multi-modal medical image, respectively.
In one embodiment, when the image registration unit 520 registers the multi-modal medical image in the target coordinate system, the registration operation includes at least one of a rigid transformation, a non-linear transformation, a projective transformation, and an affine transformation, and each registration operation has a registration preset range corresponding to the target portion.
In one embodiment, the image registration unit 520 is further configured to perform secondary registration on the registered modality image in the target coordinate system or the world coordinate system after obtaining the registered modality image.
In an embodiment, as shown in fig. 6, the multi-modality medical image registration system further includes an image fusion unit 530 for fusing the registered modality images to obtain a multi-modality registered fusion image.
The multi-modal medical image registration system of the embodiment of the application corresponds to the multi-modal medical image registration method one by one, and the technical features and the beneficial effects thereof described in the embodiment of the multi-modal medical image registration method are both suitable for the embodiment of the multi-modal medical image registration system.
According to the multi-modality medical image registration method, the readable storage medium and the multi-modality medical image registration device are further provided.
A readable storage medium having stored thereon an executable program which, when executed by a processor, implements the steps of the multi-modality medical image registration method described above.
The readable storage medium can realize the registration of the multi-modal medical image under a target coordinate system by running an executable program on a processor, can realize the registration of the cross-plane three-dimensional direction in the target coordinate system different from a world coordinate system, and solves the problems that the registration of the three-dimensional direction is difficult to realize and the scanning direction is difficult to distinguish by point registration and automatic registration in the related art, thereby improving the accuracy of the medical image registration.
A multi-modal medical image registration device comprises a memory and a processor, wherein the memory stores an executable program, and the processor realizes the steps of the multi-modal medical image registration method when executing the executable program.
According to the multi-modal medical image registration device, the multi-modal medical images can be registered under the target coordinate system by running the executable program on the processing server, the cross-plane three-dimensional direction registration can be realized in the target coordinate system different from the world coordinate system, the problem that the three-dimensional direction registration and the point registration and the automatic registration are difficult to distinguish the scanning direction in the related technology is solved, and the accuracy of medical image registration is improved.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be implemented by a program instructing the relevant hardware. The program may be stored in a readable storage medium. Which when executed comprises the steps of the method described above. The storage medium includes: ROM/RAM, magnetic disk, optical disk, etc.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of multi-modality medical image registration, characterized in that the method comprises the steps of:
acquiring a multi-modal medical image and a target coordinate system; the target coordinate system comprises an anatomical coordinate system and/or an image coordinate system;
and registering the multi-modal medical image under the target coordinate system to obtain a registered modal image.
2. The method of multi-modality medical image registration according to claim 1, wherein the registering the multi-modality medical images in the target coordinate system includes the steps of:
identifying a target part according to the multi-modal medical image;
if the identified target part belongs to a preset target type, registering the multi-modal medical image by adopting the anatomical coordinate system;
conversely, the multi-modality medical image is registered using the anatomical coordinate system or the image coordinate system.
3. The multi-modality medical image registration method of claim 1, further comprising, in registering with the anatomical coordinate system, the steps of:
and respectively labeling the positions of the anatomical coordinate systems on the multi-modal medical images.
4. The method of claim 2, wherein the registration operation comprises at least one of a rigid transformation, a non-linear transformation, a projective transformation, and an affine transformation when registering the multi-modal medical images in the target coordinate system, and each registration operation has a registration preset range corresponding to the target portion.
5. The method of multi-modality medical image registration according to claim 1, further comprising, after the obtaining of the registered modality images, the steps of:
and carrying out secondary registration on the registered modal image under the target coordinate system or the world coordinate system.
6. The method of multi-modality medical image registration according to claim 1, further comprising the steps of:
and fusing the registered modal images to obtain a multi-modal registered fused image.
7. A multi-modality medical image registration system, characterized in that the system comprises:
a data acquisition unit for acquiring a multi-modal medical image and a target coordinate system, the target coordinate system comprising an anatomical coordinate system and/or an image coordinate system;
and the image registration unit is used for registering the multi-modal medical image under the target coordinate system to obtain a registered modal image.
8. The multi-modality medical image registration system of claim 7, further comprising an image fusion unit configured to fuse the registered modality images to obtain a multi-modality registered fused image.
9. A readable storage medium having stored thereon an executable program, wherein the executable program, when executed by a processor, implements the steps of the multi-modality medical image registration method of any one of claims 1 to 6.
10. A multi-modality medical image registration apparatus comprising a memory and a processor, the memory having stored thereon an executable program, characterized in that the processor, when executing the executable program, implements the steps of the multi-modality medical image registration method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111056802.0A CN113689477A (en) | 2021-09-09 | 2021-09-09 | Multi-modality medical image registration method, system, readable storage medium and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111056802.0A CN113689477A (en) | 2021-09-09 | 2021-09-09 | Multi-modality medical image registration method, system, readable storage medium and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113689477A true CN113689477A (en) | 2021-11-23 |
Family
ID=78586184
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111056802.0A Pending CN113689477A (en) | 2021-09-09 | 2021-09-09 | Multi-modality medical image registration method, system, readable storage medium and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113689477A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022242046A1 (en) * | 2021-05-18 | 2022-11-24 | 上海商汤智能科技有限公司 | Method and apparatus for presenting medical image, and electronic device, storage medium and computer program |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107066184A (en) * | 2016-12-13 | 2017-08-18 | 海纳医信（北京）软件科技有限责任公司 | Method for displaying image and device |
| CN107137107A (en) * | 2017-05-09 | 2017-09-08 | 上海联影医疗科技有限公司 | Multi-modality medical image acquisition methods and multi-modality medical image imaging system |
| CN108171738A (en) * | 2018-01-25 | 2018-06-15 | 北京雅森科技发展有限公司 | Multimodal medical image registration method based on brain function template |
| CN109961834A (en) * | 2019-03-22 | 2019-07-02 | 上海联影医疗科技有限公司 | The generation method and equipment of diagnostic imaging report |
| CN110462681A (en) * | 2017-02-23 | 2019-11-15 | 港大科桥有限公司 | For physics to image/image to multiple surfaces of physics registration and image authentication |
| CN110533641A (en) * | 2019-08-20 | 2019-12-03 | 东软医疗系统股份有限公司 | A kind of multimodal medical image registration method and apparatus |
| CN110693513A (en) * | 2018-10-27 | 2020-01-17 | 上海联影医疗科技有限公司 | Control method, system and storage medium for multi-modal medical system |
| CN111311655A (en) * | 2020-02-13 | 2020-06-19 | 东软医疗系统股份有限公司 | Multi-modal image registration method and device, electronic equipment and storage medium |
| CN113081018A (en) * | 2021-03-31 | 2021-07-09 | 上海联影医疗科技股份有限公司 | PET medical imaging system |
-
2021
- 2021-09-09 CN CN202111056802.0A patent/CN113689477A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107066184A (en) * | 2016-12-13 | 2017-08-18 | 海纳医信（北京）软件科技有限责任公司 | Method for displaying image and device |
| CN110462681A (en) * | 2017-02-23 | 2019-11-15 | 港大科桥有限公司 | For physics to image/image to multiple surfaces of physics registration and image authentication |
| CN107137107A (en) * | 2017-05-09 | 2017-09-08 | 上海联影医疗科技有限公司 | Multi-modality medical image acquisition methods and multi-modality medical image imaging system |
| CN108171738A (en) * | 2018-01-25 | 2018-06-15 | 北京雅森科技发展有限公司 | Multimodal medical image registration method based on brain function template |
| CN110693513A (en) * | 2018-10-27 | 2020-01-17 | 上海联影医疗科技有限公司 | Control method, system and storage medium for multi-modal medical system |
| CN109961834A (en) * | 2019-03-22 | 2019-07-02 | 上海联影医疗科技有限公司 | The generation method and equipment of diagnostic imaging report |
| CN110533641A (en) * | 2019-08-20 | 2019-12-03 | 东软医疗系统股份有限公司 | A kind of multimodal medical image registration method and apparatus |
| CN111311655A (en) * | 2020-02-13 | 2020-06-19 | 东软医疗系统股份有限公司 | Multi-modal image registration method and device, electronic equipment and storage medium |
| CN113081018A (en) * | 2021-03-31 | 2021-07-09 | 上海联影医疗科技股份有限公司 | PET medical imaging system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022242046A1 (en) * | 2021-05-18 | 2022-11-24 | 上海商汤智能科技有限公司 | Method and apparatus for presenting medical image, and electronic device, storage medium and computer program |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Brock et al. | Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132 | |
| Slomka et al. | Multimodality image registration with software: state-of-the-art | |
| Van den Elsen et al. | Medical image matching-a review with classification | |
| Han et al. | A nonlinear biomechanical model based registration method for aligning prone and supine MR breast images | |
| Slomka | Software approach to merging molecular with anatomic information | |
| Brock et al. | Results of a multi-institution deformable registration accuracy study (MIDRAS) | |
| US8942455B2 (en) | 2D/3D image registration method | |
| CN109961491A (en) | Multi-modality images truncation and compensation method, apparatus, computer equipment and medium | |
| Navab et al. | Merging visible and invisible: Two camera-augmented mobile C-arm (CAMC) applications | |
| De Buck et al. | An augmented reality system for patient-specific guidance of cardiac catheter ablation procedures | |
| US20050031176A1 (en) | Method and apparatus of multi-modality image fusion | |
| Khmelinskii et al. | Articulated whole-body atlases for small animal image analysis: construction and applications | |
| Hill et al. | Correcting scaling errors in tomographic images using a nine degree of freedom registration algorithm | |
| CN111904379A (en) | Scanning method and device of multi-modal medical equipment | |
| Tomaževič et al. | “Gold standard” data for evaluation and comparison of 3D/2D registration methods | |
| Shekhar et al. | High-speed registration of three-and four-dimensional medical images by using voxel similarity | |
| Klabbers et al. | Matching PET and CT scans of the head and neck area: development of method and validation | |
| CN114943714A (en) | Medical image processing system, medical image processing apparatus, electronic device, and storage medium | |
| Hallpike et al. | Medical image registration: an overview | |
| CN113689477A (en) | Multi-modality medical image registration method, system, readable storage medium and device | |
| Fischer et al. | An MR-based model for cardio-respiratory motion compensation of overlays in X-ray fluoroscopy | |
| US11495346B2 (en) | External device-enabled imaging support | |
| Birkfellner et al. | Multi-modality imaging: a software fusion and image-guided therapy perspective | |
| Mezheritsky et al. | Population-based 3D respiratory motion modelling from convolutional autoencoders for 2D ultrasound-guided radiotherapy | |
| Viergever et al. | Integration of functional and anatomical brain images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |