CN110368027B - Image fusion method and device - Google Patents

Image fusion method and device Download PDF

Info

Publication number
CN110368027B
CN110368027B CN201810330209.2A CN201810330209A CN110368027B CN 110368027 B CN110368027 B CN 110368027B CN 201810330209 A CN201810330209 A CN 201810330209A CN 110368027 B CN110368027 B CN 110368027B
Authority
CN
China
Prior art keywords
electromagnetic
coordinate system
ultrasonic
marker
dimensional
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810330209.2A
Other languages
Chinese (zh)
Other versions
CN110368027A (en
Inventor
郭楚
刘达
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baihui Weikang Technology Co Ltd
Original Assignee
Beijing Baihui Weikang Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baihui Weikang Technology Co Ltd filed Critical Beijing Baihui Weikang Technology Co Ltd
Priority to CN201810330209.2A priority Critical patent/CN110368027B/en
Publication of CN110368027A publication Critical patent/CN110368027A/en
Application granted granted Critical
Publication of CN110368027B publication Critical patent/CN110368027B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Detecting organic movements or changes, e.g. tumours, cysts, swellings
    • A61B8/0833—Detecting organic movements or changes, e.g. tumours, cysts, swellings involving detecting or locating foreign bodies or organic structures
    • A61B8/0841—Detecting organic movements or changes, e.g. tumours, cysts, swellings involving detecting or locating foreign bodies or organic structures for locating instruments
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
    • A61B8/5238—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
    • A61B8/5238—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image
    • A61B8/5246—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image combining images from the same or different imaging techniques, e.g. color Doppler and B-mode
    • A61B8/5253—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image combining images from the same or different imaging techniques, e.g. color Doppler and B-mode combining overlapping images, e.g. spatial compounding

Abstract

The embodiment of the application provides an image fusion method and device, and belongs to the technical field of image processing. The method comprises the steps of establishing a conversion relation between an ultrasonic coordinate system and a three-dimensional coordinate system by utilizing an electromagnetic tracking device, an electromagnetic identifier and a marker; and unifying the ultrasonic image and the three-dimensional image to the same coordinate system according to the conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain the image formed by fusing the ultrasonic image and the three-dimensional image in real time. According to the embodiment of the application, the ultrasonic image and the three-dimensional image can be fused in real time according to the movement of the body surface of the human body and the movement of the ultrasonic probe, and the actual space position of each internal organ of the human body can be accurately obtained when the human body breathes.

Description

Image fusion method and device
Technical Field
The embodiment of the application relates to the technical field of image processing, in particular to an image fusion method and device.
Background
Ultrasonic imaging is an imaging technique in which a human body is scanned with an ultrasonic sound beam, and an image of an internal organ is obtained by receiving and processing a reflected signal.
The general conditions of the blood vessel, the focus and each organ can be judged medically by an ultrasonic image obtained by an ultrasonic imaging technology, and the ultrasonic imaging method has the advantages that the information in the image is real-time, but the ultrasonic image is a sector image, the image quality is not ideal, and the ultrasonic image is a two-dimensional image, can only reflect various physiological information under one plane, and cannot reflect the relative positions of the blood vessel, the focus and each organ on the whole.
The ultrasound image obtained by the above ultrasound imaging technology cannot reflect the relative positions of the blood vessel, the lesion and each organ on the whole, so that an accurate actual spatial position cannot be obtained, and in many cases in medicine, the actual spatial positions of the blood vessel, the lesion and each organ in the human body need to be obtained, so how to apply the ultrasound imaging technology to these cases is a problem to be solved at present.
Disclosure of Invention
In view of the above, one of the technical problems to be solved by the embodiments of the present application is to provide an image fusion method and apparatus, which can accurately obtain the actual spatial position of each internal organ of a human body when the human body breathes.
In a first aspect, an embodiment of the present application provides an image fusion method, including:
establishing a conversion relation between an ultrasonic coordinate system and a three-dimensional coordinate system by using an electromagnetic tracking device, an electromagnetic identifier and a marker;
and unifying the ultrasonic image and the three-dimensional image to the same coordinate system according to the conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain the image formed by fusing the ultrasonic image and the three-dimensional image in real time.
Optionally, the step of establishing a transformation relationship between the ultrasound coordinate system and the three-dimensional coordinate system by using the electromagnetic tracking device, the electromagnetic marker, and the marker includes:
according to the position relation between a marker arranged on the body surface of a human body and an electromagnetic marker corresponding to the marker, real-time tracking of the electromagnetic marker by an electromagnetic tracking device and the position of the marker in a three-dimensional image are carried out, and a conversion relation between a three-dimensional coordinate system and an electromagnetic coordinate system is established;
establishing a conversion relation between an ultrasonic coordinate system and an electromagnetic coordinate system according to the position relation between an electromagnetic identifier arranged on an ultrasonic probe and the real-time tracking of the electromagnetic identifier by an electromagnetic tracking device;
and establishing a conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system according to the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system and the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system.
Optionally, the establishing, by the electromagnetic tracking device, a conversion relationship between the three-dimensional coordinate system and the electromagnetic coordinate system according to a position relationship between a marker disposed on a body surface of a human body and an electromagnetic marker corresponding to the marker, the real-time tracking of the electromagnetic marker by the electromagnetic tracking device, and a position of the marker in the three-dimensional image includes:
obtaining the coordinates of the marker in a marker coordinate system owned by the electromagnetic marker arranged on the body surface of the human body according to the position relation between the marker arranged on the body surface of the human body and the electromagnetic marker;
according to the real-time tracking of the electromagnetic identification by the electromagnetic tracking equipment, the conversion relation between an identification coordinate system and an electromagnetic coordinate system is obtained;
obtaining the coordinates of the marker in the electromagnetic coordinate system according to the coordinates of the marker in the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system;
and establishing a conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system according to the coordinates of the marker in the three-dimensional coordinate system and the coordinates of the marker in the electromagnetic coordinate system, which are obtained according to the position of the marker in the three-dimensional image.
Optionally, the step of establishing a conversion relationship between an ultrasonic coordinate system and an electromagnetic coordinate system according to a position relationship between an electromagnetic identifier arranged on the ultrasonic probe and a real-time tracking of the electromagnetic identifier by an electromagnetic tracking device specifically includes:
obtaining a conversion relation between an ultrasonic coordinate system and an identification coordinate system owned by an electromagnetic identification fixed on the ultrasonic probe according to the position relation between the ultrasonic probe and the electromagnetic identification fixed on the ultrasonic probe;
according to the real-time tracking of the electromagnetic identification by the electromagnetic tracking equipment, the conversion relation between an identification coordinate system and an electromagnetic coordinate system is obtained;
and obtaining the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system according to the conversion relation between the ultrasonic coordinate system and the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system.
Optionally, the position relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically:
at least one of the at least two electromagnetic markers fixed on the body surface of the human body is fixed in position relation with at least three non-collinear markers, each marker has a fixed position relation with at most one electromagnetic marker, and each electromagnetic marker has a fixed position relation with at least one marker.
Optionally, the position relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically: in at least two electromagnetic marks fixed on the body surface of the human body, the position relation between each electromagnetic mark and at least three non-collinear markers is fixed, and each marker has a fixed position relation with at most one electromagnetic mark.
Optionally, the unifying the ultrasound image and the three-dimensional image into the same coordinate system according to the conversion relationship between the ultrasound coordinate system and the three-dimensional coordinate system to obtain the image after the ultrasound image and the three-dimensional image are fused in real time specifically includes:
unifying each pixel in the ultrasonic image and each pixel in the three-dimensional image to the same coordinate system according to the coordinate of each pixel in the ultrasonic image in the ultrasonic coordinate system, the coordinate of each pixel in the three-dimensional image in the three-dimensional coordinate system and the conversion relationship between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain the real-time position relationship between the ultrasonic image and the three-dimensional image, and carrying out image fusion according to the real-time position relationship between the ultrasonic image and the three-dimensional image to obtain an image after the real-time fusion of the ultrasonic image and the three-dimensional image.
In a second aspect, an embodiment of the present application further provides an image fusion apparatus, which includes a three-dimensional ultrasound transformation relation module, and a three-dimensional ultrasound image fusion module, wherein,
the three-dimensional ultrasonic conversion relation module is connected with the three-dimensional ultrasonic image fusion module and is used for: establishing a conversion relation between an ultrasonic coordinate system and a three-dimensional coordinate system by using an electromagnetic tracking device, an electromagnetic identifier and a marker, and sending the conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system to a three-dimensional ultrasonic image fusion module;
the three-dimensional ultrasonic image fusion module is connected with the three-dimensional ultrasonic conversion relation module and is used for:
and unifying the ultrasonic image and the three-dimensional image to the same coordinate system according to the conversion relation between the received ultrasonic coordinate system and the three-dimensional coordinate system to obtain the image formed by fusing the ultrasonic image and the three-dimensional image in real time.
Optionally, the three-dimensional ultrasonic conversion relation module comprises a three-dimensional electromagnetic conversion relation module, an ultrasonic electromagnetic conversion relation module and a three-dimensional ultrasonic conversion relation module, wherein,
the three-dimensional electromagnetic conversion relation module is connected with the three-dimensional ultrasonic conversion relation module and is used for: according to the position relation between a marker arranged on the body surface of a human body and an electromagnetic marker corresponding to the marker, real-time tracking of the electromagnetic marker by an electromagnetic tracking device and the position of the marker in a three-dimensional image, establishing a conversion relation between a three-dimensional coordinate system and an electromagnetic coordinate system, and sending the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system to a three-dimensional ultrasonic conversion relation module;
the ultrasonic electromagnetic conversion relation module is connected with the three-dimensional ultrasonic conversion relation module and is used for: establishing a conversion relation between an ultrasonic coordinate system and an electromagnetic coordinate system according to the position relation between an electromagnetic identifier arranged on the ultrasonic probe and the real-time tracking of the electromagnetic identifier by an electromagnetic tracking device, and sending the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system to a three-dimensional ultrasonic conversion relation module;
the three-dimensional ultrasonic conversion relation module is connected with the three-dimensional electromagnetic conversion relation module and the ultrasonic electromagnetic conversion relation module and is used for:
and establishing a conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system according to the received conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system and the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system.
Optionally, the three-dimensional electromagnetic conversion relationship module further includes an identification coordinate unit, an identification electromagnetic conversion relationship unit, an electromagnetic coordinate unit, and a three-dimensional electromagnetic conversion relationship unit, wherein:
the identification coordinate unit is connected with the electromagnetic coordinate unit and used for obtaining the coordinates of the marker in an identification coordinate system owned by the electromagnetic marker arranged on the body surface of the human body according to the position relation between the marker and the electromagnetic marker arranged on the body surface of the human body and sending the coordinates of the marker in the identification coordinate system to the electromagnetic coordinate unit;
the identification electromagnetic conversion relation unit is connected with the electromagnetic coordinate unit and used for obtaining the conversion relation between an identification coordinate system and an electromagnetic coordinate system according to the real-time tracking of the electromagnetic identification by the electromagnetic tracking equipment and sending the conversion relation between the identification coordinate system and the electromagnetic coordinate system to the electromagnetic coordinate unit;
the electromagnetic coordinate unit is connected with the identification coordinate unit and the identification electromagnetic conversion relation unit and is used for obtaining the coordinates of the marker in the electromagnetic coordinate system according to the received coordinates of the marker in the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system and sending the coordinates of the marker in the electromagnetic coordinate system to the three-dimensional electromagnetic conversion relation unit;
the three-dimensional electromagnetic conversion relation unit is connected with the electromagnetic coordinate unit and used for establishing the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system according to the coordinates of the marker in the three-dimensional coordinate system obtained by the position of the marker in the three-dimensional image and the received coordinates of the marker in the electromagnetic coordinate system.
Optionally, the ultrasonic-electromagnetic conversion relation module includes: the ultrasonic identification conversion relation unit, the identification electromagnetic conversion relation unit and the ultrasonic electromagnetic conversion relation unit, wherein:
the ultrasonic identification conversion relation unit is connected with the ultrasonic electromagnetic conversion relation unit and is used for: according to the position relation between the ultrasonic probe and the electromagnetic mark fixed on the ultrasonic probe, obtaining the conversion relation between an ultrasonic coordinate system and a mark coordinate system owned by the electromagnetic mark fixed on the ultrasonic probe, and sending the conversion relation between the ultrasonic coordinate system and the mark coordinate system to an ultrasonic electromagnetic conversion relation unit;
the identification electromagnetic conversion relation unit is connected with the ultrasonic electromagnetic conversion relation unit and is used for: according to the real-time tracking of the electromagnetic tracking equipment on the electromagnetic identification, the conversion relation between the identification coordinate system and the electromagnetic coordinate system is obtained, and the conversion relation between the identification coordinate system and the electromagnetic coordinate system is sent to an ultrasonic electromagnetic conversion relation unit;
the ultrasonic electromagnetic conversion relation unit is connected with the ultrasonic identification conversion relation unit and the identification electromagnetic conversion relation unit, and is used for:
and obtaining the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system according to the conversion relation between the received ultrasonic coordinate system and the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system.
Optionally, in the image fusion apparatus, the position relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically:
at least one of the at least two electromagnetic markers fixed on the body surface of the human body is fixed in position relation with at least three non-collinear markers, each marker has a fixed position relation with at most one electromagnetic marker, and each electromagnetic marker has a fixed position relation with at least one marker.
Optionally, in the image fusion apparatus, the position relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically:
in at least two electromagnetic marks fixed on the body surface of the human body, the position relation between each electromagnetic mark and at least three non-collinear markers is fixed, and each marker has a fixed position relation with at most one electromagnetic mark.
Optionally, the three-dimensional ultrasound image fusion module is specifically configured to:
unifying each pixel in the ultrasonic image and each pixel in the three-dimensional image to the same coordinate system according to the coordinate of each pixel in the ultrasonic image in the ultrasonic coordinate system, the coordinate of each pixel in the three-dimensional image in the three-dimensional coordinate system and the conversion relationship between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain the real-time position relationship between the ultrasonic image and the three-dimensional image, and carrying out image fusion according to the real-time position relationship between the ultrasonic image and the three-dimensional image to obtain an image after the real-time fusion of the ultrasonic image and the three-dimensional image.
According to the technical scheme, the real-time conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system can be established according to the movement of the body surface of the human body and the movement of the ultrasonic probe by utilizing the electromagnetic tracking equipment, the electromagnetic marks and the markers, and then the ultrasonic image and the three-dimensional image are fused in real time on the basis of the real-time conversion relation, so that the actual space positions of all internal organs of the human body can be accurately obtained when the human body breathes.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1 is a schematic flowchart of an image fusion method in an embodiment of the present application;
FIG. 2 is a schematic structural diagram of an image fusion apparatus according to an embodiment of the present disclosure;
FIG. 3 is a three-dimensional image in an embodiment of the present application;
FIG. 4 is an ultrasound image in an embodiment of the present application;
fig. 5 is an image obtained by fusing a three-dimensional image and an ultrasound image in the embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application shall fall within the scope of the protection of the embodiments in the present application.
In practical operation, the three-dimensional image shown in fig. 3 generated by scanning a human body with a CT (Computer Tomography) device or an MRI (Magnetic Resonance Imaging) Imaging device or a nuclear Magnetic device can show various physiological structures in the human body as a whole, but cannot provide a real-time image, and the ultrasound image shown in fig. 4 can obtain a real-time image, but can only display an image under one plane, so that various physiological structures in the human body cannot be shown as a whole; in addition, both the two modes are difficult to help medical staff to judge the actual spatial positions of blood vessels, focus and organs in the human body; the inventor finds that the two modes are combined, so that medical staff can be effectively helped to judge the actual spatial positions of blood vessels, focus and organs in a human body in real time, and the method is widely applied to medicine; therefore, the embodiment of the application provides a method for fusing an ultrasonic image and a three-dimensional image, so that the accurate actual spatial position of each physiological structure can be obtained through the fused image.
Referring to fig. 1, an embodiment of the present application provides an image fusion method, including steps S100-S200, specifically:
s100: and establishing a conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system by using the electromagnetic tracking equipment, the electromagnetic marker and the marker.
It should be noted that, in this application embodiment, the electromagnetic marking is two kinds including the electromagnetic marking that sets up on ultrasonic probe and the electromagnetic marking that sets up at human body surface, no matter set up the electromagnetic marking on ultrasonic probe, still set up the electromagnetic marking on human body surface, all can be traced and discerned in real time by electromagnetic tracking equipment, the difference is the position difference that sets up, for the convenience of explanation, this application embodiment will set up the electromagnetic marking on ultrasonic probe as probe electromagnetic marking, will set up the electromagnetic marking on human body surface as body surface electromagnetic marking.
In the implementation process, in order to finally obtain the conversion relationship between the three-dimensional coordinate system and the ultrasonic coordinate system, a body surface electromagnetic marker and a marker are fixedly arranged on the body surface, and a fixed position relationship is established between the body surface electromagnetic marker and the marker, so that the relationship between the marker and the electromagnetic tracking device can be established by real-time tracking of the body surface electromagnetic marker by the electromagnetic tracking device, and the relationship between the three-dimensional image and the electromagnetic tracking device can be established by combining the relationship between the marker and the electromagnetic tracking device after the relationship between the marker and the three-dimensional image is established because the marker and the human body can be scanned by a CT (Computer Tomography) device or a MRI (Magnetic Resonance Imaging) device and imaged in the three-dimensional image.
In addition, because a fixed position relation also exists between the probe electromagnetic mark and the ultrasonic probe, the relation between the ultrasonic probe and the electromagnetic tracking equipment can be established by tracking the probe electromagnetic mark in real time through the electromagnetic tracking equipment.
In summary, the relationship between the ultrasound probe and the three-dimensional image can be established by the intermediary of an electromagnetic tracking device.
Optionally, the step of establishing a transformation relationship between the ultrasound coordinate system and the three-dimensional coordinate system using the electromagnetic tracking device, the electromagnetic marker, and the marker includes:
according to the position relation between a marker arranged on the body surface of a human body and an electromagnetic marker corresponding to the marker, real-time tracking of the electromagnetic marker by an electromagnetic tracking device and the position of the marker in a three-dimensional image are carried out, and a conversion relation between a three-dimensional coordinate system and an electromagnetic coordinate system is established;
establishing a conversion relation between an ultrasonic coordinate system and an electromagnetic coordinate system according to the position relation between an electromagnetic identifier arranged on an ultrasonic probe and the real-time tracking of the electromagnetic identifier by an electromagnetic tracking device;
and establishing a conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system according to the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system and the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system.
As can be seen from the above, in the embodiment of the present application, the electromagnetic markers are divided into two types, i.e. an ultrasound electromagnetic marker and a body surface electromagnetic marker, according to the different set positions, and each electromagnetic marker has a marker coordinate system, and for the sake of clearer description, the marker coordinate system is divided into two types, i.e. an ultrasound marker coordinate system owned by the ultrasound electromagnetic marker and a body surface marker coordinate system owned by the body surface electromagnetic marker, and the electromagnetic tracking device has an electromagnetic coordinate system, so that the transformation relationship between the electromagnetic coordinate system and the body surface marker coordinate system can be obtained by real-time tracking of the body surface electromagnetic marker by the electromagnetic tracking device, and the coordinates of the marker in the body surface marker coordinate system can be obtained according to the fixed position relationship between the marker set on the body surface of the human body and the body surface electromagnetic marker, and the transformation relationship between the electromagnetic coordinate system and the body surface marker coordinate system can be combined, the coordinates of the marker in the electromagnetic coordinate system can be further obtained, the marker and the human body can be scanned through CT equipment or nuclear magnetic equipment together to generate a three-dimensional image, and the coordinates of the marker in the three-dimensional coordinate system owned by the three-dimensional image can be obtained according to the position of the marker in the three-dimensional image, so that the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system can be obtained by combining the coordinates of the marker in the electromagnetic coordinate system.
Further, according to the position relationship between the marker arranged on the body surface of the human body and the electromagnetic marker corresponding to the marker, the electromagnetic tracking device tracks the electromagnetic marker in real time, and the position of the marker in the three-dimensional image, and the establishment of the conversion relationship between the three-dimensional coordinate system and the electromagnetic coordinate system comprises the following steps:
obtaining the coordinates of the marker in a marker coordinate system owned by the electromagnetic marker arranged on the body surface of the human body according to the position relation between the marker arranged on the body surface of the human body and the electromagnetic marker;
according to the real-time tracking of the electromagnetic identification by the electromagnetic tracking equipment, the conversion relation between an identification coordinate system and an electromagnetic coordinate system is obtained;
obtaining the coordinates of the marker in the electromagnetic coordinate system according to the coordinates of the marker in the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system;
and establishing a conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system according to the coordinates of the marker in the three-dimensional coordinate system and the coordinates of the marker in the electromagnetic coordinate system, which are obtained according to the position of the marker in the three-dimensional image.
More specifically, in the embodiment of the present application, each body surface electromagnetic identifier has a coordinate system belonging to its own, if there are N individual table electromagnetic identifiers (N is an integer greater than or equal to 2), there are N individual table identifier coordinate systems, and in addition, there is a fixed position relationship between the marker and the body surface electromagnetic identifier, and the coordinates of the center point of the marker in the body surface identifier coordinate system owned by the body surface electromagnetic identifier having a fixed position relationship with the marker can be obtained by a processing drawing calculation or actual measurement method. Meanwhile, the electromagnetic tracking equipment can track and recognize each body surface electromagnetic mark in real time, so that the conversion relation between each body surface mark coordinate system and the electromagnetic coordinate system owned by the electromagnetic tracking equipment can be obtained by tracking the body surface electromagnetic marks in real time through the electromagnetic tracking equipment.
And obtaining the conversion relation between each body surface identification coordinate system and the electromagnetic coordinate system, and obtaining the coordinates of the central point of the marker in the body surface identification coordinate system owned by the body surface electromagnetic identification which has a fixed position relation with the central point of the marker, so as to obtain the coordinates of the central point of the marker in the electromagnetic coordinate system. And because the electromagnetic tracking equipment tracks and identifies each body surface electromagnetic mark in real time, the real-time coordinate of the central point of the marker in the electromagnetic coordinate system can be obtained, and the pose of the electromagnetic tracking equipment is fixed, so that the marker can be represented in a form that the coordinate is continuously changed along with the movement of the body surface in the electromagnetic coordinate system.
Because the CT equipment or the nuclear magnetic equipment scans the human body to obtain the three-dimensional image of the human body, the markers arranged on the body surface of the human body can be presented in the three-dimensional image, and the coordinates of the central point of each marker in the three-dimensional coordinate system of the three-dimensional image are further obtained. The coordinate of the marker center point in the three-dimensional coordinate system and the coordinate of the marker center point in the electromagnetic coordinate system are obtained according to the position of the marker in the three-dimensional image, and the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system can be further obtained according to the coordinates of the same point in the two coordinate systems.
It should be noted that, because the coordinates of the center point of the marker in the electromagnetic coordinate system are obtained in real time, the transformation relationship between the three-dimensional coordinate system and the electromagnetic coordinate system, which is established according to the coordinates of the center point of the marker in the electromagnetic coordinate system obtained in real time, also changes in real time, that is, when the coordinates of the center point of the marker in the electromagnetic coordinate system change, the transformation relationship between the three-dimensional coordinate system and the electromagnetic coordinate system changes accordingly.
In order to obtain a more accurate conversion relation in real time in a breathing process, the embodiment of the application provides a configuration scheme of body surface electromagnetic markers and markers, wherein at least two body surface electromagnetic markers and at least four markers are required in the configuration scheme, in the at least two electromagnetic markers fixed on the body surface of a human body, the position relation between at least one electromagnetic marker and at least three non-collinear markers is fixed, each marker has a fixed position relation with at most one electromagnetic marker, and each electromagnetic marker has a fixed position relation with at least one marker.
On the basis of the above scheme, in order to make the conversion relationship change with breathing more stably, at least two body surface electromagnetic markers and at least six markers need to be fixedly arranged on the body surface, wherein, in the at least two electromagnetic markers fixed on the body surface, the position relationship between each electromagnetic marker and at least three non-collinear markers is fixed, and each marker has a fixed position relationship with at most one electromagnetic marker.
There are various ways to establish the fixed position relationship between the body surface electromagnetic identifier and the marker, for example, the body surface electromagnetic identifier and the marker can be installed on an object and the object can be stuck on the body surface; or sticking the body surface electromagnetic mark on the body surface and fixing the marker on the body surface electromagnetic mark, etc.
Optionally, at least two body surface electromagnetic markers fixed on the body surface are adhered to the body surface, and the markers are fixedly arranged on the surfaces of the at least two body surface electromagnetic markers.
Specifically, can paste two at least body surface electromagnetic marking on the body surface with medical sticky tape, then will have the marker setting of fixed position relation with every body surface electromagnetic marking and be in two at least body surface electromagnetic marking's surface, also can be on the basis of establishing the fixed position relation between body surface electromagnetic marking and the marker like this for body surface electromagnetic marking and marker move along with the body surface.
Optionally, the body surface electromagnetic markers and markers fixed to the body surface are mounted on an electromagnetic tracking base arranged on the body surface of the human body.
The electromagnetic tracking base can be respectively provided with at least one electromagnetic tracking identification position and at least three marker positions so as to respectively fix the body surface electromagnetic identification and the markers, and a fixed position relation is established between the body surface electromagnetic identification and the markers, so that the body surface electromagnetic identification and the markers can move along with the body surface on the basis of not influencing the tracking and identification of the electromagnetic tracking equipment on the body surface electromagnetic identification.
In the actual operation process, also can set up the sucking disc on the body surface for the electromagnetism is tracked the base and is fixed with the body surface, and this application embodiment does not do specifically to the mode of establishing fixed position relation between body surface electromagnetic marking and marker and does not injectd, as long as can satisfy body surface electromagnetic marking and marker can be fixed with the body surface, and do not influence the real-time tracking of electromagnetic tracking equipment to body surface electromagnetic marking discern can.
Having described the procedure of obtaining the conversion relationship between the three-dimensional coordinate system and the electromagnetic coordinate system in the foregoing, in order to finally obtain the conversion relationship between the ultrasonic coordinate system and the three-dimensional coordinate system, it is also necessary to determine the conversion relationship between the ultrasonic coordinate system and the electromagnetic coordinate system.
Optionally, the step of establishing a conversion relationship between an ultrasonic coordinate system and an electromagnetic coordinate system according to a position relationship between an electromagnetic identifier arranged on the ultrasonic probe and a real-time tracking of the electromagnetic identifier by an electromagnetic tracking device specifically includes:
obtaining a conversion relation between an ultrasonic coordinate system and an identification coordinate system owned by an electromagnetic identification fixed on the ultrasonic probe according to the position relation between the ultrasonic probe and the electromagnetic identification fixed on the ultrasonic probe;
according to the real-time tracking of the electromagnetic identification by the electromagnetic tracking equipment, the conversion relation between an identification coordinate system and an electromagnetic coordinate system is obtained;
and obtaining the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system according to the conversion relation between the ultrasonic coordinate system and the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system.
Because the probe electromagnetic mark is fixedly arranged on the ultrasonic probe, the conversion relation between the probe mark coordinate system and the ultrasonic coordinate system can be obtained through the fixed position relation between the probe electromagnetic mark and the ultrasonic probe, and further, the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system can be obtained by combining the conversion relation between the electromagnetic coordinate system and the probe mark coordinate system obtained by the real-time tracking of the probe electromagnetic mark by the electromagnetic tracking equipment.
In summary, the electromagnetic coordinate system is used as a conversion medium, and a conversion relation can be established between the ultrasonic coordinate system and the three-dimensional coordinate system according to the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system and the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system.
S200: and unifying the ultrasonic image and the three-dimensional image to the same coordinate system according to the conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain the image formed by fusing the ultrasonic image and the three-dimensional image in real time.
In the implementation process, the ultrasound image and the three-dimensional image may be unified into an ultrasound coordinate system, may also be unified into a three-dimensional coordinate system, or may be unified into an electromagnetic coordinate system … … in this embodiment of the present application, which coordinate system the ultrasound image and the three-dimensional image are unified into is not specifically limited, as long as the ultrasound image and the three-dimensional image are unified into the same coordinate system, and an image obtained by fusing the ultrasound image and the three-dimensional image in real time as shown in fig. 5 can be obtained.
Further, according to the conversion relationship between the ultrasound coordinate system and the three-dimensional coordinate system, unifying the ultrasound image and the three-dimensional image in the same coordinate system, and obtaining the real-time fused image of the ultrasound image and the three-dimensional image specifically comprises:
unifying each pixel in the ultrasonic image and each pixel in the three-dimensional image to the same coordinate system according to the coordinate of each pixel in the ultrasonic image in the ultrasonic coordinate system, the coordinate of each pixel in the three-dimensional image in the three-dimensional coordinate system and the conversion relationship between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain the real-time position relationship between the ultrasonic image and the three-dimensional image, and carrying out image fusion according to the real-time position relationship between the ultrasonic image and the three-dimensional image to obtain an image after the real-time fusion of the ultrasonic image and the three-dimensional image.
Optionally, each pixel in the ultrasound image has a corresponding coordinate in the ultrasound coordinate system, and in addition, the transformation relationship between the ultrasound coordinate system and the three-dimensional coordinate system is already obtained, so according to the transformation relationship between the ultrasound coordinate system and the three-dimensional coordinate system and the coordinate of each pixel in the ultrasound image in the ultrasound coordinate system, the coordinate of each pixel in the ultrasound image in the three-dimensional image can be obtained, and after the coordinate of each pixel in the ultrasound image in the three-dimensional image is obtained, an image obtained by fusing the ultrasound image and the three-dimensional image can be obtained, and further, the ultrasound image and the three-dimensional image can be displayed in a combined manner.
Similarly, each pixel in the three-dimensional image has a corresponding coordinate in the three-dimensional coordinate system, and the conversion relationship between the ultrasound coordinate system and the three-dimensional coordinate system is obtained, so that the coordinate of each pixel in the three-dimensional image in the ultrasound image can be obtained according to the conversion relationship between the ultrasound coordinate system and the three-dimensional coordinate system and the coordinate of each pixel in the three-dimensional image in the three-dimensional coordinate system, and after the coordinate of each pixel in the three-dimensional image in the ultrasound image is obtained, the image obtained by fusing the ultrasound image and the three-dimensional image can be obtained, and the ultrasound image and the three-dimensional image can be displayed in a combined manner.
It should be noted that the transformation relationship between the ultrasound coordinate system and the electromagnetic coordinate system and the transformation relationship between the three-dimensional coordinate system and the electromagnetic coordinate system are changed along with the movement of the body surface and the movement of the ultrasound probe, and therefore the transformation relationship between the ultrasound coordinate system and the three-dimensional coordinate system is also changed along with the movement of the body surface and the movement of the ultrasound probe. Furthermore, the fusion of the ultrasonic image and the three-dimensional image performed on the basis of the conversion relationship between the ultrasonic coordinate system and the three-dimensional coordinate system is also changed along with the movement of the body surface and the movement of the ultrasonic probe, so that the actual spatial position of each organ in the human body can be accurately obtained even when the human body breathes.
As shown in fig. 2, based on the same inventive concept, an embodiment of the present application further provides an image fusion apparatus, which includes a three-dimensional ultrasound transformation relation module 201, and a three-dimensional ultrasound image fusion module 202, wherein,
the three-dimensional ultrasound conversion relation module 201 is connected to the three-dimensional ultrasound image fusion module 202, and is configured to: establishing a conversion relation between an ultrasonic coordinate system and a three-dimensional coordinate system by using an electromagnetic tracking device, an electromagnetic identifier and a marker, and sending the conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system to the three-dimensional ultrasonic image fusion module 202;
the three-dimensional ultrasonic image fusion module 202 is connected to the three-dimensional ultrasonic transformation relation module 201, and is configured to: and unifying the ultrasonic image and the three-dimensional image to the same coordinate system according to the conversion relation between the received ultrasonic coordinate system and the three-dimensional coordinate system to obtain the image formed by fusing the ultrasonic image and the three-dimensional image in real time.
Optionally, the three-dimensional ultrasonic conversion relation module 201 includes a three-dimensional electromagnetic conversion relation module, an ultrasonic electromagnetic conversion relation module and a three-dimensional ultrasonic conversion relation module, wherein;
the three-dimensional electromagnetic conversion relation module is connected with the three-dimensional ultrasonic conversion relation module and is used for:
according to the position relation between a marker arranged on the body surface of a human body and an electromagnetic marker corresponding to the marker, real-time tracking of the electromagnetic marker by an electromagnetic tracking device and the position of the marker in a three-dimensional image, establishing a conversion relation between a three-dimensional coordinate system and an electromagnetic coordinate system, and sending the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system to a three-dimensional ultrasonic conversion relation module;
the ultrasonic electromagnetic conversion relation module is connected with the three-dimensional ultrasonic conversion relation module and is used for:
establishing a conversion relation between an ultrasonic coordinate system and an electromagnetic coordinate system according to the position relation between an electromagnetic identifier arranged on the ultrasonic probe and the real-time tracking of the electromagnetic identifier by an electromagnetic tracking device, and sending the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system to a three-dimensional ultrasonic conversion relation module;
the three-dimensional ultrasonic conversion relation module is connected with the three-dimensional electromagnetic conversion relation module and the ultrasonic electromagnetic conversion relation module and is used for:
and establishing a conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system according to the received conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system and the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system.
Optionally, the three-dimensional electromagnetic conversion relationship module further includes an identification coordinate unit, an identification electromagnetic conversion relationship unit, an electromagnetic coordinate unit, and a three-dimensional electromagnetic conversion relationship unit, wherein:
the identification coordinate unit is connected with the electromagnetic coordinate unit and used for obtaining the coordinates of the marker in an identification coordinate system owned by the electromagnetic marker arranged on the body surface of the human body according to the position relation between the marker and the electromagnetic marker arranged on the body surface of the human body and sending the coordinates of the marker in the identification coordinate system to the electromagnetic coordinate unit;
the identification electromagnetic conversion relation unit is connected with the electromagnetic coordinate unit and used for obtaining the conversion relation between an identification coordinate system and an electromagnetic coordinate system according to the real-time tracking of the electromagnetic identification by the electromagnetic tracking equipment and sending the conversion relation between the identification coordinate system and the electromagnetic coordinate system to the electromagnetic coordinate unit;
the electromagnetic coordinate unit is connected with the identification coordinate unit and the identification electromagnetic conversion relation unit and is used for obtaining the coordinates of the marker in the electromagnetic coordinate system according to the received coordinates of the marker in the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system and sending the coordinates of the marker in the electromagnetic coordinate system to the three-dimensional electromagnetic conversion relation unit;
the three-dimensional electromagnetic conversion relation unit is connected with the electromagnetic coordinate unit and used for establishing the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system according to the coordinates of the marker in the three-dimensional coordinate system obtained by the position of the marker in the three-dimensional image and the received coordinates of the marker in the electromagnetic coordinate system.
Optionally, the ultrasonic-electromagnetic conversion relation module includes: the ultrasonic identification conversion relation unit, the identification electromagnetic conversion relation unit and the ultrasonic electromagnetic conversion relation unit, wherein:
the ultrasonic identification conversion relation unit is connected with the ultrasonic electromagnetic conversion relation unit and is used for:
according to the position relation between the ultrasonic probe and the electromagnetic mark fixed on the ultrasonic probe, obtaining the conversion relation between an ultrasonic coordinate system and a mark coordinate system owned by the electromagnetic mark fixed on the ultrasonic probe, and sending the conversion relation between the ultrasonic coordinate system and the mark coordinate system to an ultrasonic electromagnetic conversion relation unit;
the identification electromagnetic conversion relation unit is connected with the ultrasonic electromagnetic conversion relation unit and is used for:
according to the real-time tracking of the electromagnetic tracking equipment on the electromagnetic identification, the conversion relation between the identification coordinate system and the electromagnetic coordinate system is obtained, and the conversion relation between the identification coordinate system and the electromagnetic coordinate system is sent to an ultrasonic electromagnetic conversion relation unit;
the ultrasonic electromagnetic conversion relation unit is connected with the ultrasonic identification conversion relation unit and the identification electromagnetic conversion relation unit, and is used for:
and obtaining the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system according to the conversion relation between the received ultrasonic coordinate system and the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system.
Optionally, in the image fusion apparatus, the position relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically:
at least one of the at least two electromagnetic markers fixed on the body surface of the human body is fixed in position relation with at least three non-collinear markers, each marker has a fixed position relation with at most one electromagnetic marker, and each electromagnetic marker has a fixed position relation with at least one marker.
Optionally, in the image fusion apparatus, the position relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically:
in at least two electromagnetic marks fixed on the body surface of the human body, the position relation between each electromagnetic mark and at least three non-collinear markers is fixed, and each marker has a fixed position relation with at most one electromagnetic mark.
Optionally, the three-dimensional ultrasound image fusion module 202 is specifically configured to:
unifying each pixel in the ultrasonic image and each pixel in the three-dimensional image to the same coordinate system according to the coordinate of each pixel in the ultrasonic image in the ultrasonic coordinate system, the coordinate of each pixel in the three-dimensional image in the three-dimensional coordinate system and the conversion relationship between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain the real-time position relationship between the ultrasonic image and the three-dimensional image, and carrying out image fusion according to the real-time position relationship between the ultrasonic image and the three-dimensional image to obtain an image after the real-time fusion of the ultrasonic image and the three-dimensional image.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative functional modules and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the above-described modules or units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The integrated unit may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium and can implement the steps of the embodiments of the method when the computer program is executed by a processor. The computer program includes computer program code, and the computer program code may be in a source code form, an object code form, an executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable storage medium may contain content that is subject to appropriate increase or decrease according to the requirements of legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable storage media does not include electrical carrier signals and telecommunication signals according to legislation and patent practice.
The above embodiments are only for illustrating the technical solutions of the present invention, and not for limiting the same. While preferred embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the true scope of the embodiments of the application. It will be apparent to those skilled in the art that various changes and modifications may be made in the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if such modifications and variations of the embodiments of the present application fall within the scope of the claims of the embodiments of the present application and their equivalents, the embodiments of the present application are intended to include such modifications and variations as well.

Claims (12)

1. An image fusion method, comprising:
establishing a conversion relation between an ultrasonic coordinate system and a three-dimensional coordinate system by using an electromagnetic tracking device, an electromagnetic identifier and a marker;
unifying the ultrasonic image and the three-dimensional image to the same coordinate system according to the conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain an image obtained by fusing the ultrasonic image and the three-dimensional image in real time;
wherein the step of establishing a transformation relationship between the ultrasound coordinate system and the three-dimensional coordinate system using the electromagnetic tracking device, the electromagnetic marker, and the marker comprises: according to the position relation between a marker arranged on the body surface of a human body and an electromagnetic marker corresponding to the marker, real-time tracking of the electromagnetic marker by an electromagnetic tracking device and the position of the marker in a three-dimensional image are carried out, and a conversion relation between a three-dimensional coordinate system and an electromagnetic coordinate system is established; establishing a conversion relation between an ultrasonic coordinate system and an electromagnetic coordinate system according to the position relation between an electromagnetic identifier arranged on an ultrasonic probe and the real-time tracking of the electromagnetic identifier by an electromagnetic tracking device; and establishing a conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system according to the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system and the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system.
2. The image fusion method according to claim 1, wherein the establishing of the conversion relationship between the three-dimensional coordinate system and the electromagnetic coordinate system according to the position relationship between the marker disposed on the body surface of the human body and the electromagnetic marker corresponding to the marker, the real-time tracking of the electromagnetic marker by the electromagnetic tracking device, and the position of the marker in the three-dimensional image comprises:
obtaining the coordinates of the marker in a marker coordinate system owned by the electromagnetic marker arranged on the body surface of the human body according to the position relation between the marker arranged on the body surface of the human body and the electromagnetic marker;
according to the real-time tracking of the electromagnetic identification by the electromagnetic tracking equipment, the conversion relation between an identification coordinate system and an electromagnetic coordinate system is obtained;
obtaining the coordinates of the marker in the electromagnetic coordinate system according to the coordinates of the marker in the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system;
and establishing a conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system according to the coordinates of the marker in the three-dimensional coordinate system and the coordinates of the marker in the electromagnetic coordinate system, which are obtained according to the position of the marker in the three-dimensional image.
3. The image fusion method according to claim 1, wherein the step of establishing the transformation relationship between the ultrasound coordinate system and the electromagnetic coordinate system according to the position relationship between the electromagnetic marker set on the ultrasound probe and the real-time tracking of the electromagnetic marker by the electromagnetic tracking device comprises:
obtaining a conversion relation between an ultrasonic coordinate system and an identification coordinate system owned by an electromagnetic identification fixed on the ultrasonic probe according to the position relation between the ultrasonic probe and the electromagnetic identification fixed on the ultrasonic probe;
according to the real-time tracking of the electromagnetic identification by the electromagnetic tracking equipment, the conversion relation between an identification coordinate system and an electromagnetic coordinate system is obtained;
and obtaining the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system according to the conversion relation between the ultrasonic coordinate system and the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system.
4. The image fusion method according to claim 1, 2 or 3, wherein the positional relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically:
at least one of the at least two electromagnetic markers fixed on the body surface of the human body is fixed in position relation with at least three non-collinear markers, each marker has a fixed position relation with at most one electromagnetic marker, and each electromagnetic marker has a fixed position relation with at least one marker.
5. The image fusion method according to claim 1, 2 or 3, wherein the positional relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically:
in at least two electromagnetic marks fixed on the body surface of the human body, the position relation between each electromagnetic mark and at least three non-collinear markers is fixed, and each marker has a fixed position relation with at most one electromagnetic mark.
6. The image fusion method according to claim 1, wherein the step of unifying the ultrasound image and the three-dimensional image into the same coordinate system according to the conversion relationship between the ultrasound coordinate system and the three-dimensional coordinate system to obtain the real-time fused image of the ultrasound image and the three-dimensional image is specifically as follows:
unifying each pixel in the ultrasonic image and each pixel in the three-dimensional image to the same coordinate system according to the coordinate of each pixel in the ultrasonic image in the ultrasonic coordinate system, the coordinate of each pixel in the three-dimensional image in the three-dimensional coordinate system and the conversion relationship between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain the real-time position relationship between the ultrasonic image and the three-dimensional image, and carrying out image fusion according to the real-time position relationship between the ultrasonic image and the three-dimensional image to obtain an image after the real-time fusion of the ultrasonic image and the three-dimensional image.
7. An image fusion device is characterized by comprising a three-dimensional ultrasonic conversion relation module and a three-dimensional ultrasonic image fusion module, wherein,
the three-dimensional ultrasonic conversion relation module is connected with the three-dimensional ultrasonic image fusion module and is used for:
establishing a conversion relation between an ultrasonic coordinate system and a three-dimensional coordinate system by using an electromagnetic tracking device, an electromagnetic identifier and a marker, and sending the conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system to a three-dimensional ultrasonic image fusion module;
the three-dimensional ultrasonic conversion relation module comprises a three-dimensional electromagnetic conversion relation module, an ultrasonic electromagnetic conversion relation module and a three-dimensional ultrasonic conversion relation module, wherein,
the three-dimensional electromagnetic conversion relation module is connected with the three-dimensional ultrasonic conversion relation module and is used for:
according to the position relation between a marker arranged on the body surface of a human body and an electromagnetic marker corresponding to the marker, real-time tracking of the electromagnetic marker by an electromagnetic tracking device and the position of the marker in a three-dimensional image, establishing a conversion relation between a three-dimensional coordinate system and an electromagnetic coordinate system, and sending the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system to a three-dimensional ultrasonic conversion relation module;
the ultrasonic electromagnetic conversion relation module is connected with the three-dimensional ultrasonic conversion relation module and is used for:
establishing a conversion relation between an ultrasonic coordinate system and an electromagnetic coordinate system according to the position relation between an electromagnetic identifier arranged on the ultrasonic probe and the real-time tracking of the electromagnetic identifier by an electromagnetic tracking device, and sending the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system to a three-dimensional ultrasonic conversion relation module;
the three-dimensional ultrasonic conversion relation module is connected with the three-dimensional electromagnetic conversion relation module and the ultrasonic electromagnetic conversion relation module and is used for:
establishing a conversion relation between the ultrasonic coordinate system and the three-dimensional coordinate system according to the received conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system and the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system;
the three-dimensional ultrasonic image fusion module is connected with the three-dimensional ultrasonic conversion relation module and is used for:
and unifying the ultrasonic image and the three-dimensional image to the same coordinate system according to the conversion relation between the received ultrasonic coordinate system and the three-dimensional coordinate system to obtain the image formed by fusing the ultrasonic image and the three-dimensional image in real time.
8. The image fusion apparatus according to claim 7, wherein the three-dimensional electromagnetic conversion relationship module further comprises an identification coordinate unit, an identification electromagnetic conversion relationship unit, an electromagnetic coordinate unit, and a three-dimensional electromagnetic conversion relationship unit, wherein:
the identification coordinate unit is connected with the electromagnetic coordinate unit and used for obtaining the coordinates of the marker in an identification coordinate system owned by the electromagnetic marker arranged on the body surface of the human body according to the position relation between the marker and the electromagnetic marker arranged on the body surface of the human body and sending the coordinates of the marker in the identification coordinate system to the electromagnetic coordinate unit;
the identification electromagnetic conversion relation unit is connected with the electromagnetic coordinate unit and used for obtaining the conversion relation between an identification coordinate system and an electromagnetic coordinate system according to the real-time tracking of the electromagnetic identification by the electromagnetic tracking equipment and sending the conversion relation between the identification coordinate system and the electromagnetic coordinate system to the electromagnetic coordinate unit;
the electromagnetic coordinate unit is connected with the identification coordinate unit and the identification electromagnetic conversion relation unit and is used for obtaining the coordinates of the marker in the electromagnetic coordinate system according to the received coordinates of the marker in the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system and sending the coordinates of the marker in the electromagnetic coordinate system to the three-dimensional electromagnetic conversion relation unit;
the three-dimensional electromagnetic conversion relation unit is connected with the electromagnetic coordinate unit and used for establishing the conversion relation between the three-dimensional coordinate system and the electromagnetic coordinate system according to the coordinates of the marker in the three-dimensional coordinate system obtained by the position of the marker in the three-dimensional image and the received coordinates of the marker in the electromagnetic coordinate system.
9. The image fusion device of claim 7, wherein the ultrasound electromagnetic conversion relation module comprises: the ultrasonic identification conversion relation unit, the identification electromagnetic conversion relation unit and the ultrasonic electromagnetic conversion relation unit, wherein:
the ultrasonic identification conversion relation unit is connected with the ultrasonic electromagnetic conversion relation unit and is used for:
according to the position relation between the ultrasonic probe and the electromagnetic mark fixed on the ultrasonic probe, obtaining the conversion relation between an ultrasonic coordinate system and a mark coordinate system owned by the electromagnetic mark fixed on the ultrasonic probe, and sending the conversion relation between the ultrasonic coordinate system and the mark coordinate system to an ultrasonic electromagnetic conversion relation unit;
the identification electromagnetic conversion relation unit is connected with the ultrasonic electromagnetic conversion relation unit and is used for:
according to the real-time tracking of the electromagnetic tracking equipment on the electromagnetic identification, the conversion relation between the identification coordinate system and the electromagnetic coordinate system is obtained, and the conversion relation between the identification coordinate system and the electromagnetic coordinate system is sent to an ultrasonic electromagnetic conversion relation unit;
the ultrasonic electromagnetic conversion relation unit is connected with the ultrasonic identification conversion relation unit and the identification electromagnetic conversion relation unit, and is used for:
and obtaining the conversion relation between the ultrasonic coordinate system and the electromagnetic coordinate system according to the conversion relation between the received ultrasonic coordinate system and the identification coordinate system and the conversion relation between the identification coordinate system and the electromagnetic coordinate system.
10. The image fusion device according to claim 7, 8 or 9, wherein the position relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically:
at least one of the at least two electromagnetic markers fixed on the body surface of the human body is fixed in position relation with at least three non-collinear markers, each marker has a fixed position relation with at most one electromagnetic marker, and each electromagnetic marker has a fixed position relation with at least one marker.
11. The image fusion device according to claim 7, 8 or 9, wherein the position relationship between the marker and the electromagnetic marker disposed on the body surface of the human body is specifically:
in at least two electromagnetic marks fixed on the body surface of the human body, the position relation between each electromagnetic mark and at least three non-collinear markers is fixed, and each marker has a fixed position relation with at most one electromagnetic mark.
12. The image fusion device of claim 7, wherein the three-dimensional ultrasound image fusion module is specifically configured to:
unifying each pixel in the ultrasonic image and each pixel in the three-dimensional image to the same coordinate system according to the coordinate of each pixel in the ultrasonic image in the ultrasonic coordinate system, the coordinate of each pixel in the three-dimensional image in the three-dimensional coordinate system and the conversion relationship between the ultrasonic coordinate system and the three-dimensional coordinate system to obtain the real-time position relationship between the ultrasonic image and the three-dimensional image, and carrying out image fusion according to the real-time position relationship between the ultrasonic image and the three-dimensional image to obtain an image after the real-time fusion of the ultrasonic image and the three-dimensional image.
CN201810330209.2A 2018-04-13 2018-04-13 Image fusion method and device Active CN110368027B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810330209.2A CN110368027B (en) 2018-04-13 2018-04-13 Image fusion method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810330209.2A CN110368027B (en) 2018-04-13 2018-04-13 Image fusion method and device

Publications (2)

Publication Number Publication Date
CN110368027A CN110368027A (en) 2019-10-25
CN110368027B true CN110368027B (en) 2022-02-18

Family

ID=68243941

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810330209.2A Active CN110368027B (en) 2018-04-13 2018-04-13 Image fusion method and device

Country Status (1)

Country Link
CN (1) CN110368027B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113160221B (en) * 2021-05-14 2022-06-28 深圳市奥昇医疗科技有限责任公司 Image processing method, image processing device, computer equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101108140A (en) * 2007-09-05 2008-01-23 新奥博为技术有限公司 Calibration mould used for image navigation operation system and calibration method thereof
CN101569541A (en) * 2008-04-29 2009-11-04 香港理工大学 Three-dimensional ultrasonic imaging system
CN101681504A (en) * 2006-11-27 2010-03-24 皇家飞利浦电子股份有限公司 System and method for fusing real-time ultrasound images with pre-acquired medical images
CN101862205A (en) * 2010-05-25 2010-10-20 中国人民解放军第四军医大学 Intraoperative tissue tracking method combined with preoperative image
CN102319117A (en) * 2011-06-16 2012-01-18 上海交通大学医学院附属瑞金医院 Arterial intervention implant implanting system capable of fusing real-time ultrasonic information based on magnetic navigation
CN107157516A (en) * 2017-07-05 2017-09-15 四川省肿瘤医院 A kind of ultrasound scanning device

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050054910A1 (en) * 2003-07-14 2005-03-10 Sunnybrook And Women's College Health Sciences Centre Optical image-based position tracking for magnetic resonance imaging applications
US8526688B2 (en) * 2006-03-09 2013-09-03 General Electric Company Methods and systems for registration of surgical navigation data and image data
US8068648B2 (en) * 2006-12-21 2011-11-29 Depuy Products, Inc. Method and system for registering a bone of a patient with a computer assisted orthopaedic surgery system
CN101336831B (en) * 2008-08-13 2010-09-01 汕头市超声仪器研究所有限公司 Rebuilding method of real-time three-dimensional medical ultrasonic image
JP5624481B2 (en) * 2010-02-03 2014-11-12 株式会社東芝 Ultrasonic diagnostic apparatus and medical image processing apparatus
CN102385748B (en) * 2010-08-31 2013-12-25 上海微创医疗器械（集团）有限公司 Image registration method
US9119550B2 (en) * 2012-03-30 2015-09-01 Siemens Medical Solutions Usa, Inc. Magnetic resonance and ultrasound parametric image fusion
DE102013214479B4 (en) * 2013-07-24 2017-04-27 Siemens Healthcare Gmbh Method for tracking a 2D 3D registration of motion and computing device
CN104771232A (en) * 2015-05-05 2015-07-15 北京汇影互联科技有限公司 Electromagnetic positioning system and selection method for three-dimensional image view angle of electromagnetic positioning system
EP3103396B1 (en) * 2015-06-10 2018-10-24 Helmholtz Zentrum München Deutsches Forschungszentrum für Gesundheit und Umwelt GmbH Device and method for hybrid optoacoustic tomography and ultrasonography
CN107714082A (en) * 2017-09-04 2018-02-23 北京航空航天大学 A kind of ultrasonic probe caliberating device and method based on optical alignment

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101681504A (en) * 2006-11-27 2010-03-24 皇家飞利浦电子股份有限公司 System and method for fusing real-time ultrasound images with pre-acquired medical images
CN101108140A (en) * 2007-09-05 2008-01-23 新奥博为技术有限公司 Calibration mould used for image navigation operation system and calibration method thereof
CN101569541A (en) * 2008-04-29 2009-11-04 香港理工大学 Three-dimensional ultrasonic imaging system
CN101862205A (en) * 2010-05-25 2010-10-20 中国人民解放军第四军医大学 Intraoperative tissue tracking method combined with preoperative image
CN102319117A (en) * 2011-06-16 2012-01-18 上海交通大学医学院附属瑞金医院 Arterial intervention implant implanting system capable of fusing real-time ultrasonic information based on magnetic navigation
CN107157516A (en) * 2017-07-05 2017-09-15 四川省肿瘤医院 A kind of ultrasound scanning device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
陈锐锋;陈锐锋;《中国优秀硕士学位论文全文数据库 信息科技辑》;20150315;I138-2152 *

Also Published As

Publication number Publication date
CN110368027A (en) 2019-10-25

Similar Documents

Publication Publication Date Title
WO2017185540A1 (en) Neurosurgical robot navigation positioning system and method
EP3238649B1 (en) Self-localizing medical device
US6490473B1 (en) System and method of interactive positioning
CN103735312A (en) Multimode image navigation system for ultrasonic guidance operation
CN107105972A (en) Model register system and method
CN107854177A (en) A kind of ultrasound and CT/MR image co-registrations operation guiding system and its method based on optical alignment registration
CN102266250A (en) Ultrasonic operation navigation system and ultrasonic operation navigation method
Burgner et al. A study on the theoretical and practical accuracy of conoscopic holography‐based surface measurements: toward image registration in minimally invasive surgery
CN101019771A (en) Navigation system and method backing up several modes
CN110547872A (en) Operation navigation registration system
CN112515767A (en) Surgical navigation device, surgical navigation apparatus, and computer-readable storage medium
CN110368027B (en) Image fusion method and device
US9357979B2 (en) Method and system for organic specimen feature identification in ultrasound image
CN110368028B (en) Real-time registration method and device of space coordinate system
CN112381750A (en) Multi-mode registration fusion method for ultrasonic image and CT/MRI image
CN110368026B (en) Operation auxiliary device and system
Preiswerk et al. Open-source neuronavigation for multimodal non-invasive brain stimulation using 3D Slicer
Brudfors et al. ConoSurf: Open‐source 3D scanning system based on a conoscopic holography device for acquiring surgical surfaces
Kang et al. Towards a clinical stereoscopic augmented reality system for laparoscopic surgery
CN112472132A (en) Device and method for positioning imaging area and medical imaging device
Beitone et al. Towards real‐time free‐hand biopsy navigation
EP3277186B1 (en) Medical imaging apparatus
Basafa et al. Fast, intuitive, vision-based: performance metrics for visual registration, instrument guidance, and image fusion
Qi et al. A Novel Registration Method for a Mixed Reality Navigation System Based on a Laser Crosshair Simulator: A Technical Note
Pyciński et al. Image guided core needle biopsy of the breast

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20210617

Address after: 100191 Room 608, 6 / F, building 9, 35 Huayuan North Road, Haidian District, Beijing

Applicant after: Beijing Baihui Wei Kang Technology Co.,Ltd.

Address before: 100191 506, 5th floor, building 9, 35 Huayuan North Road, Haidian District, Beijing

Applicant before: BEIJING REMEBOT MEDICAL ROBOT TECHNOLOGY Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant
CP03 Change of name, title or address

Address after: 100191 Room 501, floor 5, building 9, No. 35 Huayuan North Road, Haidian District, Beijing

Patentee after: Beijing Baihui Weikang Technology Co.,Ltd.

Address before: 100191 Room 608, 6 / F, building 9, 35 Huayuan North Road, Haidian District, Beijing

Patentee before: Beijing Baihui Wei Kang Technology Co.,Ltd.

CP03 Change of name, title or address