CN111640106A - Multimode medical image conversion method based on artificial intelligence - Google Patents

Abstract

The invention discloses a multi-modal medical image conversion method based on artificial intelligence, and mainly relates to the field of medical images. Comprises acquiring OCT image data of aortic dissection; acquiring IVUS image data of an aortic dissection; respectively adopting a global threshold segmentation method for the background of OCT image data and IVUS image data to extract the outline of the aorta, and cleaning noise information by using a morphological hole filling and area threshold method; and carrying out image information fusion on the OCT image data and the IVUS image data by a frequency domain information fusion method of discrete cosine transform to obtain a fused image. The invention has the beneficial effects that: the method combines the thoracic aorta interlayer images of various modes, fuses the structural features in the ultrasonic image into the OCT image, and effectively enhances the features of a real cavity, a false cavity and thrombus in the fused multimode image.

Description

Multimode medical image conversion method based on artificial intelligence

Technical Field

The invention relates to the field of medical images, in particular to a multi-modal medical image conversion method based on artificial intelligence.

Background

In recent years, with the rapid development of various medical imaging technologies (ct.mri.ultrasounds.pet, OCT, Microscop, etc.), and the technologies have been widely applied to early detection, diagnosis and treatment of diseases, so that in clinical practice, imaging experts need to face a large amount of medical image data of patients every day, and the image data is simply read and determined by people, which is time-consuming, labor-consuming and subjective. Therefore, computer-aided diagnosis technology is a powerful measure to solve the problem. In particular, the introduction of a machine learning method of "deep learning" which has been activated in recent years has gradually accelerated the pace of intellectualization and precision in the calculation and analysis of modern computer medical images.

The thoracic aortic dissection is a very dangerous and aggressive cardiovascular disease with high disability and mortality rates. Statistics show that once acute thoracic dissecting aneurysms, especially dissecting aneurysms of type i, are found, their one-month mortality rate can reach 70% -80% if not actively managed. The thoracic aortic dissection is a pathological state that on the basis of the existence or nonexistence of the aortic wall, the aortic intima is torn under the action of a series of possible external factors (such as hypertension, trauma and the like), blood enters the middle layer of the aortic wall from the intima laceration port to cause the separation of the aortic media along the long axis, and the lumen presents true and false two cavities. The dissection good hair part is in an arc area from the root of the aorta to the far side of the opening of the left subclavian artery. The disease is sudden, and the patient can have severe pain, shock and compression symptoms. A small number of patients can rapidly die due to cardiac tamponade, massive hemorrhage, malignant hypertension, severe aortic valve regurgitation and persistent ischemia of the myocardium, central nerves and kidneys, etc. Therefore, the timely diagnosis and treatment have a vital effect on the life and the life quality of the patient.

The examination of thoracic aortic dissections is now commonly aided by medical imaging of the chest. Such as CT, X-ray, nuclear magnetic resonance, etc. Among them, OCT has become an emerging imaging modality with the highest imaging resolution in the vascular lumen, and its importance is gradually highlighted in the diagnosis of thoracic aortic dissection diseases. However, considering that a plurality of complex anatomical structures such as a false lumen, a true lumen and thrombus exist in a thoracic aorta interlayer, and due to the characteristics of three layers of tissue structures of an artery wall, backscattering exists in OCT imaging to cause attenuation and permeation of signals, speckle noise interference exists at the same time, and a weak edge phenomenon is finally formed, which brings difficulty for research on automatic segmentation of an OCT image.

Disclosure of Invention

The invention aims to provide a multi-modal medical image conversion method based on artificial intelligence, which combines a chest aorta interlayer image with multiple modalities, fuses structural features in an ultrasonic image into an OCT image, and effectively enhances the features of a real cavity, a false cavity and thrombus in the fused multi-modal image.

In order to achieve the purpose, the invention is realized by the following technical scheme:

a multi-modal medical image conversion method based on artificial intelligence comprises the following steps:

acquiring OCT image data of an aortic dissection;

acquiring IVUS image data of an aortic dissection;

respectively adopting a global threshold segmentation method for the background of OCT image data and IVUS image data to extract the outline of the aorta, and cleaning noise information by using a morphological hole filling and area threshold method;

and carrying out image information fusion on the OCT image data and the IVUS image data by a frequency domain information fusion method of discrete cosine transform to obtain a fused image.

The OCT image data of the aortic dissection is acquired by adopting the laser wavelength lambda of 924 nanometers, the longitudinal resolution of 7 micrometers and the penetration capacity of 1.725 millimeters.

The method for acquiring the OCT image data of the aortic dissection comprises the steps of scanning under the states that the axial rotation angle is 0 degrees, +90 degrees, -90 degrees and 180 degrees, acquiring a four-angle OCT image, acquiring a two-dimensional OCT cross section image of the aortic dissection by registering the four-angle OCT image, extracting 200 layers of OCT cross section image data for each thoracic aortic blood vessel,

and the number of the first and second groups,

the acquiring IVUS image data of aortic dissection comprises extracting 200 layers of IVUS image data at a time from thoracic aortic vessels.

The image information fusion of the OCT image data and the IVUS image data comprises the following steps:

firstly, dividing overlapped subblocks of OCT image data and IVUS image data, wherein the block size is 8 x 8;

secondly, two-dimensional discrete cosine transform is respectively carried out on the obtained sub-block images to realize the transformation from a space domain to a frequency domain space, and the DCT transform uses a formula 1:

h and p are respectively coordinates in the frequency domain information, and the value range of h, p is 0, 1, 2 … …, N-1,

n represents the divided image block size,

the above-mentioned

Expression 2:

inverse DCT transform equation 3:

the i, j is a space domain coordinate variable, the value range of i, j is 0, 1, 2 … …, N-1, N represents the size of the divided image block,

the DC coefficient of the main part containing the image energy information can be obtained according to the formula 1, namely D (0, 0), and the rest D (h, p) values are AC coefficients;

the normalized transform coefficients can be represented by equation 4:

third, the variance of the block images is calculated, the mean μ and variance σ of each block image²The formula is as follows:

equation 5:

equation 6:

fourth, the fusion matrix is updated based on the variance result comparison,

and (4) selecting the inverse transformation of the sub-block DCT transformation result with larger variance to update the fusion matrix until the fusion of all the sub-blocks is completed, and obtaining the final fusion image matrix.

Compared with the prior art, the invention has the beneficial effects that:

by fusing the OCT image and the ultrasonic image, high-resolution thoracic aorta wall surface information is obtained, and the multimode fusion can effectively utilize different imaging advantages of the OCT and IVUS on a tissue focus area, so that the problem of weak edge structure information in a single OCT image is effectively solved. Deep feature characterization is carried out on the fused result image by adopting a deep neural network method, so that final effective detection of true lumen, false lumen and thrombus is realized. The problem of fuzzy edges of the false cavity and the thrombus in the thoracic aorta interlayer is effectively solved, the false cavity in the thoracic aorta interlayer can be effectively extracted, the thrombus in the false cavity can be accurately monitored, and a good image foundation is laid for further analysis of diseases.

Drawings

FIG. 1 is a schematic diagram of the basic process of image fusion according to the present invention.

Detailed Description

The invention will be further illustrated with reference to the following specific examples. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and these equivalents also fall within the scope of the present application.

The instruments, reagents, materials and the like used in the following examples are conventional instruments, reagents, materials and the like in the prior art and are commercially available in a normal manner unless otherwise specified. Unless otherwise specified, the experimental methods, detection methods, and the like described in the following examples are conventional experimental methods, detection methods, and the like in the prior art.

Example 1: multimode medical image conversion method based on artificial intelligence

Simply using the single gray scale information of the OCT image can only segment the outer wall and the inner wall of the artery, but cannot clearly describe the middle part of the intima-adventitia of the artery, that is, cannot accurately segment the false lumen and the thrombus boundary. Thus, the present example addresses the problem through multimodal information fusion.

The method specifically comprises the following steps:

1) obtaining OCT image data of aortic dissection

The blood vessel of the thoracic aorta interlayer is imaged in vitro by adopting Optical Coherence Tomography (OCT),

the OCT imaging technology is the current intravascular imaging technology with the highest resolution, can effectively display the inner wall structure of a blood vessel, identify the phenomena of tearing of the intima of the blood vessel, tissue components of plaque blocks, poor support bonding and the like, and further can provide a powerful detection tool for pathological changes (such as interlayers and tiny thrombi) occurring on the artery wall, support bonding conditions and the like.

The OCT image acquisition method of this example is:

scanning is carried out by adopting a laser wavelength lambda of 924 nanometers, a longitudinal resolution of 7 micrometers and a penetration capacity of 1.725 millimeters,

because the aorta diameter is great (about being located 2.5-3.5 centimetres interval), and OCT imaging technique penetrating power is weak (about between 1 millimeter-2 millimeters), when the vascular wall is because the intermediate layer is crescent thickened, it is difficult to clearly distinguish adventitia structure, can not directly carry out whole vascular wall imaging all around through the diameter promptly, and need form images many times in a plurality of discrete axle rotation angle directions, specifically: the rotation interval is 90 degrees,

scanning is carried out under the conditions that the axis rotation angle is 0 degrees, +90 degrees, -90 degrees and 180 degrees, four-angle OCT images are obtained, two-dimensional OCT cross section images of an aortic dissection are obtained by registering the four-angle OCT images, and 200 layers of OCT cross section image data are extracted for each thoracic aorta vessel.

2) Acquiring IVUS (ultrasound) image data of aortic dissection

IVUS has a high penetration capacity but shows a relatively low resolution, typically between 80 and 120 microns. It cannot distinguish tissue structures below 80 microns, i.e., the internal structures of the inner wall of the aorta. However, IVUS allows clear identification of the vessel wall and luminal structure of the aorta. Therefore, the position of the false cavity and the position of the thrombus can be roughly estimated by the position estimation method.

200 layers of IVUS image data were extracted for each thoracic aortic vessel.

3) And respectively adopting a global threshold segmentation method for the background of OCT image data and IVUS image data to extract the outline of the aorta, and cleaning noise information by using a morphological hole filling and area threshold method.

The method comprises the following steps: OCT image data/IVUS image data-binary segmentation based on global threshold-morphological hole filling-background noise region removal-blood vessel region extraction.

4) Carrying out image information fusion on the OCT image data and the IVUS image data to obtain a fused image

The information fusion is carried out on the two imaging images, so that the characteristic information of the false cavity and the thrombus area in the vascular interlayer wall is enriched, and the accuracy of the subsequent links on characteristic learning is improved.

The method is characterized in that OCT image data and IVUS image data are fused by a frequency domain information fusion method of discrete cosine transform, and the method comprises the following specific steps:

firstly, dividing overlapped subblocks of OCT image data and IVUS image data, wherein the block size is 8 x 8;

secondly, two-dimensional discrete cosine transform is respectively carried out on the obtained sub-block images to realize the transformation from a space domain to a frequency domain space, and the formula of DCT transform is as follows:

equation 1:

and h and p are coordinates in the frequency domain information respectively, and the value range of h is 0, 1, 2 … …, N-1, and N represents the size of the divided image block.

The above-mentioned

The expression is as follows:

expression 2:

the inverse DCT transform is defined as follows:

equation 3:

the i, j is a spatial domain coordinate variable, whose value range is i, j is 0, 1, 2 … …, N-1, N represents the size of the divided image block, a DC coefficient including the main part of the image energy information, i.e., D (0, 0), can be obtained according to formula 1, and the remaining D (h, p) values are AC coefficients.

The normalized transform coefficients may be expressed by the following formula:

expression 4:

third, the variance of the block image is calculated

Mean μ and variance σ for each block image²The formula is as follows:

equation 5:

equation 6:

fourth, fusion matrix updates are compared based on variance results

The variance obtained through calculation is used as an index for measuring image contrast information, the greater the variance is, the clearer the image contains details, and the richer the features are. And (4) selecting the inverse transformation of the DCT transformation result of the subblock with larger variance to update the element value of the corresponding position in the fusion matrix 1_ fusion, and so on until the fusion of all the subblocks is completed, thus obtaining the final fusion image matrix.

Claims (4)

1. A method for converting multi-modal medical images based on artificial intelligence is characterized by comprising the following steps:

acquiring OCT image data of an aortic dissection;

acquiring IVUS image data of an aortic dissection;

respectively adopting a global threshold segmentation method for the background of OCT image data and IVUS image data to extract the outline of the aorta, and cleaning noise information by using a morphological hole filling and area threshold method;

and carrying out image information fusion on the OCT image data and the IVUS image data by a frequency domain information fusion method of discrete cosine transform to obtain a fused image.

2. The method as claimed in claim 1, wherein the OCT image data of the aortic dissection is acquired with a laser wavelength λ of 924 nm, a longitudinal resolution of 7 μm, and a penetration capability of 1.725 mm.

3. The method of claim 1, wherein the obtaining of OCT image data of aortic dissection comprises scanning at axial rotation angles of 0 °, +90 °, -90 °, and 180 ° to obtain four-angle OCT images, obtaining two-dimensional OCT cross-sectional images of aortic dissection by registering the four-angle OCT images, extracting 200 layers of OCT cross-sectional image data for each thoracic aortic vessel,

and the number of the first and second groups,

the acquiring IVUS image data of aortic dissection comprises extracting 200 layers of IVUS image data at a time from thoracic aortic vessels.

4. The method for converting multi-modal medical image based on artificial intelligence as claimed in claim 1, wherein the image information fusion of the OCT image data and the IVUS image data comprises the following steps:

firstly, dividing overlapped subblocks of OCT image data and IVUS image data, wherein the block size is 8 x 8;

secondly, two-dimensional discrete cosine transform is respectively carried out on the obtained sub-block images to realize the transformation from a space domain to a frequency domain space, and the DCT transform uses a formula 1:

h and p are respectively coordinates in the frequency domain information, and the value range of h, p is 0, 1, 2 … …, N-1,

n represents the divided image block size,

the above-mentioned

Expression 2:

inverse DCT transform equation 3:

the i, j is a space domain coordinate variable, the value range of i, j is 0, 1, 2 … …, N-1, N represents the size of the divided image block,

the DC coefficient of the main part containing the image energy information can be obtained according to the formula 1, namely D (0, 0), and the rest D (h, p) values are AC coefficients;

the normalized transform coefficients can be represented by equation 4:

third, the variance of the block images is calculated, the mean μ and variance σ of each block image²The formula is as follows:

equation 5:

equation 6:

fourth, the fusion matrix is updated based on the variance result comparison,

and (4) selecting the inverse transformation of the sub-block DCT transformation result with larger variance to update the fusion matrix until the fusion of all the sub-blocks is completed, and obtaining the final fusion image matrix.

Images