CN111640106A - Multimode medical image conversion method based on artificial intelligence - Google Patents

Abstract

The invention discloses a multi-modal medical image conversion method based on artificial intelligence, which mainly relates to the field of medical images. Including obtaining OCT image data of aortic dissection; obtaining IVUS image data of aortic dissection; using global threshold segmentation method for the background of OCT image data and IVUS image data respectively, extracting the outer contour of the aorta, and using morphological hole filling and area threshold method to clean up the noise information; through the frequency domain information fusion method of discrete cosine transform, the image information of OCT image data and IVUS image data is fused to obtain the fusion image. The beneficial effect of the present invention is that it combines the thoracic aortic dissection images of multiple modalities, fuses the structural features in the ultrasound image into the OCT image, and effectively enhances the true cavity and the false cavity in the fused multimodal image. and thrombus characteristics.

Description

A method for converting multimodal medical images based on artificial intelligence

technical field

The invention relates to the field of medical imaging, in particular to a multimodal medical imaging conversion method based on artificial intelligence.

Background technique

In recent years, with the rapid development of a variety of medical imaging technologies (CT, MRI, Utrasound, PET, OCT, Microscop, etc.), they have been widely used in the early detection, diagnosis and treatment of diseases. Experts need to face a large amount of patient medical image data every day. Simply relying on manual reading and judgment of these image data is not only time-consuming and labor-intensive, but also subjective. Therefore, computer aided diagnosis technology has become a powerful measure to solve this problem. In particular, the introduction of "deep learning", a machine learning method that has emerged in recent years, has gradually accelerated the pace of modern computer medical image computing and analysis toward intelligence and precision.

Thoracic aortic dissection is a very sinister and dangerous cardiovascular disease with high morbidity and mortality. Statistics show that once an acute thoracic dissecting aneurysm is found, especially a type I dissecting aneurysm, its one-month mortality rate can reach 70%-80% if it is not actively treated. Thoracic aortic dissection refers to the tear of the aortic intima under the action of a series of possible external factors (such as hypertension, trauma, etc.) on the basis of the presence or absence of its own pathological changes in the aortic wall. The rupture enters the middle layer of the aortic wall, causing the middle layer of the aorta to separate along the long axis, and the lumen presents a pathological state of true and false lumens. The prevalent site of dissection is the arcuate region from the aortic root to the opening of the left subclavian artery. The onset of the disease is sudden, with severe pain, shock, and compression. A small number of patients can die rapidly due to cardiac tamponade, massive hemorrhage, malignant hypertension, severe aortic regurgitation, and persistent ischemia of the myocardium, central nervous system, and kidneys. Therefore, timely diagnosis and treatment play a vital role in the patient's life and quality of life.

At present, the examination of thoracic aortic dissection is generally supplemented by medical imaging of the chest. Such as CT, X-ray, nuclear magnetic resonance and so on. Among them, OCT has become an emerging imaging modality with the highest resolution in vascular intraluminal imaging, and has gradually highlighted its important position in the diagnosis of thoracic aortic dissection. However, considering that there are various complex anatomical structures of false lumen, true lumen and thrombus in thoracic aortic dissection, and due to the characteristics of the three-layer tissue structure of the arterial wall, there will be backscattering in OCT imaging, resulting in signal" At the same time, there is the interference of speckle noise, which eventually forms a weak edge phenomenon, which brings difficulties to the automatic segmentation of OCT images.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a multi-modal medical image conversion method based on artificial intelligence, which combines multiple modalities of thoracic aortic dissection images, fuses the structural features in the ultrasound image into the OCT image, and fuses the structural features in the OCT image. Effective enhancement of true lumen, false lumen and thrombus features in multimodal images of

The present invention is achieved by the following technical solutions in order to achieve the above object:

An artificial intelligence-based multimodal medical image conversion method, including:

Obtain OCT image data of aortic dissection;

Obtain IVUS image data of aortic dissection;

The background of the OCT image data and the IVUS image data was segmented by global threshold, the outer contour of the aorta was extracted, and the noise information was cleaned up by morphological hole filling and area threshold methods;

Through the frequency domain information fusion method of discrete cosine transform, the OCT image data and the IVUS image data are fused to obtain the fusion image.

The OCT image data of the aortic dissection is obtained by using a laser with a wavelength λ of 924 nm, a longitudinal resolution of 7 μm, and a penetration capacity of 1.725 mm.

The obtaining of the OCT image data of the aortic dissection includes scanning in a state where the axis rotation angles are 0°, +90°, -90°, and 180° to obtain a four-angle OCT image, which is obtained by registering the four-angle OCT image. Two-dimensional OCT cross-sectional images of aortic dissection, for each thoracic aortic vessel, 200 slices of OCT cross-sectional image data were extracted,

as well as,

The acquiring IVUS image data of aortic dissection includes extracting 200 slices of IVUS image data of the thoracic aorta each time.

Described carrying out image information fusion with OCT image data and IVUS image data, comprises the following steps:

First, the OCT image data and the IVUS image data are divided into overlapping sub-blocks, and the block size is 8*8;

Second, two-dimensional discrete cosine transform is performed on the obtained sub-block images to realize the transformation from the space domain to the frequency domain space. The DCT transform uses formula 1:

The h, p are the coordinates in the frequency domain information respectively, and the value range is h, p=0, 1, 2..., N-1,

N represents the divided image block size,

表达式2：said

Expression 2:

Inverse DCT transform formula 3:

The i, j are coordinate variables in the space domain, and the value range is i, j=0, 1, 2..., N-1, N represents the size of the divided image block,

According to formula 1, the DC coefficient containing the main part of the image energy information can be obtained, that is, D(0, 0), and the rest D(h, p) are AC coefficients;

The normalized transform coefficient can be obtained from Equation 4:

Third, calculate the variance of the block image, the average μ and variance σ ² of each block image are as follows:

Formula 5:

Formula 6:

Fourth, the fusion matrix is updated based on the variance result comparison,

Select the inverse transform of the DCT transform result of the sub-block with larger variance to update the fusion matrix, until the fusion of all sub-blocks is completed, the final fusion image matrix can be obtained.

Compared with the prior art, the beneficial effects of the present invention are:

By fusing OCT images with ultrasound images, high-resolution thoracic aortic wall information can be obtained. Multimodal fusion can effectively take advantage of the different imaging advantages of OCT and IVUS in tissue lesion areas, and effectively compensate for weak edges in a single OCT image. Problems with structural information. We use the deep neural network method to perform deep feature representation for the fused result images, so as to achieve the final effective detection of true lumen, false and thrombus. It effectively solves the blurred edge of false lumen and thrombus in thoracic aortic dissection. It can not only effectively extract false lumen in thoracic aortic dissection, but also accurately monitor thrombus in false lumen, laying a solid foundation for further analysis of the disease. image base.

Description of drawings

FIG. 1 is a schematic diagram of the basic flow of the image fusion of the present invention.

Detailed ways

The present invention will be further described below in conjunction with specific embodiments. It should be understood that these examples are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the teaching content of the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the present application.

The instruments, reagents, materials, etc. involved in the following examples, unless otherwise specified, are all conventional instruments, reagents, materials, etc. existing in the prior art, and can be obtained through regular commercial channels. The experimental methods, detection methods, etc. involved in the following examples, unless otherwise specified, are all conventional experimental methods, detection methods, etc. in the prior art.

Example 1: A method for converting multimodal medical images based on artificial intelligence

Simply using the single grayscale information of the OCT image can only segment the outer and inner walls of the artery, but it cannot clearly describe some details from the intima to the adventitia of the artery, that is, it cannot accurately segment the false lumen and the inner wall. Thrombus borders. Therefore, this example solves the problem through multimodal information fusion.

It specifically includes the following steps:

1) Obtain OCT image data of aortic dissection

In vitro imaging of vessels with thoracic aortic dissection using optical coherence tomography (OCT),

OCT imaging technology is currently the highest resolution intraluminal imaging technology, which can effectively display the inner wall structure of blood vessels, identify the tearing of the vascular intima, plaque tissue components, poor stent fit and other phenomena, and then can be used for the arterial wall. The occurrence of lesions (eg, dissection, microthrombosis) and stent fit provide powerful detection tools.

The OCT image acquisition method for this example is:

The laser wavelength λ is 924 nanometers, the longitudinal resolution is 7 microns, and the penetration ability is 1.725 mm for scanning.

Due to the large diameter of the aorta (about 2.5-3.5 cm) and the weak penetration of OCT imaging (about 1 mm-2 mm), it is difficult to clearly distinguish the blood vessels when the vessel wall is gradually thickened due to dissection The adventitial structure, that is, cannot be directly imaged around the complete blood vessel wall through the diameter, but needs to be imaged multiple times in multiple discrete axis rotation angles, specifically: with 90° as the rotation interval,

Scan under the state of axis rotation at 0°, +90°, -90°, and 180° to obtain a four-angle OCT image, and obtain a two-dimensional OCT cross-sectional image of aortic dissection by registering the four-angle OCT image. For each thoracic aortic vessel, 200 slices of OCT cross-sectional image data were extracted.

2) Obtain IVUS (ultrasound) image data of aortic dissection

IVUS has high penetrating power, but shows relatively low resolution, usually between 80-120 microns. Therefore, it cannot distinguish tissue structures below 80 microns, that is, the internal structure of the inner wall of the aorta. However, IVUS can clearly identify the wall and luminal structure of the aorta. Therefore, it can be used to roughly estimate the location of false lumen and thrombus.

200 slices of IVUS image data were extracted for each thoracic aortic vessel.

3) The background of the OCT image data and the IVUS image data was segmented by global threshold, the outer contour of the aorta was extracted, and the noise information was cleaned up by the morphological hole filling and area threshold methods.

The steps are: OCT image data/IVUS image data - binarization segmentation based on global threshold value - morphological hole filling - background noise removal area - blood vessel area extraction.

4) Fusion of OCT image data and IVUS image data to obtain a fused image

By information fusion of the two imaging images, the feature information of the false lumen and the thrombus region in the vessel dissection wall is enriched, and the accuracy of feature learning in subsequent links is improved.

Through the frequency domain information fusion method of discrete cosine transform, the OCT image data and the IVUS image data are fused. The specific steps are:

First, the OCT image data and the IVUS image data are divided into overlapping sub-blocks, and the block size is 8*8;

Second, two-dimensional discrete cosine transform is performed on the obtained sub-block images to realize the transformation from the space domain to the frequency domain space. The formula of DCT transform is as follows:

Formula 1:

The h and p are the coordinates in the frequency domain information respectively, and the value range is h, p=0, 1, 2..., N-1, and N represents the size of the divided image blocks.

表达式如下：said

The expression is as follows:

Expression 2:

The inverse DCT transform is defined as follows:

Formula 3:

The i, j are coordinate variables in the space domain, and the value range is i, j=0, 1, 2..., N-1, N represents the size of the divided image block, according to formula 1, the main elements including the image energy information can be obtained. Part of the DC coefficients, namely D(0, 0), and the rest D(h, p) are AC coefficients.

The normalized transform coefficients can be expressed by the following formula:

Expression 4:

Third, calculate the variance of the tiled image

The formulas for the mean μ and variance σ ² of each segmented image are as follows:

Formula 5:

Formula 6:

Fourth, update the fusion matrix based on the variance result comparison

The variance obtained by calculation is used as an index to measure the contrast information of the image. The larger the variance, the clearer the image, the clearer the details and the richer the features. Select the inverse transform of the DCT transform result of the sub-block with large variance to update the element value of the corresponding position in the fusion matrix 1_fusion, and so on, until the fusion of all sub-blocks is completed, and the final fusion image matrix can be obtained.

Claims (4)

1. a method for converting multimodal medical images based on artificial intelligence, is characterized in that, comprises: Obtain OCT image data of aortic dissection; Obtain IVUS image data of aortic dissection; The background of the OCT image data and the IVUS image data was segmented by global threshold, the outer contour of the aorta was extracted, and the noise information was cleaned up by morphological hole filling and area threshold methods; Through the frequency domain information fusion method of discrete cosine transform, the OCT image data and the IVUS image data are fused to obtain the fusion image. 2. the conversion method of a kind of artificial intelligence-based multimodal medical image according to claim 1, is characterized in that, the described acquisition of the OCT image data of aortic dissection is 924 nanometers for using laser wavelength λ, and the longitudinal resolution is 924 nanometers. The acquisition was performed at 7 microns with a penetration capacity of 1.725 mm. 3. a kind of artificial intelligence-based multi-modal medical image conversion method according to claim 1, is characterized in that, described acquisition of the OCT image data of aortic dissection, including in the axis rotation angle is 0 °, +90 ° Scan at -90°, 180° to obtain four-angle OCT images, and obtain two-dimensional OCT cross-sectional images of aortic dissection by registering the four-angle OCT images. For each thoracic aortic vessel, extract 200 layers of OCT cross-sectional image data, as well as, The acquiring IVUS image data of aortic dissection includes extracting 200 slices of IVUS image data of the thoracic aorta each time. 4. the conversion method of a kind of artificial intelligence-based multimodal medical image according to claim 1, is characterized in that, described carrying out image information fusion with OCT image data and IVUS image data, comprises the following steps: First, the OCT image data and the IVUS image data are divided into overlapping sub-blocks, and the block size is 8*8; Second, two-dimensional discrete cosine transform is performed on the obtained sub-block images to realize the transformation from the space domain to the frequency domain space. The DCT transform uses formula 1:

The h, p are the coordinates in the frequency domain information respectively, and the value range is h, p=0, 1, 2..., N-1, N represents the divided image block size, said

Expression 2:

Inverse DCT transform formula 3:

The i, j are coordinate variables in the space domain, and the value range is i, j=0, 1, 2..., N-1, N represents the size of the divided image block, According to formula 1, the DC coefficient containing the main part of the image energy information can be obtained, that is, D(0, 0), and the rest D(h, p) are AC coefficients; The normalized transform coefficient can be obtained from Equation 4:

Third, calculate the variance of the block image, the average μ and variance σ ² of each block image are as follows: Formula 5:

Formula 6:

Fourth, the fusion matrix is updated based on the variance result comparison, Select the inverse transform of the DCT transform result of the sub-block with large variance to update the fusion matrix, until the fusion of all sub-blocks is completed, the final fusion image matrix can be obtained.

Images