KR20230113062A - Vascular heterogeneity model-based image reconstruction apparatus and method - Google Patents

Abstract

An image restoration method performed by an image restoration apparatus according to an embodiment includes the steps of separating magnetic resonance image data obtained in a space-time encoding domain into image intensity information and image phase information, and based on a pre-learned vascular system polymorphism model. performing limited processing for each blood vessel scale separated into a macrovascular system signal and a microvascular system signal through vasculature adaptive image processing on the image intensity information; Selectively performing detailed processing on signals, removing artifacts from the image phase information, and forcing physical consistency on the image intensity information on which the detailed processing has been performed and the image phase information from which the artifacts have been removed. It includes the step of combining the solution.

Description

Vascular system polymorphism model-based image restoration apparatus and method {VASCULAR HETEROGENEITY MODEL-BASED IMAGE RECONSTRUCTION APPARATUS AND METHOD}

The present invention relates to an image restoration device and an image restoration method for visualizing the structure and function of a vascular system based on a vascular heterogeneity model with respect to magnetic resonance imaging (MRI) data.

In the case of the prior art for obtaining a functional map of the macro-vascular system and the micro-vascular system, it is necessary to inject a contrast medium twice to obtain a functional map of the macro-vascular system image and the micro-vascular system image. That is, the prior art cannot simultaneously restore two types of information from a single magnetic resonance image data through a single contrast agent injection.

In addition, in the prior art, in order to extract a microvascular system functional map, an image is reconstructed from raw data of k-space, and microvascular system function information is extracted from the reconstructed image through a pattern recognition method. Therefore, the prior art requires two processes (ie, image restoration and pattern analysis from original data) to extract microvascular function information. That is, the prior art cannot simultaneously extract macro-vascular system structure and micro-vascular system function maps directly from raw data. Therefore, the prior art cannot efficiently eliminate artifact generation and noise amplification that may occur in an intermediate step.

In the prior art, image restoration and tissue image post-processing (segmentation) must be performed separately. That is, in the prior art, contrast enhancement image restoration and image post-processing are not implemented in a single mathematical frame.

Republic of Korea Patent Registration No. 2204371, registration date January 12, 2021.

According to an embodiment, after separating magnetic resonance image data into image intensity information and image phase information, vascular system adaptive image processing is performed on the image intensity information based on a vascular system polymorphism model, and artifacts are removed. Provided is an image restoration method and apparatus combining image phase information.

The problem to be solved by the present invention is not limited to those mentioned above, and another problem to be solved that is not mentioned will be clearly understood by those skilled in the art from the description below.

An image restoration method performed by the image restoration apparatus according to the first aspect includes the steps of separating magnetic resonance image data acquired in a space-time encoding domain into image intensity information and image phase information, and based on a pre-learned vascular system polymorphism model. performing limited processing for each blood vessel scale separated into a macrovascular system signal and a microvascular system signal through vasculature adaptive image processing on the image intensity information; Selectively performing detailed processing on signals, removing artifacts from the image phase information, and forcing physical consistency on the image intensity information on which the detailed processing has been performed and the image phase information from which the artifacts have been removed. It includes the step of combining the solution.

Here, the magnetic resonance image data may be obtained from a single contrast enhancement data through a single contrast agent injection.

The magnetic resonance image data may be undersampled data in k-space, and before the magnetic resonance image data is separated into the image intensity information and the image phase information, the undersampled data in k-space and the k-space A coil sensitivity map estimated from the entire sampling data of the space may be preprocessed through a sensitivity linear operator.

In the performing of the limiting processing for each vessel scale, the signal limiting level of the macrovascular signal may be lowered while the signal limiting level of the microvascular system signal may be increased.

In the step of selectively performing detailed processing, functional segmentation may be performed on the microvascular system signal through expression of a low-dimensional image signal.

The magnetic resonance image data acquired in the space-time encoding domain may be undersampled data in k-space, and the combining by forcing the physical coherence may include the undersampled data and the detailed-processed image intensity information. The physical coherence may be enforced in a direction of reducing a gap between , and between the undersampled data and image phase information from which the artifact is removed.

After repeatedly performing the limiting processing for each vessel scale, the selectively performing the detailed processing, the removing the artifact, and the forcibly combining the physical consistency, the artifact is changed. The removed macroscopic vascular system structure image and microvascular system function image may be acquired at the same time.

An image restoration apparatus according to a second aspect includes a memory for storing one or more programs and a processor for executing the one or more programs, wherein the processor converts magnetic resonance image data obtained in a space-time encoding domain to image intensity information. and image phase information, and performs limited processing for each blood vessel scale separated into macrovascular system signals and microvascular system signals through vascular system adaptive image processing for the image intensity information based on the previously learned vascular system polymorphism model, Among the image intensity information subject to limited processing for each vessel scale, detailed processing is selectively performed on the microvasculature signal, artifacts are removed from the image phase information, and the image intensity information on which the detailed processing has been performed and the artifact are The removed image phase information is combined by forcing physical consistency.

A computer readable recording medium storing a computer program according to the third aspect includes instructions for causing the processor to perform the image restoration method when the computer program is executed by a processor.

A computer program stored in a computer readable recording medium according to a fourth aspect includes instructions for causing the processor to perform the image restoration method when the computer program is executed by a processor.

According to an embodiment of the present invention, magnetic resonance image data is separated into image intensity information and image phase information, vasculature-adaptive image processing is performed on the image intensity information based on a vascular system polymorphism model, and image phase from which artifacts are removed. combine with information Accordingly, it is possible to simultaneously restore the structure of the macrovascular system and the function of the microvascular system based on the vascular system polymorphism model from the contrast-enhanced magnetic resonance image data through a single contrast agent injection.

In addition, by adjusting the parameters related to the image restoration process differently based on the vascular polymorphism model instead of applying the same signal penalty level to the entire vascular system, the macroscopic vascular system signal has high signal intensity and the change information and size of blood flow in the vascular system It is possible to extract without loss of information, and extract the noise-removed basis vector in the time direction and the corresponding functional map from the microvascular system signal.

1 is a configuration diagram of an image restoration system including an image restoration apparatus according to an embodiment of the present invention.
2 is a configuration diagram of an image restoration apparatus according to an embodiment of the present invention shown in FIG. 1;
3 is a network configuration diagram of an artificial neural network model that may be included in a processor in an image restoration apparatus according to an embodiment of the present invention.
FIG. 4 is a configuration diagram of a deep attention artificial neural network layer in which a vascular contrast agent is dynamically weighted shown in FIG. 3;
5 is a configuration diagram illustrating the attention mechanism shown in FIG. 4;
6 is a configuration diagram of an artificial neural network layer for removing artifacts shown in FIG. 3;
7 is a network configuration diagram of an artificial neural network model that may be included in a processor in an image restoration apparatus according to another embodiment of the present invention.
8 is a flowchart illustrating an image restoration method performed by a processor in an image restoration apparatus according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the embodiments described below in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims.

The terms used in this specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected from general terms that are currently widely used as much as possible while considering the functions in the present invention, but these may vary depending on the intention of a person skilled in the art or precedent, the emergence of new technologies, and the like. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, not simply the name of the term.

In the entire specification, when a part is said to 'include' a certain component, it means that it may further include other components, rather than excluding other components, unless otherwise stated.

In addition, the term 'unit' used in the specification means software or a hardware component such as FPGA or ASIC, and 'unit' performs certain roles. However, 'wealth' is not limited to software or hardware. 'Unit' may be configured to reside in an addressable storage medium and may be configured to reproduce one or more processors. Thus, as an example, 'unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and 'parts' may be combined into a smaller number of elements and 'parts' or further separated into additional elements and 'parts'.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted.

1 is a configuration diagram of an image restoration system including an image restoration apparatus according to an embodiment of the present invention, and FIG. 2 is a configuration diagram of an image restoration apparatus according to an embodiment of the present invention shown in FIG. 1 .

Referring to FIGS. 1 and 2 , the image restoration system 100 according to an embodiment separates magnetic resonance image data into image intensity information and image phase information, and based on a vasculature polymorphism model pre-learned for the image intensity information, After performing vasculature adaptive image processing, an image restoration device 110 combining image phase information from which artifacts have been removed is included.

Here, the magnetic resonance image data provided to the image restoration apparatus 110 may be data 10 undersampled in k-space from single contrast enhancement data through a single contrast agent injection. In addition, the coil sensitivity map 20 estimated from the undersampled data 10 in the k-space and the entire sampling data in the k-space is preprocessed through the sensitivity linear operator 101 and then provided to the image restoration apparatus 110. It can be. In this way, data input to the image reconstruction apparatus 110 may be a complex-valued undersampled dynamic contrast enhanced image (30). In this case, the image reconstruction apparatus 110 may simultaneously acquire and provide a macro-vascular system structure image (complex-valued reconstructed dynamic contrast enhanced image) 50 and a micro-vascular system function image 60 from which artifacts are removed. Here, the image reconstruction apparatus 110 uses a complex-valued reference dynamic contrast enhanced image (40) and mean squared error (MSE) loss to determine the optimal parameters. It can include pre-trained artificial neural network models in an end-to-end supervised learning method using functions.

The image restoration device 110 includes a memory 111 and a processor 112 .

The memory 111 stores one or more programs for acquiring the macroscopic vascular system structure image 50 and the microvascular system function image 60 from which artifacts are removed.

The processor 112 executes one or more programs stored in the storage unit 111 .

The processor 112 separates the magnetic resonance image data acquired in the space-time encoding domain into image intensity information and image phase information, and macroscopically analyzes the image intensity information based on the pre-learned vasculature polymorphism model through vasculature-adaptive image processing. Limited processing is performed for each vessel scale that is divided into vascular system signal and microvascular system signal, selectively performs detailed processing on the microvascular system signal among the image intensity information on which limited processing for each vessel scale has been performed, and artifacts are removed for the image phase information. removed, and the image intensity information on which detailed processing has been performed and the image phase information on which artifacts have been removed are combined by forcing physical consistency.

Here, when the processor 112 performs limiting processing for each blood vessel scale, it is possible to lower the penalty level for macrovascular signals and increase the penalty level for microvascular signals. For example, the processor 112 may perform functional segmentation on the microvasculature signal through expression of a low-dimensional image signal.

Further, when the processor 112 enforces physical consistency, the image phase information from which the undersampled data and the artifacts are removed between the undersampled data in the k-space and the intensity information of the image on which detailed processing has been performed, and the undersampled data in the k-space Physical consistency can be enforced in the direction of reducing the gap between

Meanwhile, the processor 112 repeatedly performs limited processing for each vessel scale, selectively performing detailed processing, removing artifacts, and combining by forcing physical consistency, and then, according to the iterative execution, It is possible to simultaneously acquire a macroscopic vascular system structure image and a microvascular system function image from which artifacts are removed.

3 is a network configuration diagram of an artificial neural network model that may be included in the processor 112 in the image restoration apparatus 110 according to an embodiment of the present invention, and FIG. 4 is a dynamic weighted deep attention artificial neural network model shown in FIG. It is a configuration diagram of a neural network layer (vascular contrast dynamics-weighted deep attention 3D U-Net layer), FIG. 5 is a configuration diagram showing the attention mechanism shown in FIG. 4, and FIG. 6 is an artificial neural network layer for removing artifacts shown in FIG. (3D U-Net layer), Figure 7 is a network configuration diagram of an artificial neural network model that can be included in the processor 112 in the image restoration device 110 according to another embodiment of the present invention, Figure 8 is A flowchart illustrating an image restoration method performed by the processor 112 in the image restoration apparatus 110 according to an embodiment of the present invention.

Hereinafter, an image restoration process performed by the image restoration system 100 including the image restoration apparatus 110 according to an embodiment of the present invention will be described with reference to FIGS. 1 to 8 .

First, data 10 undersampled in the k-space encoding domain from single contrast enhancement data through single contrast agent injection is input to the sensitivity linear operator 101, and the sensitivity linear operator 101 outputs the entire sampling data of the k-space. The undersampled data 10 is preprocessed using the coil sensitivity map 20 estimated in , and the undersampled dynamic contrast agent enhanced image 30 of complex value is provided to the image restoration apparatus 110 .

The processor 112 of the image reconstruction device 110 executes one or more programs for acquiring the macro-vascular system structure image 50 and the micro-vascular system function image 60 from which artifacts are removed stored in the memory 111, or In the executed state, image restoration of the complex-valued undersampled dynamic contrast agent enhanced image 30 starts.

The processor 112 divides the complex-valued undersampled dynamic contrast agent-enhanced image 30 into image intensity information and image phase information, and then performs parallel processing on the image intensity information and image phase information in two branches. do. For example, parallel processing may be performed through a branch processing image emphasis information and a branch processing image phase information, such as the deep attention artificial neural network layer with dynamic weighting of the vascular system contrast agent shown in FIG. 3 ( S810 ).

In one branch, the processor 112 performs limited processing for each blood vessel scale, which is divided into a macrovascular system signal and a microvascular system signal, through vascular system adaptive image processing on image intensity information based on a pre-learned vascular system polymorphism model. For example, the processor 112 may lower the signal limiting level for macrovascular signals and increase the signal limiting level for microvascular signals. For example, the processor 112 may perform noise removal and macrovascular mask estimation on the macrovascular system signal using the vascular contrast agent dynamically weighted deep attention artificial neural network layer shown in FIGS. 3 and 4, and the noise removed An image and an estimated macroscopic vasculature mask may be generated (S821). Further, the processor 112 uses the multilayered non-negative matrix factorization (NMF) layer shown in FIG. 3 to perform microvasculature signals among the image intensity information subjected to limited processing for each vessel scale. Selectively perform detailed processing for . For example, by performing functional segmentation through low-dimensional image signal expression on the microvascular system signal passing through the macrovascular system mask, it is possible to characterize the capillary tissue (S822).

Then, the processor 112 removes artifacts included in image phase information in other branches. For example, the processor 112 may remove artifacts from the image phase information using the 3D U-Net layer shown in FIGS. 3 and 6 (S831).

Subsequently, the processor 112 enforces physical consistency with respect to the detailed processed image intensity information and the artifact-removed image phase information. For example, the processor 112 may make the image intensity information have physical consistency using the magnitude data consistency layer shown in FIG. 3 (S823), and the phase data consistency layer (phase data consistency layer). layer) may be used to have physical consistency with respect to image phase information (S833). For example, the processor 112 reduces the gap between data undersampled in k-space and image intensity information on which detailed processing has been performed, and between data undersampled in k-space and image phase information from which artifacts are removed. to enforce physical consistency.

Further, the processor 112 outputs an image in which the structure of the macrovascular system and the function of the microvascular system are simultaneously restored by combining the image intensity information and the image phase information for which physical consistency is enforced. For example, the processor 112 simultaneously acquires and provides the macroscopic vascular system structure image 50 and the microvascular system function image 60 from which artifacts are removed (S840).

Meanwhile, the processor 112 assists the aforementioned individual processing processes, that is, a process of performing limited processing for each vessel scale, a process of selectively performing detailed processing, a process of removing artifacts, and a process of forcing and combining physical consistency. As shown in Fig. 7, it can be performed repeatedly over a plurality of times, and as a result of repeated execution, a macroscopic vascular system structure image and a microvascular system function image from which artifacts are removed can be simultaneously acquired. In this case, the deep attention artificial neural network layer with dynamic weighting of the vascular system contrast agent and the artificial neural network layer for artifact removal (3D U-Net layer) may each share parameters such as a weight during a plurality of processing processes.

As described so far, according to an embodiment of the present invention, magnetic resonance image data is separated into image intensity information and image phase information, and vascular system adaptive image processing is performed on the image intensity information based on the vascular system polymorphism model. Combine the artifact-free image phase information. Accordingly, it is possible to simultaneously restore the structure of the macrovascular system and the function of the microvascular system based on the vascular system polymorphism model from the contrast-enhanced magnetic resonance image data through a single contrast agent injection.

In addition, by adjusting the parameters related to the image restoration process differently based on the vascular polymorphism model instead of applying the same signal limiting level to the entire vascular system, the macroscopic vascular signal can obtain information about changes in blood flow in the fast distance vascular system with high signal intensity and size of the information. It is extracted without loss, and the noise-removed basis vector in the time direction and the resulting functional map can be extracted from the microvascular system signal.

Meanwhile, each step included in the image restoration method according to the above-described embodiment may be implemented in a computer readable recording medium recording a computer program including instructions for performing these steps.

Combinations of each step in each flowchart attached to the present invention may be performed by computer program instructions. Since these computer program instructions may be loaded into a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions executed by the processor of the computer or other programmable data processing equipment function as described in each step of the flowchart. create a means to do them. These computer program instructions can also be stored on a computer usable or computer readable medium that can be directed to a computer or other programmable data processing equipment to implement functions in a particular way, so that the computer usable or computer readable It is also possible that the instructions stored on the recording medium produce an article of manufacture containing instruction means for performing the functions described in each step of the flowchart. The computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to generate computer or other programmable data processing equipment. Instructions for performing the processing equipment may also provide steps for executing the functions described at each step in the flowchart.

Further, each step may represent a module, segment or portion of code that includes one or more executable instructions for executing the specified logical function(s). It should also be noted that in some alternative embodiments it is possible for the functions mentioned in the steps to occur out of order. For example, two steps shown in succession may in fact be performed substantially concurrently, or the steps may sometimes be performed in reverse order depending on the function in question.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential qualities of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: video restoration system
101: sensitivity linear operator
110: image restoration device
111: memory
112: processor

Claims (16)

In the image restoration method performed by the image restoration device,
Separating magnetic resonance image data acquired in the space-time encoding domain into image magnitude information and image phase information;
Performing limited processing for each blood vessel scale separated into a macrovascular system signal and a microvascular system signal through vascular system adaptive image processing on the image intensity information based on the previously learned vascular system polymorphism model;
selectively performing detailed processing on microvasculature signals among the image intensity information on which limited processing for each vessel scale has been performed;
removing artifacts from the image phase information;
Comprising the step of combining by forcing physical consistency with respect to the image intensity information on which the detailed processing has been performed and the image phase information from which the artifact has been removed
How to restore video. According to claim 1,
The magnetic resonance image data is obtained from a single contrast enhancement data through a single contrast agent injection.
How to restore video. According to claim 2,
The magnetic resonance image data is undersampled data in k-space,
Before the magnetic resonance image data is separated into the image intensity information and the image phase information, the coil sensitivity map estimated from the undersampled data in the k-space and the entire sampling data in the k-space is preprocessed through a sensitivity linear operator. felled
How to restore video. According to claim 1,
The step of performing the limited treatment for each vessel scale,
The macrovasculature signal lowers the penalty level, while the microvascular signal increases the penalty level.
How to restore video. According to claim 1,
In the step of selectively performing detailed processing,
Functional segmentation through low-dimensional image signal expression for the microvascular system signal
How to restore video. According to claim 1,
The magnetic resonance image data obtained in the space-time encoding domain is data undersampled in k-space,
The step of forcing and combining the physical consistency,
Forcing the physical consistency in the direction of reducing the gap between the undersampled data and the image intensity information on which the detailed processing is performed and between the undersampled data and the artifact-removed image phase information.
How to restore video. According to claim 1,
After repeatedly performing the limiting processing for each vessel scale, the selectively performing the detailed processing, the removing the artifact, and the forcibly combining the physical consistency, the artifact is changed. Simultaneous acquisition of removed macroscopic vascular system image and microvascular system function image
How to restore video. a memory for storing one or more programs;
a processor for executing the stored one or more programs;
the processor,
The magnetic resonance image data obtained in the spatio-temporal encoding domain is separated into image intensity information and image phase information, and macrovascular signal and microvascular signal and microvascular signal are obtained through vascular system adaptive image processing for the image intensity information based on the previously learned vascular system polymorphism model. Limited processing is performed for each vessel scale separated into vascular system signals, selectively detailed processing is performed on microvascular system signals among the image intensity information on which the limited processing for each vessel scale has been performed, and artifacts are removed for the image phase information, Forcing and combining the image intensity information on which the detailed processing has been performed and the image phase information on which the artifact has been removed by forcing physical consistency
video restoration device. According to claim 8,
The magnetic resonance image data is obtained from a single contrast enhancement data through a single contrast agent injection.
video restoration device. According to claim 9,
The magnetic resonance image data is undersampled data in k-space,
The coil sensitivity map estimated from the undersampled data in the k-space and the entire sampling data in the k-space is preprocessed through a sensitivity linear operator and then provided to the processor
video restoration device. According to claim 8,
the processor,
The macrovasculature signal lowers the penalty level, while the microvascular signal increases the penalty level.
video restoration device. According to claim 8,
the processor,
Functional segmentation through low-dimensional image signal expression for the microvascular system signal
video restoration device. According to claim 8,
The magnetic resonance image data obtained in the space-time encoding domain is data undersampled in k-space,
the processor,
Forcing the physical consistency in the direction of reducing the gap between the undersampled data and the image intensity information on which the detailed processing is performed and between the undersampled data and the artifact-removed image phase information.
video restoration device. According to claim 8,
the processor,
After performing the limited processing for each vessel scale, selectively performing the detailed processing, removing the artifact, and forcibly combining the physical consistency, the artifact is removed according to the repetitive execution. Simultaneous acquisition of macro-vascular structure images and micro-vascular system function images
video restoration device. A computer-readable recording medium storing a computer program,
When the computer program is executed by a processor,
Separating magnetic resonance image data acquired in the spatio-temporal encoding domain into image intensity information and image phase information, and macrovascular signal through vasculature-adaptive image processing for the image intensity information based on a pre-learned vasculature polymorphism model performing limited processing for each vessel scale separated into the microvascular system signal, selectively performing detailed processing on the microvascular system signal among the image intensity information on which the limited processing for each vessel scale has been performed, and the image phase information Forcing the processor to perform an image restoration method comprising the step of removing artifacts for , and combining the image intensity information on which the detailed processing has been performed and the image phase information from which the artifacts have been removed by forcing physical consistency. A computer-readable recording medium containing instructions for As a computer program stored on a computer-readable recording medium,
When the computer program is executed by a processor,
Separating magnetic resonance image data acquired in the spatio-temporal encoding domain into image intensity information and image phase information, and macrovascular signal through vasculature-adaptive image processing for the image intensity information based on a pre-learned vasculature polymorphism model performing limited processing for each vessel scale separated into the microvascular system signal, selectively performing detailed processing on the microvascular system signal among the image intensity information on which the limited processing for each vessel scale has been performed, and the image phase information Forcing the processor to perform an image restoration method comprising the step of removing artifacts for , and combining the image intensity information on which the detailed processing has been performed and the image phase information from which the artifacts have been removed by forcing physical consistency. A computer program containing instructions for

Images