KR20180097214A - Method of predicting recurrence region and generating pet image using deep learning, and apparatus thereof - Google Patents

Abstract

Disclosed are a method and an apparatus for predicting a recurrence area by using deep learning to generate a PET image. The method according to an embodiment of the present invention comprises the steps of performing preprocessing on a plurality of images and predicting a recurrence area based on a plurality of preprocessed images, wherein the plurality of images are multimodal MR images including at least one of a T1 image, a T1CE image, a T2 image, a FLAIR image, and an ADC image.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for predicting a recurrence site using deep learning,

The following examples relate to methods and apparatus for predicting recurrence sites using deep running and generating PET images.

Deep learning is generally defined as a set of machine learning algorithms that try to achieve a high level of abstraction through a combination of several nonlinear transformation techniques and is one area of machine learning that teaches computers to people's minds in a large framework .

Many researches have been carried out to express some data in a form that can be understood by the computer (for example, in the case of images, pixel information is represented by a column vector) and applied to learning. As a result, various deep learning techniques such as deep neural networks, convolutional deep neural networks, and deep believe networks are applied to fields such as computer vision, speech recognition, natural language processing, and voice / signal processing.

Embodiments can provide a technique for predicting a brain tumor recurrence site from a multi-modal MR image of a patient suffering from a brain tumor.

Embodiments can also provide a technique for generating a PET image from a multi-modal MR image.

A method for predicting a recurrence site according to an exemplary embodiment includes performing a pre-processing on a plurality of images, and predicting a recurrence site based on a plurality of preprocessed images, wherein the plurality of images are T1- Modal (including at least one of an image, a Contrast-enhanced T1-weight (T1CE) image, a T2-weighted image, a Fluid Attenuated Inversion Recovery (FLAIR) image, an ADC (Apparent Diffusion Coefficient) multimodal MR images.

The plurality of images may be obtained from a magnetic resonance imaging (MRI) device.

The predicting step may include inputting the plurality of pre-processed images to the previously learned recognizer, and predicting the recurrence site by the previously learned recognizer.

The method may further include learning the learned recognizer so that the difference between the predicted recurrence site and the actual recurrence site is minimized.

Performing the preprocessing may include matching the T1CE image, the T2 image, the FLAIR image, and the ADC image based on the T1 image.

The step of performing the preprocessing may include performing brain skull stripping on the plurality of images.

The apparatus for predicting a recurrence site according to an embodiment includes a preprocessor for pre-processing a plurality of images, and a controller for predicting a recurrence site based on a plurality of preprocessed images, wherein the plurality of images are T1- Modal (including at least one of an image, a Contrast-enhanced T1-weight (T1CE) image, a T2-weighted image, a Fluid Attenuated Inversion Recovery (FLAIR) image, an ADC (Apparent Diffusion Coefficient) multimodal MR images.

The plurality of images may be obtained from a magnetic resonance imaging (MRI) device.

The apparatus further includes a learned recognizer that receives the plurality of preprocessed images and predicts the recurrence site, and the controller can input the plurality of preprocessed images to the learned recognizer.

The controller can learn the learned recognizer so that the difference between the predicted recurrence site and the actual recurrence site is minimized.

The preprocessor may match the T1CE image, the T2 image, the FLAIR image, and the ADC image based on the T1 image.

The preprocessor may perform brain skull stripping on the plurality of images.

A method of generating a PET image according to an exemplary embodiment includes performing a pre-processing on a T1 (T1-weighted) MR image, inputting a preprocessed T1 MR image to a pre-learned recognizer, (Positron Emission Tomography) image.

The T1 MR image may be acquired from a magnetic resonance imaging (MRI) device.

The performing may comprise performing brain skull stripping on the T1 MR image.

The method may further include learning the learned recognizer so that the difference between the generated PET image and the actual PET image is minimized.

The step of learning may include matching the generated PET image with the actual PET image.

The apparatus includes a preprocessor for pre-processing a T1 MR image, a controller for inputting the preprocessed T1 MR image to a pre-learned recognizer, and a PET image generator for generating a PET image based on the preprocessed T1 MR image And the learned recognizer.

The T1 MR image may be acquired from a magnetic resonance imaging (MRI) device.

The preprocessor may perform brain skull stripping on the T1 MR image.

The controller can learn the learned recognizer so that the difference between the generated PET image and the actual PET image is minimized.

The controller may match the generated PET image with the actual PET image.

FIG. 1 shows an example of a block diagram of a recurrence site learning apparatus according to an embodiment.
FIG. 2 is a view for explaining an example of the operation of the preprocessor shown in FIG. 1. FIG.
FIG. 3 is a diagram for explaining another example of the operation of the preprocessor shown in FIG. 1. FIG.
4A is a diagram for explaining learning for predicting a recurrence site according to an embodiment.
Fig. 4B shows an example of the structure of the recognizer shown in Fig. 4A.
5 shows an example of a block diagram of a PET image generation learning apparatus according to an embodiment.
6A is a diagram for explaining learning for generating a PET image according to an embodiment.
6B shows an example of the structure of the recognizer shown in FIG. 6A.

It is to be understood that the specific structural or functional descriptions of embodiments of the present invention disclosed herein are presented for the purpose of describing embodiments only in accordance with the concepts of the present invention, May be embodied in various forms and are not limited to the embodiments described herein.

Embodiments in accordance with the concepts of the present invention are capable of various modifications and may take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. However, it is not intended to limit the embodiments according to the concepts of the present invention to the specific disclosure forms, but includes changes, equivalents, or alternatives falling within the spirit and scope of the present invention.

The terms first, second, or the like may be used to describe various elements, but the elements should not be limited by the terms. The terms may be named for the purpose of distinguishing one element from another, for example without departing from the scope of the right according to the concept of the present invention, the first element being referred to as the second element, Similarly, the second component may also be referred to as the first component.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Expressions that describe the relationship between components, for example, "between" and "immediately" or "directly adjacent to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms " comprises ", or " having ", and the like, are used to specify one or more of the features, numbers, steps, operations, elements, But do not preclude the presence or addition of steps, operations, elements, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. Like reference symbols in the drawings denote like elements.

FIG. 1 is a block diagram of a recurrence site learning apparatus according to an embodiment of the present invention. FIG. 2 is a view for explaining an example of the operation of the preprocessor shown in FIG. 1. FIG. Fig. 8 is a diagram for explaining another example of the operation of the preprocessor.

Referring to FIGS. 1 to 3, the recurrence site learning apparatus 100 includes a preprocessor 110 and a controller 130.

The recurrence site learning apparatus 100 may be implemented (or mounted) in a personal computer (PC), a data server, or a portable electronic device for reviewing input data (or input data) and output data (or output data).

The portable electronic device may be a laptop computer, a mobile phone, a smart phone, a tablet PC, a mobile internet device (MID), a personal digital assistant (PDA), an enterprise digital assistant A digital still camera, a digital video camera, a portable multimedia player (PMP), a personal navigation device or a portable navigation device (PND), a portable game console (handheld console), an e-book e-book, or a smart device. For example, a smart device can be implemented as a smart watch or a smart band.

The recurrence site learning apparatus 100 can learn a recognizer for predicting a recurrence site from a plurality of images. The plurality of images may be multimodal MR images and may be training data including MR images before brain tumor recurrence and MR images after brain tumor recurrence. For example, the recurrence site learning apparatus 100 may receive training data including a multi-modal MR image and a recurrence site image, and may learn a recognizer that predicts a recurrence site using the received training data.

The multi-modal MR image may be a brain (e.g., a human brain) partial image acquired from a magnetic resonance imaging (MRI) device. MR images prior to the recurrence of brain tumors include T1-weighted images, Contrast-enhanced T1-weighted images, T2-weighted images, Fluid Attenuated Inversion Recovery (FLAIR) images, and ADC (Apparent Diffusion Coefficient) An image (or a map). The MR image after recurrence of a brain tumor may include at least one of T1 image and T1CE image. The characteristics of the multi-modal MR image can be as shown in Table 1.

The preprocessor 110 may perform preprocessing on a plurality of images.

When processing an MR image before the recurrence of a brain tumor, the preprocessor 110 can process the MR image before the recurrence of a brain tumor with a size of [240 x 240 x 240] and a zero image. The preprocessor 110 may scale the longest axis of the MR image before brain tumor recurrence to 240 size, scaling the entire image proportionally, and placing the T1 image in the center of the zero image. Also, the preprocessor 110 may invoke the mask image generated in the T1 image, normalize the data of the T1 image with a corresponding value in the foreground. The preprocessor 110 may equally apply this operation to the T1CE image, the T2 image, the FLAIR image, and the ADC image (or map) in addition to the T1 image. The preprocessor 110 includes six channels (6 channels) of a complete T1 image, T1CE image, T2 image, FLAIR image, ADC image (or map) Data can be generated.

In the case of MR images after the recurrence of brain tumor, the user can manually generate the T1CE_segmentation image by manually using the manual segmentation in the MR image after recurrence of the brain tumor. The T1CE_segmentation image may have a size of [ x ₇ x y ₇ x z ₇ ] and a data type of bool type. The preprocessor 110 may process the T1CE_segmentation image as a zero image of size [240 x 240 x 240]. The preprocessor 110 may scale the longest axis of the T1CE_segmentation image to 240, reduce the entire image proportionally, and position the T1CE_segmentation image in the center of the zero image. That is, the preprocessor 110 may generate [1 x 240 x 240] input data regarding the brain tumor recurrence site.

Hereinafter, the operation of the preprocessor 110 for realizing this will be described in detail.

The preprocessor 110 may perform operations such as co-registration of a plurality of images with respect to a T1 image, ascertaining the dimensions (dimensions or sizes) of a plurality of images, an operation of generating a slice image , Extracting the slice in the axial direction from the 3D image, and finally organizing the slice image for input.

In the case where the preprocessor 110 matches a plurality of images based on the T1 image, there may be a case of matching the MR image before the recurrence of the brain tumor and a case of matching the MR image after the recurrence of the brain tumor.

When the preprocessor 110 matches the MR image before brain tumor recurrence, the preprocessor 110 generates an MR image (T1 image, T1CE image, T2 image, FLAIR image, and ADC image (or map) Can be matched as [ x ₁ x y ₁ x z ₁ ] and the data type can be float type.

When the preprocessor 110 matches the MR image after the brain tumor recurrence, the preprocessor 110 resizes the MR images (T1 image, T1CE image, and T1CE_segmentation image) after brain tumor recurrence to [ x ₆ x y ₆ x z ₆ ]. The preprocessor 110 can match the data type of the T1 image and the T1CE image to the float type and the data type of the T1CE_segmentation image to the bool type.

The preprocessor 110 may generate the transformation matrix M _{t2 -> t 1} by matching the MR image after the recurrence of brain tumor with the MR image before the recurrence of brain tumor. At this time, the time point before the recurrence of brain tumors is t1, and the time point after recurrence of brain tumors is t2. The preprocessor 110 may apply the transformation matrix M _{t2 -> t 1} to the T1CE_segmentation image to match. At this time, the matched T1CE_segmentation image has a size of [ x ₁ x y ₁ x z ₁ ] and the data type can be matched to the bool type.

The preprocessor 110 can check that the dimension (or size) of all the brain image MR images is [ x ₁ x y ₁ x z ₁ ] and arrange all the images to be located in the same space as the T1 image . The result of the preprocessor 110 matching a plurality of images may be as shown in FIG. The preprocessor 110 may match the plurality of images 210 to produce a matched image 230. The preprocessor 110 can match the T1CE image, T2 image, FLAIR image, and ADC image (or map) before brain tumor recurrence to be located in the same space as the T1 image before brain tumor recurrence. The preprocessor 110 may match the T1CE image after the recurrence of the brain tumor to the same space as the T1 image after the recurrence of the brain tumor and then match the same to the T1 image before the recurrence of the brain tumor.

When the preprocessor 110 generates a slice image for a 3D image, the preprocessor 110 can perform brain skull stripping using a brain extraction tool, and based on the 3D image, a 240 x 240 x 240 ] volume It is possible to generate a skull stripping image by placing it at the center of the image and performing brain skull stripping. At this time, the generated skull stripping image is a brain mask image, the size is [240 x 240 x 240], and the data type is bool type. The brain mask generated by the preprocessor 110 may be as shown in FIG. The preprocessor 110 may generate a brain mask image 250 using brain skull stripping on the image of the matched image 230.

The preprocessor 110 may extract slices in the axial, sagital, and coronal directions in the 3D image. For example, the preprocessor 110 may extract a skull stripping image using a brain mask image. The preprocessor 110 maps a pixel value to -1 for a portion having a pixel value of 0 (a portion other than the brain) in the brain mask image, and for each portion (brain portion) having a pixel value of 1, Can be normalized. That is, the preprocessor 110 may divide the brain mask image into standard deviations by subtracting the mean of pixels having a value of 1 from the brain mask image. Preprocessor 110 may not proceed with extraction if the pixel values in the brain mask slice image of the slice image to be extracted are all zeros.

The characteristics of the slice images extracted in the axial direction, sagital direction, and coronal direction of the preprocessor 110 may be as shown in Table 2.

The preprocessor 110 can generate a segmentation image using a recognizer that has been learned for segmentation of MR images (T1 image, T1CE image, T2 image, and FLAIR image) before brain tumor recurrence. At this time, the segmentation image may have a size of [240 x 240 x 1 x k] and a data type of int type. The generated image is also used as input to the recognizer.

When the preprocessor 110 finally arranges the slice image for input to the input node of the recognizer, the modality of each slice image may be configured in the form of a channel. That is, the preprocessor 110 may set the slice image size to [240 x 240 x 6] x k and set the data type to the float type.

The preprocessor 110 sets the size of the image for calculating the loss associated with the back propagation technique of the recurrence site prediction at the output node of the recognizer as [240 x 240 x 1] xk, The data type can be set to bool type.

The preprocessor 110 may finally organize the slice images for all patients. For example, the preprocessor 110 can set the size of the slice image input to the input node of the recognizer as [240 x 240 x 6] x k x patient numbers and set the data type to float type. The preprocessor 110 may set the size of the image for loss calculation related to the back propagation technique of the recurrence site prediction at the output node of the recognizer to [240 x 240 x 1] x k x patient number and set the data type to bool type.

The preprocessor 110 may be implemented as a module. The module may denote hardware capable of performing the functions and operations of the preprocessor 110, may refer to program codes capable of performing specific functions and operations, For example, a processor or a microprocessor having computer program code embodied therein. In other words, a module may refer to a functional and / or structural combination of software for operating hardware and / or hardware to perform the operation of the preprocessor 110.

The controller 130 can predict the recurrence site based on the plurality of preprocessed images. For example, the controller 130 may generate a recurrence site image based on a plurality of preprocessed images. The controller 130 may analyze the plurality of images and the recurrence site images to obtain a correlation between the plurality of images and the recurrence site images.

The controller 130 may learn the recognizer to predict the recurrence site (or generate a recurrence site image) based on the correlation between the plurality of images and the recurrence site image. The learning method of the recognizer will be described in detail with reference to Figs. 4A and 4B.

FIG. 4A is a diagram for explaining learning for predicting a recurrence site according to an embodiment, and FIG. 4B illustrates an example of a structure of the recognizer shown in FIG. 4A.

Referring to FIG. 4A, the controller 130 may input the correlation between the plurality of images and the recurrence site images to the recognizer 150-1.

The controller 130 shown in FIG. 4A may have substantially the same configuration and operation as the controller 130 shown in FIG.

The recognizer 150-1 can predict a recurrence site (or generate a recurrence site image) based on a plurality of input values. The recognizer 150-1 may be implemented as a Convolutional Neural Network (CNN). The CNN may have a multiple input-single output structure that produces a single output image based on a plurality of input values. For example, the recognizer 150-1 may include a convolution layer, a pooling layer, an un-pooling player, a fully connected layer, or various combinations thereof To generate an output value. That is, the recognizer 150-1 may be configured as an artificial network.

The artificial neural network may include an input layer, a hidden layer, and an output layer. Each layer includes a plurality of nodes, and nodes between adjacent layers can be connected to each other with connection weights. Each node may operate based on an activation model. The output value corresponding to the input value can be determined according to the activation model. The output value of an arbitrary node can be input to the node of the next layer connected to that node. A node of the next layer can receive values output from a plurality of nodes. In the process of inputting the output value of an arbitrary node to the node of the next layer, a connection weight can be applied. The node of the next layer can output the output value corresponding to the input value to the node of the next layer connected to the node based on the activation model.

The output layer may include nodes corresponding to the depth of the distant image object. The nodes of the output layer can output characteristic values corresponding to the depth of the distant image object.

The recognizer 150-1 can predict the recurrence site (or generate the recurrence site image) based on the feature values output from the artificial neural network. Since the controller 130 knows the actual recurrence site, it can calculate the difference between the recurrence site image output (generated) through the recognizer 150-1 and the actual recurrence site.

Accordingly, the controller 130 can update the recognizer 150-1 so that the difference is reduced using a back propagation technique. For example, the controller 130 may propagate the difference from the output layer of the artificial neural network in the recognizer 150-1 to the input layer via the hidden layer in the reverse direction. In the process of propagating the difference in the reverse direction, the connection weights between nodes may be updated such that the difference is reduced. In this manner, the controller 130 can learn the recognizer 150-1 in consideration of the difference. The above-described learning operation can be repeatedly performed until the difference becomes less than a predetermined threshold value.

In order to update the artificial neural network that generates the recurrent site image, the controller 130 may use the parameters as shown in Table 3 in the back propagation method.

At this time, the learning rate can be reduced by 5% every time the Epoch increases by 10. The learning rate is 0.0015 * 0.95 when Epoch is 1 ~ 10, the learning rate is 0.0015 * 0.95 when Epoch is 11 ~ 20, the learning rate is 0.0015 * 0.95 ² when Epoch is 21 ~ 30, At 200, the learning rate may be 0.0015 * 0.95 ¹⁹ .

The controller 130 can learn the recognizer 150-1 so that the difference between the recurrence site image output (generated) through the recognizer 150-1 and the actual recurrence site image is minimized.

Referring to FIG. 4B, the recognizer 150-1 may receive a plurality of slice images. For example, a plurality of slice images may have a size of [240 x 240], and may include a T1 image, a T1CE image, a T2 image, a FLAIR image, an ADC image, and a segmentation image composed of 6 channels (6 channels). The recognizer 150-1 can estimate a recurrence site (or generate a recurrence site image) based on a plurality of slice images.

The numerical value on the left side of the plurality of slice images means the size of [ x x y ], and the numerical value on the upper side can mean the number of channels. The recognizer 150-1 can perform contraction of the image as it goes down to the next layer and output abstract meaningful core information to the region of interest for what. The recognizer 150-1 can expand the image as it goes to the upper level (layer), and can output a high-resolution image precisely where and what.

Different operations within the recognizer 150-1 are represented by different kinds of arrows. For example, the recognizer 150-1 may perform convolution based on [3 x 3], copy, max pooling based on [2 x 2], and [2 x 2] And up-convolution based on the received signal.

5 shows an example of a block diagram of a PET image generation learning apparatus according to an embodiment.

Referring to FIG. 5, the PET image generation learning apparatus 300 includes a preprocessor 310 and a controller 330.

The preprocessor 330 and the controller 330 shown in FIG. 5 may have substantially the same configuration as the preprocessor 110 and the controller 130 shown in FIG.

The PET image generation learning apparatus 300 can learn a recognizer for PET image generation from a T1 MR image. The PET image generation learning apparatus 300 may generate different types of PET images according to the tracer used. The PET image may be a type of any of the types of PET images produced by the PET image generation learning device 300. For example, the PET image may be a tau-PET image, an AV45-PET image, or the like. The T1 MR image may be training data to train the recognizer. The T1 MR image may be a brain (e.g., a human brain) partial image acquired from a magnetic resonance imaging (MRI) device.

The training data input to the recognizer may be a T1 MR image and a PET image. The characteristics of the training data can be as shown in Table 5.

The preprocessor 310 may perform preprocessing on the T1 MR image and the PET image.

The preprocessor 310 includes operations for co-registration of the PET image with respect to the T1 image, operation for ascertaining the dimension (or size) of the T1 MR image and the PET image, generating a slice image for the 3D image , Extracting slices in an axial direction from the 3D image, and finally organizing the slice images for input.

If the preprocessor 310 matches the PET image with respect to the T1 MR image, then the PET image can be matched to the size of the T1 MR image [ x ₁ x y ₁ x z ₁ ]. The global SUVR value ( gSUVR ) can be extracted from the matched PET image and the T1 MR image, and an average SUVR value ( lSUVR _i ) can be extracted for each region. Here, i is an arbitrary number from 1 to N. The region to be extracted may also be a defined region (Desikan-Killiany Atlas, Destrieux Atlas, Automated Anatomical Labeling) and refers to an area defined arbitrarily by the person performing this work.

The preprocessor 310 may map a value to a pixel corresponding to each region using a global SUVR value ( gSUVR ) and an average SUVR value ( lSUVR _i ). For example, the preprocessor 310 may map true (1) or false (0) values to pixels corresponding to each region through gSUVR > lSUVR _i . Accordingly, the preprocessor 310 can generate a PET image whose data type is bool type.

Preprocessor 310 may determine whether the PET image dimension T1 MR image (or size) is [x ₁ x ₁ x y z _1]. If the dimension does not fit, the preprocessor 310 can perform a dimension matching operation.

When the preprocessor 310 generates a slice image for a 3D image, a skull stripping image is generated by performing brain skull stripping, and a 3D image is formed on the basis of this image in a [240 x 240 x 240] (center). At this time, the preprocessor 310 can perform brain skull stripping using a brain extraction tool. The generated skull stripping image is a brain mask image, the size is [240 x 240 x 240], and the data type is bool type.

The preprocessor 310 may extract the slice in the axial direction from the 3D image. For example, the preprocessor 310 may extract a skull stripping image using a brain mask image. The preprocessor 310 maps a pixel value to -1 for a portion having a pixel value of 0 (a portion other than a brain) in the brain mask image, and for each portion (brain portion) having a pixel value of 1, Can be normalized. That is, the preprocessor 310 may divide the brain mask image into standard deviations by subtracting the mean of pixels having a value of 1 from the brain mask image. The preprocessor 310 may not proceed with extraction if the pixel values in the brain mask slice image of the slice image to be extracted are all zeros. The slice extracted by the preprocessor 310 has a size of [240 x 240 x 1 xk] for a T1 MR image, a size of [240 x 240 x 1 xk] for a PET image, a data type of a float, The type may be a bool type.

When the preprocessor 310 finally arranges the slice image for input to the input node of the recognizer, the modality of each slice image may be configured in the form of a channel. That is, the preprocessor 310 can set the size of the slice image to [240 x 240 x 1 x k] and set the data type to the float type.

The preprocessor 310 sets the size of the image for calculating the loss associated with the back propagation technique of PET image generation at the output node of the recognizer to [240 x 240 x 1 xk] The data type can be set to bool type.

The preprocessor 310 may finally organize the slice images for all patients. For example, the preprocessor 310 may set the size of the slice image input to the input node of the recognizer as [240 x 240 x 1] x k slices x the number of patients and set the data type to the float type. The preprocessor 310 sets the size of the image for loss calculation related to the back propagation technique of PET image generation at the output node of the recognizer as [240 x 240 x 1] xk slice x patient number, and sets the data type to bool type have.

The preprocessor 310 may be implemented as a module. The module may denote hardware capable of performing the functions and operations of the preprocessor 310, may refer to program codes capable of performing specific functions and operations, For example, a processor or a microprocessor having computer program code embodied therein. In other words, a module may refer to a functional and / or structural combination of hardware and / or software to drive the operations of the preprocessor 310. [

The controller 330 may generate a PET image based on the preprocessed T1 MR image. The controller 330 may analyze the T1 MR image and the PET image to obtain a correlation between the T1 MR image and the PET image. The controller 330 may learn the recognizer to produce a PET image based on the correlation between the T1 MR image and the PET image. The learning method of the recognizer will be described in detail with reference to Figs. 6A and 6B.

FIG. 6A is a diagram for explaining learning for generating a PET image according to an embodiment, and FIG. 6B shows an example of the structure of the recognizer shown in FIG. 6A.

Referring to FIG. 6A, the controller 330 may input the correlation between the T1 MR image and the PET image to the recognizer 150-2.

The controller 330 shown in FIG. 6A is substantially the same in configuration and operation as the controller 330 shown in FIG. 5, and the recognizer 150-2 includes the recognizer 150-1 shown in FIG. May be substantially the same.

The recognizer 150-2 may generate a PET image based on the T1 MR image. The recognizer 150-2 may be implemented as a CNN. CNN may have a multiple input-single output structure that generates a single output value based on a plurality of input values. CNN may be based on a myriad of T1 MR images and PET images. The recognizer 150-2 may output the output based on a convolution layer, a pooling layer, an un-pooling player, a full connected layer, or various combinations thereof. Value can be generated. That is, the recognizer 150-2 may be configured as an artificial network.

The recognizer 150-2 can generate the PET image based on the feature values output from the artificial neural network. Since the controller 330 knows the actual PET image, it can calculate the difference between the PET image output (generated) and the actual PET image through the recognizer 150-2.

Accordingly, the controller 330 may update the recognizer 150-2 so that the difference is reduced using a back propagation technique. For example, the controller 330 can propagate the difference from the output layer of the artificial neural network in the recognizer 150-2 to the input layer via the hidden layer in the reverse direction. In the process of propagating the difference in the reverse direction, the connection weights between nodes may be updated such that the difference is reduced. In this way, the controller 330 can learn the recognizer 150-2 in consideration of the difference. The above-described learning operation can be repeatedly performed until the difference becomes less than a predetermined threshold value.

In order to update the artificial neural network that produces the PET image, the controller 330 may use parameters as shown in Table 4 for the back propagation technique.

The controller 330 may learn the recognizer 150-2 so that the difference between the PET image output (generated) and the actual PET image through the recognizer 150-2 is minimized.

Referring to FIG. 6B, the recognizer 150-2 may receive a T1 MR image. The T1 MR image is [128 x 128] in size and can consist of one channel. The recognizer 150-2 may generate a PET image based on the T1 MR image.

The numerical value on the left side of the plurality of slice images means the size of [ x x y ], and the numerical value on the upper side can mean the number of channels. As the recognizer 150-2 proceeds to the lower level (layer), the contraction of the image occurs and the abstracted meaningful core information of the region of interest with respect to what can be output. The recognizer 150-2 can expand the image as it goes to the upper level (layer), and can output a high-resolution image precisely where and what.

Different operations within the recognizer 150-2 are represented by different kinds of arrows. For example, the recognizer 150-2 may perform convolution based on [3 x 3], copy, max pooling based on [2 x 2], and [2 x 2] And up-convolution based on the received signal.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA) , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing apparatus may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (22)

Performing preprocessing on a plurality of images; And
Predicting a recurrence site based on a plurality of preprocessed images
Lt; / RTI >
The plurality of images may include a T1-weighted image, a Contrast-enhanced T1-weighted image, a T2-weighted image, a Fluid Attenuated Inversion Recovery (FLAIR) image, an ADC (Apparent Diffusion Coefficient) Segmentation image that is a multimodal MR image comprising at least one of a segmentation image and a segmentation image.
The method according to claim 1,
Wherein the plurality of images are obtained from a magnetic resonance imaging (MRI) apparatus.
The method according to claim 1,
Wherein the predicting comprises:
Inputting the preprocessed plurality of images to the learned recognizer; And
Wherein the recognizer recognizes the recurrence site
Of the recurrence site.
The method of claim 3,
Learning the learned recognizer so that the difference between the predicted recurrence site and the actual recurrence site is minimized
Wherein the recurrence site predicting method further comprises:
The method according to claim 1,
The step of performing the pre-
Matching the T1CE image, the T2 image, the FLAIR image, and the ADC image based on the T1 image
Of the recurrence site.
The method according to claim 1,
The step of performing the pre-
Performing brain skull stripping on the plurality of images
Of the recurrence site.
A preprocessor for performing preprocessing on a plurality of images; And
A controller for predicting a recurrence site based on a plurality of preprocessed images
Lt; / RTI >
The plurality of images may include a T1-weighted image, a Contrast-enhanced T1-weighted image, a T2-weighted image, a Fluid Attenuated Inversion Recovery (FLAIR) image, an ADC (Apparent Diffusion Coefficient) Segmentation image, wherein the MR image is a multimodal MR image comprising at least one of the segmentation images.
8. The method of claim 7,
Wherein the plurality of images are obtained from a magnetic resonance imaging (MRI) apparatus.
8. The method of claim 7,
A plurality of pre-processed images are received,
Further comprising:
The controller comprising:
And inputs the pre-processed plurality of images to the pre-learned recognizer.
10. The method of claim 9,
The controller comprising:
Learning the learned recognizer so that the difference between the predicted recurrence site and the actual recurrence site is minimized
Recurrence site prediction device.
8. The method of claim 7,
The pre-
The T1CE image, the T2 image, the FLAIR image, and the ADC image based on the T1 image
Recurrence site prediction device.
8. The method of claim 7,
The pre-
Performing brain skull stripping on the plurality of images
Recurrence site prediction device.
Performing a pre-processing on a T1 (T1-weighted) MR image;
Inputting a preprocessed T1 MR image to a pre-learned recognizer; And
The step of generating the positron emission tomography (PET) image by the learned recognizer
/ RTI >
14. The method of claim 13,
Wherein the T1 MR image is obtained from a magnetic resonance imaging (MRI) device.
14. The method of claim 13,
Wherein the performing comprises:
Performing brain skull stripping on the T1 MR image
/ RTI >
14. The method of claim 13,
Learning the learned recognizer so that the difference between the generated PET image and the actual PET image is minimized
Further comprising the steps of:
17. The method of claim 16,
Wherein the learning step comprises:
Matching the generated PET image with the actual PET image
/ RTI >
A preprocessor for pre-processing the T1 MR image;
A controller for inputting the preprocessed T1 MR image to a pre-learned recognizer; And
A processor for generating a PET image based on the preprocessed T1 MR image,
And a PET image generating device.
19. The method of claim 18,
Wherein the T1 MR image is obtained from a magnetic resonance imaging (MRI) device.
19. The method of claim 18,
The pre-
And performing brain skull stripping on the T1 MR image
PET imaging device.
19. The method of claim 18,
The controller comprising:
The learned recognizer is learned so that the difference between the generated PET image and the actual PET image is minimized
PET imaging device.
22. The method of claim 21,
The controller comprising:
And comparing the generated PET image with the actual PET image
PET imaging device.

Images