WO2021210797A1

WO2021210797A1 - Artificial intelligence-based cloud platform system for reading medical images

Info

Publication number: WO2021210797A1
Application number: PCT/KR2021/003480
Authority: WO
Inventors: 최우식; 김태규; 이언철; 정원우; 오지영
Original assignee: 주식회사 딥노이드
Priority date: 2020-04-16
Filing date: 2021-03-22
Publication date: 2021-10-21
Also published as: KR102150647B1

Abstract

The present invention relates to a cloud platform system for reading medical images, the cloud platform system comprising: a plurality of image processing modules which are pre-programmed and modularized to perform preprocessing of medical images; a plurality of artificial intelligence modules in which an artificial intelligence algorithm is pre-programmed and modularized; a training model design unit which provides a graphical user interface for designing an artificial intelligence-based training model to a user terminal connected through a web browser; and a reading model generation unit which generates a reading model by training the training model designed by the training model design unit, wherein the training model design unit can generate the training model by displaying, on the graphical user interface, a training model design window and a module list display window on which a list of the plurality of image processing modules and a list of the plurality of artificial intelligence modules are displayed, generating module icons in the training model design window in response to dragging and dropping the lists displayed on the module list display window into the training model design window, and adopting line connections between the module icons as data flows when the module icons are connected by lines.

Description

AI-based cloud platform system for medical image reading

The present invention relates to an AI-based cloud platform system for reading medical images, and more particularly, to an AI-based AI-based system for reading medical images that supports design and creation of AI-based reading models by accessing multiple users. of the cloud platform system.

Various algorithms using artificial intelligence (AI) techniques have been developed for a long time. In particular, recently, various techniques for processing big data by applying deep learning algorithms are being developed, and successful cases of applying them are gradually increasing. is increasing

In the meantime, attempts have been made to apply artificial intelligence to medical images to assist in clinical decision-making. Tomography) and MRA (Magnetic Resonance Angiography) have been developed to apply artificial intelligence algorithms to medical images acquired from diagnostic devices to help clinicians make decisions.

It is known that the detection rate of lesions is improved in the case of an auxiliary diagnostic system that uses artificial intelligence to classify whether the tissue shown on the medical image is normal or abnormal, and whether the tumor is benign or negative, compared to the case where only a radiologist reads it. For such classification, naive bayes, Support Vector Machine (SVM), Artificial Neural Network (ANN), Hidden Markov Model (HMM), etc. are mainly used, and these are algorithms that automatically classify the presence or absence of lesions.

Machine learning algorithms can be used as artificial intelligence algorithms, and machine learning can be largely divided into supervised learning and unsupervised learning. With such a machine learning algorithm, it is possible to create a reading model (or predictive model), and use the generated reading model to construct a system that infers whether a medical image is normal or abnormal. Research for the creation of the model continues.

In the case of artificial intelligence algorithms, as described above, they are largely divided into supervised learning and unsupervised learning. In the case of supervised learning, classification, decision tree, KNN (K-nearest neighbor), and neural network ( Neural network), support vector machine (SVM), etc., and in the case of unsupervised learning, there is also clustering. In addition, semi-supervised learning and reinforcement learning are also known as artificial intelligence algorithms.

In the case of artificial intelligence algorithms used to read medical images such as lesion or disease diagnosis, algorithms such as classification algorithm, object detection, and segmentation are mainly used. In Edo, ResNet, DenseNet, MobileNet, etc. have been developed in various ways.

Due to the existence of so many different algorithms, it is not an easy task to design a learning model for reading medical images, unless you are an expert in the field of artificial intelligence, for example, an expert who can design a learning model by directly coding an algorithm. am.

In addition, in the case of medical images, it is not easy to obtain them because they are personal information, and it is also difficult to secure enough medical images to increase the reading accuracy of the learning model.

Accordingly, the present invention has been devised to solve the above problems, and while providing various types of medical images, and modularly providing image processing algorithms and artificial intelligence algorithms for pre- or post-processing, users can easily use artificial intelligence The purpose is to provide an AI-based cloud platform system for medical image reading that can provide an environment for designing a learning model based on it.

According to the present invention, in the cloud platform system for reading a medical image, a plurality of pre-programmed and modular image processing modules to perform pre-processing of a medical image, and a plurality of artificial intelligence algorithms pre-programmed and modularized An intelligence module, a learning model design unit that provides a graphical user interface for designing an artificial intelligence-based learning model to a user terminal accessed through a web browser, and a learning model designed by the learning model design unit to learn the learning model to generate a reading model and a reading model generating unit; The learning model design unit displays a module list display window in which a list of the plurality of image processing modules and a list of the plurality of artificial intelligence modules is displayed, and a learning model design window in the graphic user interface, and the list displayed in the module list display window is When dragged and dropped to the learning model design window, a module icon is generated in response to drag and drop in the learning model design window. This is achieved by a cloud platform system for reading medical images, characterized in that generating the learning model.

Here, the apparatus further includes a dataset storage unit storing a plurality of datasets classified according to at least one of a body part, a type of modality, a type of a disease to be diagnosed, and a type of an image dimension; The learning model design unit displays a list of the plurality of datasets on the module list display window, and when the dataset displayed on the module list display window is dragged and dropped to the learning model design window, data corresponding to the dataset displaying an icon on the artificial intelligence design window, and when at least one of the module icons and the data icon are line-connected, the learning model is generated by using the line connection with the module icon as a data flow; The read model generator may generate the read model by learning a dataset corresponding to the data icon on the learning model generated by the learning model design unit.

And, it further includes a model storage unit in which the pre-generated reading model and the pre-designed learning model are stored; The learning model design unit displays the read model or the list of learning models stored in the model storage unit on a graphical user interface, and any one of the read model or the list of learning models displayed on the graphical user interface is selected In this case, the design structure of the corresponding model may be displayed on the artificial intelligence design window, and a function of modifying the design structure displayed on the artificial intelligence design window may be provided.

And, the layer applied to the configuration of the artificial intelligence algorithm further includes a plurality of layer modules modularized for each function; The learning model design unit displays a layer list display window displaying a list of the plurality of layer modules and an artificial intelligence design window on the graphic user interface, and the list displayed on the layer list display window is dragged and dropped to the artificial intelligence design window. In this case, the layer icon of the corresponding layer module is displayed on the AI design window, and when the layer icons displayed on the AI design window are line-connected, the line connection between the layer icons is used as a data flow, and the AI module can create

And, doedoe composed of at least two or more layers, further comprising a plurality of layer blocks modularized for generating the artificial intelligence module; The learning model design unit displays a list of the plurality of layer blocks in the layer list display window, and drags and drops the list of layer modules displayed in the layer list display window to the AI design window, and connects the layer through a line. It can support the creation of blocks.

In addition, when the artificial intelligence module and the layer block generated through the artificial intelligence design window are set to be stored or shared, the learning model design unit may update the layer list display window.

According to the above configuration, according to the present invention, various types of medical images are provided, and an image processing algorithm and an artificial intelligence algorithm for pre-processing or post-processing are modularized and provided, while the user can easily develop an artificial intelligence-based learning model. An artificial intelligence-based cloud platform system for reading medical images that can provide an environment for designing is provided.

1 and 2 are diagrams for explaining the configuration of an artificial intelligence-based cloud platform system according to an embodiment of the present invention;

3 to 24 are diagrams for explaining a graphic user interface provided by an artificial intelligence-based cloud platform system according to an embodiment of the present invention.

The present invention relates to a cloud platform system for reading medical images, comprising: a plurality of image processing modules pre-programmed and modularized to perform pre-processing of medical images; A learning model design unit that provides a graphical user interface for designing an AI-based learning model to a user terminal accessed through a web browser, and a reading model generation that generates a reading model by learning the learning model designed by the learning model design unit includes wealth; The learning model design unit displays a module list display window in which a list of the plurality of image processing modules and a list of the plurality of artificial intelligence modules is displayed, and a learning model design window in the graphic user interface, and the list displayed in the module list display window is When dragged and dropped to the learning model design window, a module icon is generated in response to drag and drop in the learning model design window. to generate the learning model.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

1 is a diagram showing the utilization structure of an artificial intelligence-based cloud platform system 100 according to an embodiment of the present invention, and FIG. 2 is a configuration of an artificial intelligence-based cloud platform system 100 according to an embodiment of the present invention. It is a diagram showing an example of

1, in the AI-based cloud platform system 100 (hereinafter referred to as 'cloud platform system 100') according to an embodiment of the present invention, a plurality of user terminals 300 are connected to a communication network ( 500), and generates a reading model using a graphic user interface for designing a learning model and generating a reading model provided by the cloud platform system 100 according to an embodiment of the present invention.

As shown in FIG. 2 , the cloud platform system 100 according to an embodiment of the present invention includes a dataset storage unit 110 in which a plurality of datasets 111 are stored, a plurality of image processing modules 121 , and a plurality of of an artificial intelligence module 131 , a model learning unit 140 , and a medical image reading unit 150 . Here, the image processing module 121 is stored in the first module storage unit 120 , and the artificial intelligence module 131 is stored in the second module storage unit 130 , as an example, the first module storage unit 120 ) and the second module storage unit 130 do not mean physically separate storage.

The plurality of datasets 111 stored in the dataset storage unit 110 are classified according to at least one of a body part, a type of modality, a type of a disease to be read, and a type of an image dimension.

Body parts are body parts to be learned or read, Abdomen, Brain, Head, Neck, Spine, Breast, Chest, Gynecology. It may include a region, a urology region, a heart, a blood vessel, a musculoskeletal system, and the like.

The type of modality may include CT, MRI, MRA, X-Ray, ultrasound, PET, etc., and the type of disease to be read can be read using medical images such as tumor, scoliosis, pneumonia, and diabetic retinopathy. It may include various diseases and lesions. And, the type of the image dimension may include a 2D image and a 3D image.

A plurality of medical images classified according to the classification criteria as described above constitute one dataset 111 , and the user can perform learning using one dataset 111 .

The image processing module 121 is pre-programmed and modularized to perform pre-processing of a medical image. The image processing module 121 may be divided into a module for pre-processing a color image, a module for pre-processing a gray-scale image, a module for pre-processing a binary image, and other modules, which will be described in detail later.

The artificial intelligence module 131 is modularized with an artificial intelligence algorithm pre-programmed. In the present invention, it is assumed that the artificial intelligence module 131 constitutes the artificial intelligence module 131 by modularizing a neural network algorithm based on deep learning, and a detailed description thereof will be given later.

The model learning unit 140 includes a learning model design unit 141 that supports the design of the learning model, and a reading model generation unit 142 that generates a reading model by learning the learning model designed by the learning model design unit 141 . do.

In the present invention, for convenience of explanation, the description is divided into a 'learning model' and a 'reading model'. It means the model in the process, defines the state in the design process before actual learning is performed, and the read model is the model after learning the designed learning model using the dataset 111, the weight value Defines the updated state of parameters such as

The learning model design unit 141 provides a graphical user interface for designing an AI-based learning model to the user terminal 300 accessed through a web browser. And, as described above, the reading model generation unit 142 generates a reading model by learning the learning model designed by the learning model design unit 141 .

Here, the learning model design unit 141 in the design of the learning model, through the selection of the dataset 111, the image processing module 121, the artificial intelligence module 131, drag and drop, and the line connection between modules, the learning model It provides a graphical user interface that can design It becomes possible to create a reading model.

Hereinafter, with reference to FIGS. 3 to ?, using the graphic user interface provided by the cloud platform system 100 according to an embodiment of the present invention, a process of creating a new project for designing a learning model and generating a reading model explain about Here, the new project creation process is performed by the learning model design unit 141 .

First, when a user accesses a website provided by the cloud platform system 100 through the user terminal 300 , a main screen as shown in FIG. 3 is displayed on the screen of the user terminal 300 .

Then, after the user logs in through his/her ID and password (upper right in FIG. 3 ) and clicks the My Page menu, the My Page screen is displayed on the screen of the user terminal 300 .

4 is a diagram illustrating an example of a My Page screen provided by the cloud platform system 100 according to the present invention. Referring to FIG. 4 , the My Page screen according to an embodiment of the present invention may include a My Page menu window 41 , a project status window 42 , and a My Project list window 43 .

The My Page menu window 41 may include a project menu (Project) composed of My Projects and Shared Projects, a data set menu (Dataset), and a module menu (Modules), and a Q&A menu ( My Q&A) and My Profile menu (My Profile) may be added.

If you click My Project, a screen as shown in FIG. 4 is displayed. If you click Share Project, as shown in FIG. 5 , a list of projects set to be shared among projects created by other users is displayed on the screen. do.

Similarly, when the data set menu is clicked, the list of data sets currently held by the cloud platform system 100 is displayed on the right side of the My Page menu window 41 in the form shown in FIG. When clicked, detailed information may be checked. Here, the data set held by the cloud platform system 100 is displayed in the data set list on the modular screen, which will be described later.

The module menu may include an image processing item and a neural network item. When the image processing item is clicked, the list of image processing modules 121 currently held by the cloud platform system 100 may be displayed on the right side of the My Page menu window 41 in the form shown in FIG. 5 . have. Similarly, when the neural network item is clicked, the list of artificial intelligence modules 131 possessed by the cloud platform system 100 may be displayed on the right side of the My Page menu window 41 in the form shown in FIG. 5 . have. Here, the image processing module 121 and the artificial intelligence module 131 possessed by the cloud platform system 100 are displayed on the module list display window 91 on the modular screen to be described later, which will be described later.

Again, referring to FIG. 4 , in the project status window 42 on the My Project screen, the project status of the currently logged-in user, the resource status, the project-specific notice, and the entire project currently registered in the cloud platform system 100 are displayed. Information on the current status, information on the project execution history of the current user, and the like may be displayed.

In addition, a list of projects performed by the currently logged in user is displayed on the My Project list window 43 , and FIG. 4 shows an example in which one project is performed.

Here, when the '+Create' item is clicked on the my project list window 43, a property input window for inputting property information of a read model to be created as a new project is displayed on the graphic user interface.

First, when the '+Create' item is clicked to create a new project, a project basic information input pop-up window as shown in FIG. 6 is displayed on the screen of the user terminal 300 as a property input window.

In the pop-up window for entering basic project information, enter the project name, select the body part, select the modality, enter the project summary, enter the project due date, and enter the cover image (Cover). Image) items for selection may be displayed, where a body part and a modality type are included in attribute information of a read model to be created as a new project.

After entering the property information using the project basic information input pop-up window, when the Next button is clicked, as shown in FIG. 7, the model type selection pop-up window for selecting the type of the artificial intelligence model is displayed as the property input window. displayed on the graphical user interface screen.

In the present invention, examples of types of artificial intelligence models include, but not limited to, classification models, object detection models, and segmentation models as shown in FIG. 7 . is of course Here, in the model type selection pop-up window shown in FIG. 7 , it is assumed that one or two or more of the artificial intelligence models are selected.

After selecting the type of the artificial intelligence model, when the Next button is clicked, as shown in FIG. 8 , data type selection for selecting the type of the training target dataset 111 including 2D and 3D A pop-up window is displayed on the graphical user interface as a property input window. Here, it goes without saying that the two-dimensional or three-dimensional image dataset 111 can be selectively selected, or both.

When the type of the AI model and the 2D or 3D dataset 111 are selected through the above process, the type of the AI model and the type of the dataset 111 are registered as the above-described attribute information.

When the registration process of the attribute information is completed, the learning model design unit 141 displays a modular screen on the graphic user interface as shown in FIG. 9 .

The modular screen may include a module list display window 91 , a learning model design window 92 , and an information display window 93 .

The module list display window 91 displays a list of a plurality of datasets 111 , a list of a plurality of image processing modules 121 , and a list of a plurality of artificial intelligence models.

Here, in the present invention, when the learning model design unit 141 displays the list on the module list display window 91, as described above, the data set 111 that matches the song information input through the attribute input window, and the image processing module Only the list of 121 and the artificial intelligence module 131 can be displayed. Through this, in the case of a user lacking knowledge in the field of artificial intelligence, only modules that match the reading model they want to design and create and attribute information are displayed on the list, making it easier to access.

Fig. 10 (a) is a diagram showing an example of the module list display window 91 in which only a list matching the attribute information is displayed, and Fig. 10 (b) is an example of the module list display window 91 in which the entire list is displayed. It is a drawing. 10 shows an example of a list of datasets 111 .

Referring to FIG. 10 , the module list display window 91 according to the present invention may provide an entire list selection item ALL and a recommendation list selection item Recommend, and clicking the entire list selection item ALL Then, as shown in (b) of FIG. 10, the entire list of the dataset 111 is displayed, and when the recommendation list selection item (Recommend) is clicked, as shown in (a) of FIG. Only the list, that is, the list of the dataset 111 matching the previously registered attribute information, is displayed.

On the other hand, the module list display window 91 may provide a dataset selection item 91a, an image processing selection item 91b, and an artificial intelligence selection item 91c, and the learning model design unit 141 provides a dataset selection item ( When 91a) is clicked, a list of the dataset 111 can be displayed on the module list display window 91 (see FIG. 10 ).

Similarly, when the image processing selection item 91b is selected, the learning model design unit 141 displays a list of imaging processing modules on the module list display window 91. At this time, the same as in the dataset 111 . For example, through selection of the entire list selection item ALL and the recommendation list selection item Recommend, the entire list or a matching recommendation list may be displayed.

And, when the artificial intelligence selection item 91c is clicked, the learning model design unit 141 displays a list of the artificial intelligence module 131 on the module list display window 91, and identically, the entire list selection item (ALL) And through selection of the recommendation list selection item (Recommend), the entire list or a matching recommendation list may be displayed.

Meanwhile, as shown in FIG. 2 , the cloud platform system 100 according to an embodiment of the present invention may include a model storage unit 160 in which a pre-generated reading model and a pre-designed learning model are stored.

Here, the reading model or learning model stored in the model storage unit 160 is registered by the user using the cloud platform system 100 according to the present invention by sharing the learning model designed by the user or the reading model created by the user. This allows other users to access it.

Here, the learning model design unit 141 searches the model storage unit 160 for a reading model or learning model matching the attribute information input through the attribute input window, and a list of a preset number of matching learning models or reading models. A recommended model list window in which is displayed may be displayed on the graphical user interface. In the present invention, the recommended model list window is displayed in the form of a pop-up window before switching to the modeler screen after input of attribute information through the attribute input window. The model or read model is displayed on the modeler screen, and based on it, it is possible to design a new learning model through modification.

11 is a diagram illustrating an example in which a list of image processing modules 121 is displayed on the module list display window 91 in the cloud platform system 100 according to an embodiment of the present invention. In the present invention, as an example, the list of the image processing module 121 displayed on the module list display window 91 is divided into a plurality of groups for each pre-processing function and displayed as an example.

Here, the group for each preprocessing function may include a color preprocessing group (Color), a grayscale preprocessing group (Grayscale), and a binary preprocessing group (Binary). In addition, the group for each preprocessing function may include a general preprocessing group (General).

In the color preprocessing group, a list of image processing modules 121 operating on an 8-bit or more color image may be displayed.

The image processing module 121 belonging to the color preprocessing group may include a color mode converting module, a color to grayscale module, a gamma correction module for color, and a histogram smoothing module for color.

The color mode converting module is a module that converts between various color spaces, the color to grayscale module is a module that converts a color image to a grayscale image, and the gamma correction module for color is a module that performs gamma correction of a color image. The histogram smoothing module for color is a module for smoothing the histogram of a color image.

The grayscale preprocessing module may include a gamma correction module for grayscale, a histogram smoothing module for grayscale, a morphological transformation module, and a threshold transformation module.

The gamma correction module and the histogram smoothing module for grayscale perform gamma correction and histogram smoothing of grayscale images, respectively, and the Morphological transformation module is a module that performs morphological transformation of grayscale images. The conversion module is a module that performs binarization of a grayscale image to have only two values, black and white.

The binary preprocessing module may include a contour detection module, a convex hull module, a morphological transformation module for binary images, and a skeletonize module.

The contour detection module is a module that detects a contour by connecting pixels having the same pixel value, and the convex hull module is a module that generates the smallest mask including all the white pixel values in the image. The Skeletonize module is a module that makes only the skeleton of the image remain.

The general pre-processing group (General) includes a list other than the above-described image processing module 121 applied only to a color image, a grayscale image, or a binary image.

11, the general preprocessing group (General) is a bit reduction module (Bit Reduction), edge detection module (Edge Detection), inversion module (Invert), resampling module (Resample), rescaling module (Rescaling) ), a resizing module (Resize), a sharpening module (Sharpening), a smoothing module (Smooding), a zero-panning module (Zero-padding) module may be included.

The bit reduction module is a module that converts an image into an 8-bit integer form by reducing the bit value of a color used for image expression. module, the invert module (Invert) is a module that inverts the pixel values of the image, the resampling module (Resample) is a module that adjusts the size of the image using information on the distance between pixels, and the rescaling module (Rescaling) is It is a module that changes the range of pixel values located in the image sample area, for example, the profile or the entire image, the resizing module (Resize) is a module that enlarges or reduces the size of the image, and the sharpening module (Sharpening) is the edge of the image It is a module that gives an emphasizing effect, the smoothing module is a module that blurs the image by giving it a blur effect, and the zero-padding module is a module that attaches a square shape with zero values to the original image. .

As described above, in the design of the learning model, the image processing algorithms required for the pre-processing of the original medical image are modularized and grouped according to the pre-processing function to provide a list, so that users can more easily fit the learning model they want to design. image processing algorithms can be found.

12 is a diagram illustrating an example in which a list of artificial intelligence modules 131 is displayed on the module list display window 91 in the cloud platform system 100 according to an embodiment of the present invention. In the present invention, as an example, the list of artificial intelligence modules 131 displayed on the module list display window 91 is divided into groups for each AI function and displayed as an example.

Groups by AI function are: 2D classification group (Classification 2D), 3D classification group (Classification 3D), 2D detection group (Object Detection 2D), 3D detection group (Object Detection 3D), 2D segmentation group (Segmentation 2D) ), an example including a three-dimensional segmentation group (Segmentation 3D).

The two-dimensional classification group (Classification 2D) displays a list of artificial intelligence modules 131 that perform classification of the two-dimensional image, and the three-dimensional classification group (Classification 3D) performs the classification of the three-dimensional image. A list of artificial intelligence modules 131 to perform is displayed, and in the two-dimensional detection group (Object Detection 2D), a list of artificial intelligence modules 131 for performing object detection of a two-dimensional image is displayed, 3 In the dimensional object detection group (Object Detection 3D), a list of artificial intelligence modules 131 that perform object detection of a three-dimensional image is displayed, and the two-dimensional segmentation group (Segmentation 2D) is a segmentation of a two-dimensional image ( A list of artificial intelligence modules 131 for performing segmentation is displayed, and a 3D segmentation group (Segmentation 3D) is displayed with a list of artificial intelligence modules 131 for performing segmentation of a three-dimensional image.

DensNet, ResNet, and MobileNet algorithms are widely known as AI algorithms that perform image classification, and YOLO, SSD, and Retinanet algorithms are known as AI algorithms that perform object detection, and segmentation DeepLab and U-net algorithms are widely known as artificial intelligence algorithms that perform segmentation.

The artificial intelligence algorithm designed using the above algorithm is modularized and stored as the artificial intelligence module 131, and the user selects the artificial intelligence module 131 displayed on the module list display window 91 to design a learning model, Easier design is possible.

Here, on the modular screen according to the embodiment of the present invention, as shown in FIG. 9 , an information display window 93 is provided, and the learning model design unit 141 provides various information through the information display window 93 . .

The learning model design unit 141 when the user selects any one of the dataset 111, the image processing module 121, and the artificial intelligence module 131 displayed on the model list display window, the corresponding dataset ( 111) or information about the module can be displayed.

13 illustrates an example of information displayed on the information display window 93 when an edge detection module is selected among the image processing modules 121 . As information on the image processing module 121, as shown in FIG. 13, images before and after processing of the image processing module 121 are displayed, and brief information of the image processing module 121 is displayed at the bottom. displayed as an example. And, when the detailed information button (More) is clicked to provide more information, more detailed information about the image processing module 121 may be provided as shown in FIG. 14 .

Similarly, when any one of the artificial intelligence modules 131 is selected, a description of the function, main use, layer structure, etc. of the corresponding artificial intelligence module 131, and more detailed information through clicking the detailed information button (More) Information may be provided.

Hereinafter, a process of designing a learning model through the modular screen will be described.

When the user selects one of the lists displayed on the module list display window 91 and drags and drops it to the learning model design window 92, a module icon (or dataset) corresponding to the drag and drop on the learning model design window 92 (111) icon) is created.

15 shows that one dataset 111, two image processing modules 121, and one artificial intelligence module 131 are dragged and dropped on the learning model design window 92, and the dataset 111 icon and module An example of the state in which the icon is displayed is shown.

Then, when the user connects the icons of the dataset 111 and the module icons by line using a mouse cursor, the learning model design unit 141 creates a learning model by using the line connection between the icons as a data flow.

16 shows a state in which a line is connected between the icons displayed in the learning model design window 92, and the learning model design unit 141 indicates that the line connection is possible when the mouse cursor is positioned over the icon, and the mouse cursor If you move it and move it to another icon, the line connection in the form of an arrow is displayed while recognizing the data flow between the two icons in the direction of the cursor movement.

As described above, it is possible to design a learning model only with a simple operation such as drag and drop and a line connection between icons, so that it is possible to design a learning model more easily even if knowledge in the field of artificial intelligence is insufficient.

Meanwhile, as shown in FIG. 2 , the cloud platform system 100 according to an embodiment of the present invention may further include a plurality of layer modules. In the layer module, the layers applied to the configuration of the artificial intelligence algorithm are modularized for each function and stored, and are stored in the layer storage unit 170 .

The AI algorithm is formed in a network structure of multiple layers, and the layers to compose the AI algorithm are a core layer, a convolution layer, a pooling layer, and a merge layer. , including a normalization layer.

Here, when the icon of the artificial intelligence model displayed on the learning model design window 92 is selected, the learning model design unit 141 displays buttons for performing a function related to the icon as shown in FIG. 17 . 17a is an individual execution button 17a, which will be described later, 17b is a layer entry button 17b for checking and correcting the design structure of the corresponding AI model, 17c is a delete button for deleting the corresponding icon ( 17c).

When the layer entry button 17b is clicked, the learning model design unit 141 displays the layer list display window 18a in which a list of layer modules is displayed and the artificial intelligence design window 18b as shown in FIG. 18 as a graphic user interface. will be displayed in

In addition, the learning model design unit 141, as in the above-described modular screen, when a layer module or layer block displayed on the layer list display window 18a is selected, the layer information display window ( 18c) may be displayed in the graphical user interface. Here, parameter information may be displayed on the layer information display window 18c. In the present invention, it is exemplified that a parameter can be modified.

In the layer list display window 18a, a list of a plurality of layers is displayed separately for each group described above. As shown in FIG. 19 , if an icon button located at the top of the layer list display window 18a is clicked, a list corresponding to each group is displayed on the layer list display window 18a.

Here, when the user selects any one of the list of layer modules and moves it to the AI design window 18b through drag and drop, the learning model design unit 141 sets the icon of the corresponding layer module to the AI design window ( 18b).

And, as in the design of the learning model, when the layer icons displayed in the artificial intelligence design window 18b are line-connected, the learning model design unit 141 sets the line connection between the layer icons as the data flow to the AI module 131 ) will be created.

A user can create the AI module 131 designed by himself or herself by using the layer list display window 18a and the artificial intelligence design window 18b, and when sharing it, other users can utilize it. In addition, when designing another learning model, one can reuse the previously created artificial intelligence module 131 and use the artificial intelligence module 131 designed by others, etc. (131) will be able to be produced, shared and reused.

In addition, the cloud platform system 100 according to the present invention may further include a plurality of layer blocks that are configured as at least two or more layers and are modularized for generating the artificial intelligence module 131 .

Here, the learning model design unit 141 may also display a list of layer blocks on the layer list display window 18a, and drag the list of layer modules displayed on the layer list display window 18a to the artificial intelligence design window 18b. It can support the creation of layer blocks through n-drop and line connection.

In addition, a layer block may be created by dragging and dropping multiple layer modules and multiple layer blocks and connecting lines. That is, the design of the network structure of the layer module enables modularization of the layer block, the design of the network structure of the layer module and layer block enables the design of the layer block, and the design of the network structure of the layer module and layer block enables artificial intelligence It becomes a structure that can design and manufacture a model.

Such a design structure makes it possible to visually and simply check the design structure of the final artificial intelligence module 131, and it is possible to grasp the structure in the form of finding and entering the detailed structure, and the network structure, that is, the layer module, the layer Block and line connection structures can be easily understood.

That is, the detailed network structure of the artificial intelligence model in the network structure of the learning model shown in FIG. 17 is as shown in FIG. 18 . In addition, the layer block '#2 Blackbone_DenseNet121' shown in FIG. 18 has a detailed network structure as shown in FIG. 20, and the layer block '#12 dense_block' shown in FIG. 20 has the detailed network structure shown in FIG. It has a network structure.

Here, when the user sets the storage or sharing of the artificial intelligence module 131 and the layer block created through the artificial intelligence design window 18b, the learning model design unit 141 displays the corresponding artificial intelligence in the layer list display window 18a. You can update the list of modules 131 or layer blocks to make them available to other users.

In other words, the user can design various types of layer blocks using the layer module in which the basic layer algorithms that make up the AI algorithm are modularized, and can design other layer blocks by designing the network of layer modules and layer blocks. In this way, the final artificial intelligence module 131 can be generated.

In addition, as described above, through the sharing of the layer block and the artificial intelligence module 131 between users, redesign is possible by using the layer block or artificial intelligence module 131 designed by others, One AI module 131 or a layer block can also be reused, providing an environment that can be easily utilized for designing a new learning model.

Again, referring to FIG. 6 , when the 'RUN' button provided in the learning model design window 92 is clicked after the learning model is designed through the process as described above through the learning model design window 92, , the reading model generation unit 142 learns the learning model designed by the learning model design unit 141 to generate the reading model.

When normal learning is performed by the reading model generating unit 142, as shown in FIG. 22, the icon displayed on the learning model design window 92 is changed so that it can be visually confirmed that the normal operation is completed. In the invention, an example is visualized by changing from gray to green and displaying a check mark on the icon.

On the other hand, when an error occurs in learning, the error may be visually displayed in the form of displaying the icon of the image processing module 121 or the artificial intelligence module 131 in which the error occurred together with the error message in red. .

Here, the read model generator 142 according to the present invention may be provided to operate in any one of an entire learning execution mode and an individual module execution mode. The whole learning execution mode is a process of sequentially learning the whole by using the line connection as the data flow, based on the icon and the line connection displayed on the learning model design unit 141 through the click of the 'RUN' button as described above. means

On the other hand, the individual module execution mode is operable by clicking the individual execution button 17a that appears when the cursor is placed on the icons of the image processing module 121 and the artificial intelligence module 131 as shown in FIG. 17 .

Here, when the individual module execution mode is executed by clicking the individual execution button 17a activated for the corresponding icon while the cursor is long on the specific icon, the read model generation unit 142 executes the data set corresponding to the data icon. 111 executes only the image processing module 121 or the artificial intelligence module 131 corresponding to the icon selected by the individual execution button 17a according to the data flow according to the line connection.

For example, when the individual execution button 17a of the icon 'Color to Grayscale' in the learning model design window 92 of FIG. 22 is clicked, the read model generation unit 142 generates a dataset 111 corresponding to cerebral hemorrhage CT. ), only the image processing module 121 corresponding to 'Resize' and 'Color to Grayscale' is sequentially executed.

Through this, it is possible to check whether the learning model operates normally by executing only 'Resize' during pre-processing, for example, without performing learning while the entire design is completed in the design process of the learning model, so errors occurring in the design process can be checked in advance. This significantly reduces the time required for the design process.

Also, when the read model generation unit 142 operates in the entire learning execution mode and the individual module execution mode, the image processing module 121 and the artificial intelligence module 131 may store the processing results.

Then, the read model generation unit 142 changes at least one of the image processing module 121 and the artificial intelligence module 131 through the learning model design window 92, and then either the entire learning execution mode or the individual mode execution mode. When is executed, it saves time due to re-execution of the image processing module 121 or the artificial intelligence module 131 that has already been executed by calling and applying the previous processing results from the data flow according to the line connection to the line connection that is not changed. be able to do

If described with reference to FIG. 22, the number of times the user executed 'Resize' in individual module execution mode, 'Color to Grayscale' and 'VGG16' were designed to execute 'Color to Grayscale' in individual module execution mode or 'RUN' When executing in full learning execution mode by clicking the 'button, 'Resize', which has already been executed and stored the result, calls and applies the previous processing result, and only 'Color to Grayscale' and/or 'VGG16' can be executed have.

In addition, as shown in FIG. 22 , after execution is completed in the entire learning execution mode, 'Color to Grayscale' is deleted and replaced with another image processing module 121, and then the entire learning execution mode or individual module execution mode is changed. When executed, similarly, 'Resize' loads and applies the previous processing result. Here, it is natural that the module after the changed module according to the data flow is executed again because the processing result of the previous step is changed even if it is not modified.

On the other hand, when the module icon corresponding to the image processing module 121 displayed on the learning model design window 92 is selected, the learning model design unit 141 as shown in FIG. Images before and after image processing may be displayed in one area around the corresponding module icon, for example, at the top.

Through this, the user can easily check the before and after images of the images processed by each image processing module 121 in the pre-processing of the learning model currently designed by the user with only a simple mouse operation.

Here, the learning model design unit 141 changes the image included in the corresponding dataset 111 when scrolling of the mouse is recognized while the image before and after image processing is displayed around the corresponding module icon to display the image before and after image processing. can be displayed

In addition, the learning model design unit 141 is a data list in which a list of medical images constituting the dataset 111 is displayed when a module icon corresponding to the image processing module 121 displayed on the learning model design window 92 is selected. When any one of the display window 94 and the list displayed on the data list display window 94 is selected, the image display window 95 in which the medical image, ie, an image, of the corresponding list is displayed may be displayed on the graphic user interface.

In addition, the learning model design unit 141 may display the medical image displayed on the image display window 95 together with images before and after pre-processing of the image processing module 121 . Through this, the user can directly select each medical image constituting the dataset 111 and visually check the preprocessing result.

On the other hand, when any one of the icons corresponding to the image processing module 121 displayed on the learning model design window 92 is selected, the learning model design unit 141 according to the present invention displays detailed information of the corresponding image processing module 121 . The displayed information display window 93 may be displayed on the graphic user interface. 24 shows an example of the information display window 93 when the image processing module 121 is selected, and FIG. 9 shows an example of the initial information display window 93 of the modular screen.

In addition, the information display window 93 displays parameter information together with detailed information, and the parameter information determines the pre-processing of the corresponding processing module.

For example, the parameters of edge detection include a maximum threshold (Threshold_max) and a minimum threshold (Threshold_min), resampling includes a spacing value, and gamma correction includes a gamma value. , and histogram equalization may include a kernel size and a limit value.

Here, the learning model design unit 141 is set to be able to change the value of the parameter information displayed on the information display window 93 , so that the user can change the parameter value even if the modular image processing module 121 is used. The image processing module 121 may be generated. In this case, when the read model generator 142 executes the image processing module 121 , the changed parameter is applied and executed.

Through this, as described above, it is possible to immediately check the image processing result while changing the parameters in the individual module execution mode, so that it is possible to design a more efficient learning model in time.

As described above, when the design of the learning model and the generation of the reading model through learning are completed, the learning result, that is, the final artificial intelligence model to which parameters such as weights are applied, is generated, and the medical image reading unit 150 finally Using the generated artificial intelligence model, the medical image to be read is read.

Although several embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that changes may be made to these embodiments without departing from the spirit or spirit of the invention. . The scope of the invention will be defined by the appended claims and their equivalents.

[Explanation of code]

100: cloud platform system

111: dataset 121: image processing module

131: artificial intelligence module 141: learning model design unit

142: reading model generation unit 150: medical image reading unit

160: model storage unit 170: layer storage unit

The present invention is applicable to an AI-based cloud platform for reading medical images.

Claims

In the cloud platform system for medical image reading,

A plurality of pre-programmed and modularized image processing modules to perform pre-processing of medical images;

A plurality of artificial intelligence modules pre-programmed and modularized with artificial intelligence algorithms;

A learning model design unit that provides a graphic user interface for designing an artificial intelligence-based learning model to a user terminal accessed through a web browser;

a reading model generation unit configured to generate a reading model by learning the learning model designed by the learning model design unit;

The learning model design unit

Display a module list display window in which a list of the plurality of image processing modules and a list of the plurality of artificial intelligence modules are displayed, and a learning model design window on the graphic user interface,

When the list displayed on the module list display window is dragged and dropped to the learning model design window, a module icon is generated in response to the drag and drop in the learning model design window,

When the module icons are line-connected, the learning model is generated by using the line connection between the module icons as a data flow.
According to claim 1,

a data set storage unit storing a plurality of datasets classified according to at least one of a body part, a type of modality, a type of a disease target, and a type of an image dimension;

The learning model design unit

Display a list of the plurality of data sets in the module list display window,

When the dataset displayed on the module list display window is dragged and dropped to the learning model design window, a data icon corresponding to the corresponding dataset is displayed on the artificial intelligence design window,

when at least one of the module icons and the data icon are line-connected, generating the learning model by using the line connection with the module icon as a data flow;

The read model generation unit

A cloud platform system for medical image reading, characterized in that the data set corresponding to the data icon is trained on the learning model generated by the learning model design unit to generate the reading model.
According to claim 1,

It further includes a model storage unit in which the pre-generated reading model and the pre-designed learning model are stored;

The learning model design unit

displaying the list of the reading model or the learning model stored in the model storage unit on a graphical user interface,

When any one of the list of the reading model or the learning model displayed on the graphic user interface is selected, the design structure of the corresponding model is displayed on the artificial intelligence design window,

A cloud platform system for medical image reading, characterized in that it provides a function of modifying the design structure displayed on the artificial intelligence design window.
According to claim 1,

The layer applied to the configuration of the artificial intelligence algorithm further includes a plurality of layer modules modularized for each function;

The learning model design unit

A layer list display window in which a list of the plurality of layer modules is displayed and an artificial intelligence design window are displayed on the graphic user interface,

When the list displayed on the layer list display window is dragged and dropped onto the artificial intelligence design window, the layer icon of the corresponding layer module is displayed on the artificial intelligence design window,

When the layer icons displayed on the artificial intelligence design window are line-connected, the AI module is generated by using the line connection between the layer icons as a data flow.
5. The method of claim 4,

Doedoe consisting of at least two or more layers, further comprising a plurality of layer blocks modularized for generating the artificial intelligence module;

The learning model design unit

displaying a list of the plurality of layer blocks in the layer list display window,

A cloud platform system for medical image reading, characterized in that it supports the creation of the layer block through drag and drop and line connection of the list of layer modules displayed on the layer list display window to the AI design window.
6. The method of claim 5,

The learning model design unit

The cloud platform system for medical image reading, characterized in that when the artificial intelligence module and the layer block generated through the artificial intelligence design window are set to be stored or shared, the layer list display window is updated.