KR102316557B1 - Cervical cancer diagnosis system - Google Patents

Abstract

The present invention relates to an automatic cervical cancer diagnosis system that categorizes cervical data required for automatic diagnosis of cervical cancer according to accurate criteria, performs machine learning, and automatically diagnoses cervical cancer based on the machine learning,
a learning data generator for classifying unclassified photographed image data for the cervix transmitted from an external device or storage unit in a learning mode according to a combination of multi-level classification criteria and generating new learning data for each classification criterion; A photographed image pre-processing unit for pre-processing a cervical photographed image; A machine learning model for cervical cancer that learns the characteristics of the learning data generated by the classification criteria in a learning mode, wherein the machine learning model determines whether or not cervical cancer occurs with respect to the pre-processed image of the cervix in the diagnosis mode. a cervical cancer diagnosis unit for generating diagnosis information about the cervical cancer; a screen display control unit for displaying and outputting a user interface screen for displaying the diagnosis information and inputting evaluation information by a reader; a re-learning data generator for requesting re-learning of the machine learning model by extracting information necessary for re-learning from among the evaluation information input through the user interface screen; and a diagnosis and evaluation information storage unit for storing diagnosis information regarding the occurrence of cervical cancer and evaluation information input through the user interface screen.

Description

Cervical cancer automatic diagnosis system {CERVICAL CANCER DIAGNOSIS SYSTEM}

The present invention relates to an automatic cervical cancer diagnosis system, and in particular, automatic diagnosis of cervical cancer in which cervical data required for automatic diagnosis of cervical cancer is classified according to accurate criteria, machine learning, and automatic diagnosis of cervical cancer based on the machine learning It's about the system.

Cervical cancer ranks first among the cancers most feared by Korean women because hysterectomy can affect pregnancy and childbirth, and cause a sense of loss as a woman.

According to 2013 statistics, the number of cervical cancer patients in Korea was 26,207, ranking fourth among female cancers (data from the Ministry of Health and Welfare). In addition, cervical cancer is one of the seven major cancers recommended for screening in Korea, and as it was included in the national cancer screening program in 1999, the rate of early diagnosis is increasing. Recently, cervical carcinoma in situ (precancerous stage), also known as 'stage 0' cancer of the cervix, is also on the rise.

Looking at the current state of the market for the screening, the rate of cervical carcinoma in situ among young women is increasing, and from 2016, the screening target was lowered from 30 years old to 20 years old. In particular, unlike other cancers, health insurance benefits are applied to 300% of the cost of cervical cytology screening. However, as the false-negative rate (ie, false-diagnosis rate) of the screening is up to 55%, it is recommended to perform cervical magnification as a supplementary measure. As of 2013, the global cervical cancer screening market was worth about 6.86 trillion won In terms of scale, of these, cervical magnification examinations account for 30% of the total, amounting to approximately KRW 2 trillion.

1 is a conceptual diagram schematically illustrating a method of a cervical cell examination and a cervical magnification examination, which have been conventionally performed for diagnosing cervical cancer. Referring to the bottom of FIG. When a photographed image of the cervix is acquired through a photographing device (eg, a cervical magnifying glass illustrated in FIG. 1 ), the result is analyzed and the result is used to reduce the false diagnosis rate of cervical cancer screening.

However, when using a conventional cervical magnifying glass, medical staff check the occurrence of cervical cancer on the image of the cervix in light of education and experience. It may take a long time and the accuracy may decrease as well.

In order to overcome this disadvantage, a photographed image of the cervix is acquired, and from the obtained cervix image, based on a machine learning model for cervical cancer, analysis information on whether or not a subject has cervical cancer is generated and provided. An outbreak determination device is also being introduced.

The main factor for evaluating the performance of these judgment devices is that the learning should proceed by accurately classifying the images to be used for learning. If this sort of data classification is not done accurately and clearly, the analysis accuracy regarding the occurrence of cervical cancer is bound to decrease. In general, colposcopy images for cervical cancer appear in various forms differently from general medical images depending on the shooting environment and the photographer. Therefore, in devices for determining whether cervical cancer occurs, it is necessary to classify images to be used for learning according to clearer and stricter standards and proceed with learning.

In addition, even if machine learning is performed by classifying the learning data according to clear and strict criteria, the diagnosis (analysis) result may be misdiagnosed. It is urgently required to introduce a system capable of increasing the reliability of analysis devices for automatic diagnosis or supporting the diagnosis performance.

Republic of Korea Patent Publication No. 10-0850347 Republic of Korea Patent Publication No. 10-2016-0047720

Accordingly, the present invention is an invention devised in response to the above-mentioned necessity, and the main object of the present invention is to classify and machine learning cervical cancer data for learning necessary for automatic diagnosis of cervical cancer according to accurate criteria, and based on the machine learning, cervical cancer To provide an automatic cervical cancer diagnosis system that automatically diagnoses

Further, another object of the present invention is to induce an accurate diagnosis of the cervix by preventing the occurrence of excessive learning only for a specific type of image of the cervix or, conversely, no learning for a specific type of image. To provide an automatic cervical cancer diagnosis system.

Another object of the present invention is to provide convenience of reading by providing various information automatically analyzed based on a machine learning model to a reading doctor (expert), and input information evaluated by a reading physician can be used for real-time re-learning. The goal is to provide an automatic cervical cancer diagnosis system that configures the user interface screen to continuously improve the performance of the machine learning model.

An automatic cervical cancer diagnosis system according to an embodiment of the present invention for achieving the above object,

a learning data generation unit for classifying unclassified photographed image data of the cervix transmitted from an external device or storage unit in a learning mode according to a combination of multi-level classification criteria and generating new learning data for each classification criterion;

A photographed image pre-processing unit for pre-processing a cervical photographed image;

A machine learning model for cervical cancer that learns the characteristics of the learning data generated by the classification criteria in a learning mode, wherein the machine learning model determines whether or not cervical cancer occurs with respect to the pre-processed image of the cervix in the diagnosis mode. a cervical cancer diagnosis unit for generating diagnosis information about the cervical cancer;

a screen display control unit for displaying and outputting a user interface screen for displaying the diagnosis information and inputting evaluation information by a reader;

a re-learning data generator for requesting re-learning of the machine learning model by extracting information necessary for re-learning from among the evaluation information input through the user interface screen;

and a diagnosis and evaluation information storage unit for storing diagnosis information regarding the occurrence of cervical cancer and evaluation information input through the user interface screen.

In the above-described system configuration, the learning data generator further generates additional learning data by applying mirroring or cropping to the learning data for each classification criterion in order to adjust the numerical balance of the learning data for each classification criterion. and

Further, the learning data generation unit,

A first-stage classification criterion based on color, a second-stage classification criterion based on the size of the cervix in the photographed image data, a three-stage classification criterion based on the combination of color and shape in the cervical image data, and Another feature is that it is classified using a combination of at least two classification criteria among the four classification criteria based on , exposure and focus.

In some cases, the learning data generation unit,

According to the first-stage classification criterion based on color, the unclassified image data for the cervix is first classified, and the cervix size in the primary classified unclassified image data is the second-stage classification criterion as the classification criterion. According to the secondary classification, the unclassified image data for the cervix is thirdly classified according to the three-stage classification criterion based on the combination of color and shape in the secondary classified unclassified photographed image data.

On the other hand, in the automatic cervical cancer diagnosis system described above, the first-stage classification criterion includes a color value for identifying each of an acetic acid reaction image, a Lugol solution reaction image, a green filter image, and a general image as a classification criterion value,

The three-step classification criterion includes a combination of color values and shapes for identifying any one or more of blood, mucus, loops, colposcopy, treatment traces, and surgical tools as a classification criterion value in the cervical image data. characterized.

Furthermore, the cervical cancer diagnosis unit includes classification information on whether negative, atypical, benign and malignant for cervical cancer with respect to a cervical imaging image of a subject based on the machine learning model, accuracy information of the classification information, and negative It is characterized in that diagnostic information including positive determination information and morphological findings information is generated,

The screen display control unit includes a display area for all or part of a cervical photographed image of the subject, a history information display area for other images previously photographed for the same subject, a subject information exposure area, a negative/positive determination information input area, and a shape A screen divided into a medical observation information input area, a technical defect information input area related to the quality of the captured image, the diagnostic information output area derived based on the machine learning model, and an observation input area for reading is displayed and output as the user interface screen It is characterized by

According to the above-described technical problem solving means, the automatic cervical cancer diagnosis system according to an embodiment of the present invention classifies unclassified imaging image data for the cervix according to a combination of multi-stage classification criteria to train an artificial intelligence network such as a machine learning model. In this case, more accurate diagnosis of lesions can be made because learning can be made on image data having more diverse characteristics than learning using data classified using a classification criterion having only one characteristic (color, shape, brightness, etc.) There are advantages that can be

In addition, the present invention further generates additional learning data for adjusting the numerical balance of the learning data for each classification criterion and uses it for learning, so that excessive learning is carried out only for a specific type of cervical (cancer) image data, or , on the contrary, it provides an effect of accurately diagnosing the presence or absence of a lesion on the cervix by preventing a phenomenon in which learning does not proceed with respect to a specific type (or type) of an image.

Furthermore, the present invention automatically diagnoses whether there is a lesion for cervical cancer based on a machine learning model and displays the result externally through a user interface screen, and induces a reader to input evaluation information on the user interface screen, and the By extracting some of the evaluation information and using it for re-learning of the machine learning model, there is an advantage that the performance of the machine learning model can be continuously upgraded.

1 is a conceptual diagram schematically illustrating a method of a cervical cell examination and a cervical magnification examination performed to diagnose cervical cancer in the prior art.
2 is an exemplary configuration diagram of an automatic cervical cancer diagnosis system according to an embodiment of the present invention;
3 is a view for explaining a method for classifying cervical learning data according to an embodiment of the present invention.
4 is a view for explaining in more detail the multi-stage classification criteria for generating cervical learning data according to an embodiment of the present invention.
5 is a view for explaining an automatic cervical cancer diagnosis process according to an embodiment of the present invention.
6A to 6E are exemplary views of a user interface (UI) screen provided in an automatic cervical cancer diagnosis process according to an embodiment of the present invention;
7 is an exemplary view of malignant atypical vessels.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description of the present invention refers to the accompanying drawings, which show by way of illustration a specific embodiment in which the present invention may be practiced, in order to clarify the objects, technical solutions and advantages of the present invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention.

And those skilled in the art that 'learning' throughout the detailed description and claims of the present invention is a term referring to performing machine learning according to a procedure, and is not intended to refer to a mental action such as human educational activity. will be able to understand Also throughout this description and claims, the word 'comprise' and variations thereof are not intended to exclude other technical features, additions, components or steps. Other objects, advantages and characteristics of the present invention will appear to a person skilled in the art, in part from this description, and in part from practice of the present invention. The following illustrations and drawings are provided by way of illustration and are not intended to limit the invention. Moreover, the invention encompasses all possible combinations of the embodiments indicated herein. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. In addition, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the invention, if properly described, is limited only by the appended claims, along with all scope equivalents to those claimed. Like reference numerals in the drawings refer to the same or similar functions throughout the various aspects.

Unless otherwise indicated herein or otherwise clearly contradicted by context, items referred to in the singular encompass the plural unless the context requires otherwise. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, in order to enable those skilled in the art to easily practice the present invention, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

For reference, a convolutional neural network (CNN) model, which is one of the machine learning models used in the present invention, may be briefly described in the form of a multi-layered artificial neural network. That is, it is expressed as a deep neural network (deep neural network) in the sense of a network of a deep structure, and it automatically learns the characteristics of each image by learning a large amount of data from a structure consisting of a multi-layered network, This is a form of learning the network in a way that minimizes the error of the objective function. At first glance, this can be expressed as a connection between nerve cells in the human brain, and accordingly, it is positioned as a representative of artificial intelligence. In particular, CNN is a model suitable for classification of two-dimensional images such as images. It reduces the size of a convolution layer that creates a feature map using a plurality of filters for each region of the image and the size of the feature map. By repeating the pooling layer (sub-sampling layer) that allows the extraction of features that are invariant to changes in position or rotation, from low-level features such as points, lines, and planes to complex and meaningful high-level features It is possible to extract features at various levels, and finally, if the features extracted through the fully-connected layer are used as input values for the existing model, there is an advantage in that a classification model with higher accuracy can be built. .

2 is a diagram illustrating the configuration of an automatic cervical cancer diagnosis system according to an embodiment of the present invention. As shown, the automatic cervical cancer diagnosis system 200 according to an embodiment of the present invention is connected to the imaging device 100 , the storage medium 300 , the communication unit 400 , and the display unit 500 , and is a single computer operable It may be a system or a server computer system, and may be implemented as an aggregate (in the form of an application program) of executable code data that is installed and executable in the memory of the computer system.

Referring to FIG. 2 , the automatic cervical cancer diagnosis system 200 according to an embodiment of the present invention may communicate directly or indirectly with a computing device (not shown) of a doctor located at a remote location through the communication unit 400 . Of course, the reader may directly input (using a keyboard or mouse) evaluation information to be described later through the user interface screen displayed on the display unit 500 .

The imaging apparatus 100 acquires an image of the cervix of the subject in the diagnosis mode and transmits it to the automatic cervical cancer diagnosis system 200 to be described later, and the storage medium 300 stores the cervix to be learned by a machine learning model to be described later. Save the unclassified captured image data for

Hereinafter, the configuration of the automatic cervical cancer diagnosis system 200 will be described in detail with reference to FIG. 2 . The automatic cervical cancer diagnosis system 200 according to an embodiment of the present invention includes:

Learning data generation unit 210 that classifies unclassified photographed image data of the cervix transmitted from an external device or storage unit (storage medium, 300) in the learning mode according to a combination of multi-level classification criteria and generates new learning data for each classification criterion. Wow,

A photographed image pre-processing unit 220 for pre-processing (improving image quality, blurring, and noise processing) of a cervical photographed image;

A machine learning model for cervical cancer that learns the characteristics of the learning data generated by the classification criteria in a learning mode, wherein the machine learning model determines whether or not cervical cancer occurs with respect to the pre-processed image of the cervix in the diagnosis mode. A cervical cancer diagnosis unit 260 that generates diagnostic information related to;

a screen display control unit 250 for displaying and outputting a user interface screen for displaying the diagnostic information and inputting evaluation information by a reader;

a re-learning data generation unit 230 for requesting re-learning of the machine learning model by extracting information necessary for re-learning among the evaluation information input through the user interface screen;

and a diagnosis and evaluation information storage unit 240 for storing diagnosis information regarding the occurrence of cervical cancer and evaluation information input through the user interface screen.

For reference, the multi-level classification criteria, diagnostic information and evaluation information on the subject may be stored in the storage medium 300 .

The learning data generator 210 applies up/down or left/right mirroring or cropping technology to the learning data for each classification criterion to adjust the numerical balance of the learning data for each classification criterion generated in the learning mode to further learn By generating more data, excessive learning only for a specific type of cervical (cancer) image data or, conversely, a phenomenon in which normal learning does not proceed with respect to a specific type (or type) image is prevented from occurring.

On the other hand, the learning data generator 210 classifies the unclassified photographed image data for the cervix in the learning mode, the first-stage classification criterion using color as the classification criterion, and the cervix size in the photographed image data as the classification criterion. Classification using a combination of at least two of two-stage classification criteria, three-stage classification criteria based on the combination of color and shape in cervical image data, and four-stage classification criteria based on exposure and focus It is characterized by

In a specific implementation method, the learning data generation unit 210 includes:

First-class classification of unclassified imaging image data for the cervix according to the first-stage classification criterion based on color, and the second-stage classification criterion based on the cervix size in the primary classified unclassified image data. Secondary classification may be performed, and the unclassified photographed image data for the cervix may be thirdly classified according to a three-step classification criterion based on a combination of colors and shapes in the secondly classified unclassified photographed image data.

In addition, the learning data generator 210 may fourthly classify the unclassified captured image data for the tertiary-classified cervix according to the four-stage classification criteria using exposure and focus as classification criteria. These four-stage classification criteria are applied first before the first-stage classification criteria, and thus can be used as learning data for filtering the captured image data that cannot be diagnosed (no lesions are identified). For example, in the case of underexposure/overexposure, the histogram is extremely biased to one side, so it can be used for classification. It can be classified (quaternary classification) using characteristics.

Additionally, the learning data generation unit 210 first classifies the unclassified photographed image data for the cervix according to the first-stage classification criterion using color as the classification criterion, but the first-stage classification criterion is an acetic acid reaction image, Lugol A color value for identifying one or more of a solution reaction image, a green filter image, and a general image is included as a classification reference value.

In addition, the learning data generation unit 210 may include the size of the cervix in the first classified unclassified captured image data, for example, 150%, 100%, 80%, 50% of the size of the cervix in the image, including a magnifying glass and other parts. Secondary classification based on size.

Furthermore, the learning data generation unit 210 is configured to unclassify the cervix according to the three-stage classification criterion based on the combination of color and shape in the cervix image data in the secondly classified unclassified photographed image data for the cervix. A tertiary classification of the photographed image data, wherein the three-stage classification criterion is a combination of color values and shapes for identifying at least any one or more of blood, mucus, loops, colposcopy, treatment traces, and surgical tools in the cervical image data is included as a classification standard value to classify foreign substances affecting the cervix.

For example, blood usually appears as a reddish color that flows down from the center of the cervix, mucus appears mainly as a pale yellow color that flows down from the center of the cervix, and the loop is located mainly in the center of the cervix and is usually a boomerang. The shape of the wire is clearly visible. Colposcopy and other surgical instruments appear in different colors (silver, blue,...) from the pink cervix. Foreign substances can be classified.

In the illustrated 1st to 4th classification process, each classification can be performed using CNN, which is a deep learning technique. In the 1st, 2nd, and 4th classification, high-accuracy classification is possible even with a small number of layers because the features to be extracted are clear. can be configured to increase accuracy.

Hereinafter, the learning data classification method of the automatic cervical cancer diagnosis system 200 according to an embodiment of the present invention will be described in more detail with reference to FIGS. 3 and 4 .

First, FIG. 3 is a diagram for explaining a method for classifying cervical learning data according to an embodiment of the present invention, and FIG. 4 is a multi-stage classification criterion for generating cervical learning data according to an embodiment of the present invention in more detail. A drawing for explanation is shown.

Referring to FIG. 3 , before proceeding with learning of unclassified photographed image data for the cervix, unclassified photographed image data for the cervix transmitted from the imaging apparatus 100 or an external device is stored in the storage medium 300 . It is assumed that they are stored (step S100) or stored.

Under this assumption, if there is an execution command of the learning mode from the manager, the learning data generation unit 210 classifies each unclassified captured image data for the cervix based on a neural network algorithm such as CNN, but in a combination of a plurality of multi-level classification criteria. They are sorted and stored first (step S200).

For example, the learning data generator 210 first classifies the unclassified photographed image data of the cervix according to the first-stage classification criterion using color as the classification criterion.

For the primary classification, the learning data generation unit 210 includes a color value for identifying each of the acetic acid reaction image, Lugol solution reaction image, green filter image, and general image as a classification reference value to classify the above four images. can

Specifically, the acetic acid reaction image can be distinguished from the pink cervix and vagina because white stains appear on the cervix. Lugol solution reaction images show brown or dark orange color, and green filter images show strong green throughout the image. can be classified.

When the primary classification is completed, the learning data generation unit 210 performs secondary classification according to the second-stage classification criterion using the size of the cervix in the primary classified captured image data as the classification criterion.

The cervix is a circle the size of a 500 won coin and is usually located in the center of the image. Therefore, based on the size of the cervix in the image (150%, 100%, 80%, etc.), only the cervix is enlarged, the entire cervix is shown, the cervix is only 80% of the image, and the cervix is 50% of the image. Secondary classification is possible, such as an image that only shows the degree, or an image that includes the cervix + magnifying glass + other parts.

Thereafter, the learning data generation unit 210 tertiarily classifies foreign substances affecting the cervix according to a three-step classification criterion based on a combination of color and shape in the second classified cervical image data.

As mentioned earlier, blood mainly appears in the form of red flowing down from the center of the cervix, and mucus is mainly in the form of pale yellow flowing down from the center of the cervix. A boomerang-shaped wire appears clearly. Colposcopy and other surgical instruments appear in different colors (silver, blue,...) from the pink cervix. Foreign substances can be classified.

As described above, the captured image data for each classification criterion classified according to the multi-level classification criterion combination may be temporarily stored in the storage medium 300 or the internal memory of the system 200 .

When the classification of the unclassified captured image data is completed, the learning data generating unit 210 generates the captured image data for the cervix classified by the classification criterion as learning data for each classification criterion and stores it in the storage medium 300 (step S300) do. In this learning data generation process, the learning data generator 210 may further generate additional learning data to adjust the numerical balance of the learning data for each classification criterion, and the additional learning data is generated based on the learning data for each classification criterion. It is preferable to do

As a method to generate additional training data, vertical mirroring that changes left and right and horizontal mirroring that changes up and down using mirroring can be used. You can generate additional training data by running a ping. In addition, if mirroring and cropping are used together, it is possible to generate up to 16 times the additional training data.

For reference, in the above example, the training data was generated by applying the first-stage classification criterion, the second-stage classification criterion, and the three-stage classification criterion combination. And it is possible to generate learning data by applying a combination of three-stage classification criteria, and simply as shown in FIG. 4, by applying stage 1 to stage 3 classification criteria, respectively, classifying unclassified image data for the cervix by classification criteria It can also be used as training data.

As described above, when the unclassified imaging image data for the cervix is classified according to the multi-level classification criterion combination and learning data is generated, the machine learning model 262 for the cervix learns about the characteristics of the learning data generated for each classification mood. Execute verification (step S400). If unclassified imaging image data for the cervix is additionally secured, the above-described learning mode is performed again.

As described above, the automatic cervical cancer diagnosis system 200 according to the embodiment of the present invention classifies unclassified imaging image data for the cervix according to a combination of multi-stage classification criteria to generate learning data. As the number of classifications of the learning data increases, learning is performed on image data having various characteristics, resulting in an effect of more accurately diagnosing the presence or absence of a lesion.

In addition, the present invention further generates additional learning data for adjusting the numerical balance of learning data for each classification criterion and uses it for learning, so that excessive learning is carried out only for a specific type of cervical (cancer) image data, or , on the contrary, it provides an effect of accurately diagnosing the presence or absence of a lesion on the cervix by preventing a phenomenon in which learning does not proceed with respect to a specific type (or type) of an image.

Hereinafter, the automatic cervical cancer diagnosis process of the cervical cancer automatic diagnosis system 200 in which the machine learning model 262 is trained using the learning data generated by classifying the unclassified imaging image data for the cervix according to the above-described multi-step classification criterion combination. to be explained.

5 is a diagram illustrating an automatic cervical cancer diagnosis process according to an embodiment of the present invention, and FIGS. 6A to 6E are user interfaces (UIs) provided in the automatic cervical cancer diagnosis process according to an embodiment of the present invention; user interface) screen.

Referring to FIG. 5 , first, in the diagnosis mode, the automatic cervical cancer diagnosis system 200 acquires a photographed image of the cervix of the subject (step S410 ). The photographed image of the cervix of the subject may be obtained from the imaging apparatus 100 or may be obtained from the outside through the communication unit 400 or a storage medium. The captured image 514 of the cervix obtained in this way is displayed on the user interface screen by the screen display controller 250 as shown in FIG. 6A . The user interface screen may be configured so that information about the subject, which is a patient, together with the captured image 514 may be displayed through the subject information input area 510 as shown in FIG. 6A .

In order to generate more accurate diagnostic information before displaying the acquired image of the cervix of the subject on the user interface screen, it is preferable that the photographed image pre-processing unit 220 performs pre-processing on the photographed image of the cervix. . Here, the pre-processing may include at least one of RGB-HSV conversion for robustness to illuminance and noise of the captured image, image quality improvement through histogram smoothing, and blurring and noise processing.

Referring back to FIG. 5 , when the pre-processing is completed by acquiring a photographed image of the cervix of the subject, the machine learning model 262 of the cervical cancer diagnosis unit 260 determines the onset of cervical cancer with respect to the pre-processed image of the cervix. It generates diagnostic information on whether or not (step S420).

The machine learning model 262 has a plurality of pre-input training information, that is, (i) learning data for each classification criterion according to a combination of multi-stage classification criteria, and (ii) cervical cancer lesions in the plurality of learning data images. Diagnosis information regarding the occurrence of cervical cancer is generated using information including data indicating whether or not the lesion exists, and (iii) data indicating in which part of the image the lesion is present, if there is a lesion. For reference, the machine learning model 262 may be a convolutional neural network (CNN) model or a model combining CNN and a support vector machine (SVM).

A large amount of training data is required in order to increase the accuracy and reliability of the machine learning model 262 , and the more the training data is, the more the accuracy and reliability of the machine learning model can be improved.

Meanwhile, the diagnostic information may include classification information on whether the cervical cancer is negative, atypical, positive, and malignant. Also, the classification information may include probability information indicating how accurate the classification is.

More specifically, the diagnostic information may include negative-positive determination information, which is information on whether or not the diagnosis is negative, and whether the risk is high or low (low cancer risk vs. high cancer risk) if positive, Acetowhite Epithelium, Mosaic, Morphological findings such as erosion or ulceration, irregular surface contour, puncture, atypical vessels, and discolaration may be further included.

As shown in FIG. 6B , the diagnostic information may be listed and provided to correspond to a plurality of captured images 514 by the screen display controller 250 . The user interface screen shown in FIG. 6B indicates whether the onset of cervical cancer is suspected according to the subject information 520 , the input time information 522 , the classification information and the probability information calculated by the machine learning model 262 . Information (marked 'suspicious'; 524) is displayed. Furthermore, on the user interface screen shown in FIG. 6B , 'Evaluation' buttons 526 corresponding to a specific captured image are shown so that subsequent steps following step S420 can be continuously performed.

Through the 'Evaluation' button 526, the reading doctor can read whether or not the cervical cancer is diagnosed automatically based on the machine learning model and input evaluation information. That is, when the 'Evaluation' button 526 is manipulated, the screen display control unit 250 supports several screens to read whether cervical cancer has occurred.

For example, as shown in FIG. 6C , by providing all or part of the captured image on the user interface screen, the reader can make various necessary displays using the tool window while determining whether there is a medically unusual area. There are (eg, rectangles, arrows, text input, etc.).

In addition, the screen display control unit 250 may display diagnostic information corresponding to a reading command input through the user interface screen, and supports input of evaluation information by a reader (step S430).

The evaluation information refers to whether the diagnostic information provided is accurate, that is, information on whether the onset included in the diagnostic information is correct or not, and whether the classification information included in the diagnostic information is correct or incorrect. It may contain information about

In addition, the evaluation information may include information on quality of the captured image, for example, information on technical defects of the captured image. For example, such a technical defect is that it is difficult to accurately determine the captured image due to excessive mucus or blood in the photographed image, or it is difficult to determine whether cervical cancer occurs due to the angle of the photographed image or the location of the photographed area. It may also be an acetic acid reaction that should be present, but the acetic acid reaction is insufficient, or it may be a problem with the image corresponding to out of focus, overexposure or underexposure.

FIG. 6D shows a user interface screen displayed by the screen display control unit 250, in which all or part of the captured image 540 is displayed, and history information of other images previously captured for the same subject ( The area 541 is displayed, the subject information exposure area 542 as illustrated in FIG. 6A , the negative/positive determination information input area 543 into which negative/positive determination information can be input, and morphological observation information by the reader Derived by the morphological observation information input area 544 into which can be inputted, the technical defect information input area 545 into which information on technical defects of the photographed image 540 can be input, and the machine learning model 262 An artificial intelligence diagnostic information output area 546 indicating the acquired diagnostic information and a user opinion input area 547 in which the reader can input an opinion based on the captured image are displayed on the user interface screen, whereby automatic diagnosis information It makes it easier for the reader to evaluate whether or not the corresponding cervical cancer has occurred.

On the other hand, when the reading doctor inputs the evaluation information for the diagnosis information analyzed based on the captured image of the cervix of the subject and the machine learning model 262 through the user interface screen shown in FIG. 6, the diagnosis and evaluation information storage unit (240) stores the diagnosis information on the occurrence of cervical cancer and the evaluation information of the reading intention input through the user interface screen in the storage medium 300 (step S440) or the display unit 500 when an administrator or a reading request is made output it to

On the other hand, when the evaluation information is input through the user interface screen, the re-learning data generation unit 230 is information necessary for re-learning among the evaluation information input through the user interface screen, for example, yin-positive determination information, technical defect information It is possible to request re-learning of the machine learning model 262 by extracting and so forth.

In addition, the above-described evaluation information may be processed and provided in the form of a medical result report. For example, a user interface provided for this purpose is well shown in FIG. 6E , and the medical result report 550 indicates whether cervical cancer occurs, classification information may be included. Such a medical result report may be provided through another device located far from the place where the captured image was initially obtained (ie, the filming location) and the place where the captured image was read.

In the above embodiment, it is assumed that a doctor is located in the automatic cervical cancer diagnosis system 200 to input evaluation information. It can be transmitted to so that evaluation information can be input.

According to the embodiment of the present invention described above, since it is possible to automatically diagnose the onset of cervical cancer based on the machine learning model 262 learned in advance, a cervical imaging image obtained by a medical staff through a cervical magnifying glass It has the advantage of being able to automatically diagnose the onset of cervical cancer faster and more accurately than the previous method of checking the condition of the cervix one by one based on education and experience.

In addition, the present invention has the advantage that more accurate reading can be performed because the evaluation information can be used again as data for re-learning for the machine learning model.

Meanwhile, in the above-described embodiment, the first-stage classification criterion based on color, the second-stage classification criterion based on the size of the cervix in the photographed image data, and the combination of color and shape in the cervical image data are used as the classification criterion. Although the three-step classification criteria and the four-step classification criteria based on exposure and focus have been exemplified, the learning data generator 210 classifies the malignant atypical vessel pattern as shown in FIG. 7 as the classification criteria. Learning data can also be generated by classifying the unclassified image data of the cervix according to the 5-stage classification criterion alone or the combination of the 5-stage classification criterion and other classification criteria exemplified above by adding additional 5-stage classification criteria.

When the learning of the malignant atypical vascular pattern shown in FIG. 7 is performed, the cervical cancer diagnosis unit 260 generates a diagnosis on whether cervical cancer showing the malignant atypical vascular pattern occurs based on the learned machine learning model. will be able

Based on the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be achieved through a combination of software and hardware or can be achieved only with hardware. The objects of the technical solution of the present invention or parts contributing to the prior art may be implemented in the form of program instructions that can be executed through various computer components and recorded in a machine-readable recording medium. The machine-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the machine-readable recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software.

Examples of the machine-readable recording medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and DVD, and a magneto-optical medium such as a floppy disk. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa. The hardware device may include a processor, such as a CPU or GPU, coupled with a memory such as ROM/RAM for storing program instructions and configured to execute instructions stored in the memory, and may send and receive signals to and from an external device. It may include a communication unit. In addition, the hardware device may include a keyboard, a mouse, and other external input devices for receiving commands written by developers.

In the above, the present invention has been described with specific matters such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , a person of ordinary skill in the art to which the present invention pertains can make various modifications and variations from these descriptions. Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims described below, but also all modifications equivalently or equivalently to the claims described below belong to the scope of the spirit of the present invention. will do it

Claims (9)

In the learning mode, the first-stage classification criterion is based on color, the second-stage classification criterion is based on the size of the cervix in the photographed image data, and the third-stage classification criterion is based on the combination of color and shape in the cervical image data. According to a combination of multi-level classification criteria including any one or more of the four-stage classification criteria based on overexposure and focus, and the five-stage classification criterion based on the classification of malignant atypical vascular patterns, Classify the unclassified image data for the delivered cervix, first apply one classification criterion constituting the multi-level classification criterion combination to pre-classify, and sequentially apply the remaining classification criteria to the pre-classified image data for classification a learning data generator for classifying the unclassified photographed image data for the cervix in such a manner as to generate new learning data for each classification criterion;
A photographed image pre-processing unit for pre-processing a cervical photographed image;
A machine learning model for cervical cancer that learns the characteristics of the learning data generated by the classification criteria in a learning mode, wherein the machine learning model determines whether or not cervical cancer occurs with respect to the pre-processed image of the cervix in the diagnosis mode. a cervical cancer diagnosis unit for generating diagnosis information about the cervical cancer;
a screen display control unit for displaying and outputting a user interface screen for displaying the diagnosis information and inputting evaluation information by a reader;
a re-learning data generator for requesting re-learning of the machine learning model by extracting information necessary for re-learning from among the evaluation information input through the user interface screen;
A diagnosis and evaluation information storage unit for storing diagnosis information regarding the occurrence of cervical cancer and evaluation information input through the user interface screen;
and the learning data generator further generates additional learning data by applying mirroring or cropping to the learning data for each classification criterion in order to adjust the numerical balance of the learning data for each classification criterion. delete delete delete The method according to claim 1, wherein the first-stage classification criterion includes a color value for identifying each of an acetic acid reaction image, a Lugol solution reaction image, a green filter image, and a general image as a classification criterion value,
The three-step classification criterion includes a combination of color values and shapes for identifying any one or more of blood, mucus, loops, colposcopy, treatment traces, and surgical tools in the cervical image data as a classification criterion value. Cervical cancer automatic diagnosis system. The method according to claim 1, The cervical cancer diagnosis unit,
Based on the machine learning model, classification information on whether negative, atypical, benign and malignant for cervical cancer on a cervical imaging image of a subject, accuracy information of the classification information, yin-positive determination information, and morphological findings A cervical cancer automatic diagnosis system, characterized in that generating diagnostic information including the information. The method according to claim 1, wherein the screen display control unit,
Display area of all or part of the cervical imaging image of the subject, history information display area of other images taken previously for the same subject, subject information exposure area, yin-positive determination information input area, morphological observation information input area , a screen divided into a technical defect information input area related to the quality of the captured image, the diagnostic information output area derived based on the machine learning model, and an observation input area of reading, as the user interface screen, characterized in that it is displayed and output Cervical cancer automatic diagnosis system. delete delete

Images