KR20200059183A - Method for assisting in generation of pathology report from pathology image and apparatus using the same - Google Patents

Abstract

According to the present invention, disclosed are a method for assisting in the creation of a pathology report from a pathological image, and a computing device using the same. Specifically, according to the method of the present invention, the computing device acquires a pathological image to be read; calculates a section of a predetermined lesion from the pathological image through a first determination unit configured to determine the predetermined lesion included in the pathological image; calculates a section of an anatomical object from the pathological image through a second determination unit configured to determine the anatomical object included in the pathological image; generates an analysis result of a pathology report for the predetermined lesion by reflecting at least one of an inclusion relationship between the section of the predetermined lesion and the section of the anatomical object, the dimension of the section of the predetermined lesion, the number of sections of the predetermined lesion, the dimension of the section of the anatomical object, and the number of sections of the anatomical object; and provides the generated analysis result of the pathology report to an external entity. According to the present invention, a clinical stage can be more conveniently and efficiently calculated.

Description

A method for assisting in the generation of a pathology report from pathology images and a device using the same {METHOD FOR ASSISTING IN GENERATION OF PATHOLOGY REPORT FROM PATHOLOGY IMAGE AND APPARATUS USING THE SAME}

The present disclosure discloses a method for assisting the generation of a pathology report from a pathology image, and a computing device using the method. Specifically, according to the method according to the present invention, the computing device acquires a pathology image to be read, and partitions the predetermined lesion from the pathology image through a first determination unit configured to determine a predetermined lesion included in the pathology image. Meanwhile, on the other hand, a section of the anatomical object is calculated from the pathological image through a second determination unit configured to determine an anatomical object included in the pathological image, and the section between the section of the predetermined lesion and the section of the anatomical object is included. Analysis of a pathology report on a given lesion by reflecting at least one of a relationship, the dimensions of the compartments of the given lesion, the number of compartments of the given lesion, the dimensions of the compartments of the anatomical object, and the number of compartments of the anatomical object A result is generated, and the result of analyzing the generated pathology report is provided to an external entity.

Whole slide images or microscopic pathological diagnosis of sections of body organs, including cells and tissues, play a decisive role in determining treatment options and predicting prognosis for patients. The diagnosis is mainly made by considering tissue and cell morphology. This pathological diagnosis is one of the problems of high degree of difficulty that differs not only between medical workers in the same country but also between countries. The detailed diagnosis is divided into various carcinomas and classes, and it is very difficult to find a target image that can be used as a reference for diagnosis because of large variation in characteristics between samples. The difficulty of this diagnosis is highly dependent on the pathologist's experience and intuition, and in practice, it is often ambiguous for an experienced pathologist to make the diagnosis by himself. This is one of the main causes of the mismatch of diagnosis between doctors and the waste of medical services such as consultation and re-reading.

In the diagnosis of lesions, especially cancer, among pathological diagnosis, confirming the degree of spread of cancer cells is an important indicator in the future treatment plan setting. This is called the stage of cancer, and TNM is generally used to determine the stage. In TNM, T (tumor) is the size and degree of infiltration of the primary tumor in the primary organ, N (node; lymph node) is how far it has spread from the primary tumor to surrounding lymph nodes, and M (metastasis) is another organ in the body. It means whether the cancer has spread. When the condition of cancer by TNM is determined, the stages of stage 1, stage 2, stage 3, and stage 4 can be briefly summarized. In general, early cancer, advanced cancer, and late stage cancer are included, including the concept of treatment results. Classification is also used.

As described above, it is necessary to calculate the number of lymph nodes including cancer in the calculation of the stage using the TNM method. The operation of identifying cancer in the lymph nodes and counting the lymph nodes takes a long time, and there is a problem in that there is generally no high agreement among readers.

Accordingly, in this disclosure, a first determination unit configured to determine whether or not a predetermined lesion such as cancer is present in a local area of a pathological image such as a lymph node slice image using artificial intelligence technology, such as a learned first deep neural network module and , Anatomy corresponding to a predetermined lesion from a pathology image using a second determination unit, for example, a second deep neural network module, capable of finding an anatomical object such as a lymph node from the pathology image and deriving the area and number of anatomical objects It is intended to propose a method for calculating the number of objects and the number of anatomical objects that do not correspond to a predetermined lesion, and thereby assisting in the preparation of a pathology report.

KR 10-1874348 B

An object of the present invention is to make it possible to assist in the preparation of a pathology report, such as calculating the stage more conveniently and efficiently than the conventional method for determining whether a lesion is present.

In addition, the present invention is a method for overcoming the performance limitations of an algorithm for detecting an anatomical object by changing the shape of the anatomical object by a lesion, and the user can modify the compartment for the automatically calculated anatomical object segment. It aims to provide a user interface that enables the user.

Specifically, the present invention aims to contribute to proper treatment for a patient by accurately calculating stages based on pathological images.

The characteristic configuration of the present invention for achieving the objects of the present invention as described above and realizing the characteristic effects of the present invention described below is as follows.

According to an aspect of the present invention, there is provided a method for assisting the generation of a pathology report from a pathological image, the method comprising: (a) a computing device acquires a pathological image to be read, or another device linked to the computing device Supporting a user to acquire the pathology image; (b) the computing device comprising: (i) a process of partitioning a predetermined lesion from the pathological image; And (ii) performing a process of partitioning an anatomical object from the pathological image, or assisting the other device to perform the process; (c) the computing device includes an inclusion relationship between a section of the predetermined lesion and a section of the anatomical object, a dimension of the section of the predetermined lesion, a number of sections of the predetermined lesion, a dimension of the section of the anatomical object, and the Generating an analysis result of a pathology report on the predetermined lesion by reflecting at least one of the number of sections of the anatomical object, or assisting the other device to generate the analysis result; And (d) the computing device providing the generated pathology report analysis result to an external entity or assisting the other device to provide the result.

According to another aspect of the present invention, a computer program comprising instructions implemented to perform the method according to the present invention is also provided.

According to another aspect of the present invention, a computing device for assisting in the generation of a pathology report from a pathological image is provided, the computing device comprising: a communication unit for obtaining a pathological image to be read; And (A) a process of partitioning a predetermined lesion from the pathological image. And (B) a processor that performs a process of partitioning an anatomical object from the pathological image or assists another device interlocked through the communication unit to perform the process, wherein the processor comprises: (i) a partition of the predetermined lesion; Reflecting at least one of the inclusion relationship between the sections of the anatomical object, the dimensions of the sections of the predetermined lesion, the number of sections of the predetermined lesion, the dimensions of the sections of the anatomical object, and the number of sections of the anatomical object A process of generating an analysis result of a pathology report on a given lesion, and (ii) performing a process of providing the generated result of the analysis of the pathology report to an external entity or assisting the other device to perform the process.

* According to an exemplary embodiment of the present disclosure, it is possible to present a certain criterion regarding the counting of anatomical objects, such as the count of lymph nodes, which has previously been inconsistent among readers, e.g., medical professionals, so that the inconsistency between readers It has the effect of lowering.

Therefore, according to an exemplary embodiment, it is possible not only to facilitate the diagnosis of an anatomical object, but also to facilitate the task of displaying and dataating a region of a predetermined lesion and an anatomical object. It has the effect of contributing as positive feedback to development.

Therefore, the time of the medical staff for diagnosing a specific disease is saved, and there is an effect that safety for more patients can be secured.

Such an effect according to an embodiment of the present invention can be applied not only to pathology images of a specific format (modality) but also to 2D images or 3D images of various formats, and the method of the present invention depends on a specific format or platform. Of course not.

The following drawings attached for use in the description of the embodiments of the present invention are only some of the embodiments of the present invention, and for those of ordinary skill in the art to which the present invention pertains (hereinafter referred to as "ordinary technicians") Other drawings can be obtained based on these drawings without effort leading to a new invention.
1 is a conceptual diagram schematically showing an exemplary configuration of a computing device that performs a method of assisting in the generation of a pathology report from a pathology image (hereinafter referred to as "pathology report generation assistance method") according to an embodiment of the present invention. .
2 is a conceptual diagram exemplarily showing a hardware and software architecture of a computing device that performs a pathology report generation assist method according to an embodiment of the present invention.
3 is a flowchart exemplarily showing a method for assisting in generating a pathology report according to an embodiment of the present invention.
4 is a diagram conceptually illustrating a specific deep neural network architecture as an example of a deep neural network that can be used in the first and second determination units of the method for generating a pathology report according to the present invention.
5A and 5B are conceptual diagrams exemplarily showing user interface elements provided according to embodiments of the method for generating a pathology report according to the present invention.

For a detailed description of the present invention to be described later, reference is made to the accompanying drawings that illustrate, by way of illustration, specific embodiments in which the invention may be practiced in order to clarify the objects, technical solutions and advantages of the invention. These embodiments are described in detail enough to enable a person skilled in the art to practice the present invention. In the description with reference to the accompanying drawings, the same reference numerals are assigned to the same components regardless of reference numerals, and redundant descriptions thereof will be omitted.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be implemented in various forms. Therefore, the embodiments are not limited to a specific disclosure form, and the scope of the present specification includes modifications, equivalents, or substitutes included in the technical spirit.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from other components. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

When an element is said to be "connected" to another element, it should be understood that other elements may be present, either directly connected to or connected to the other element.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “include” or “have” are intended to designate the existence of a described feature, number, step, action, component, part, or combination thereof, and one or more other features, numbers, or steps. It should be understood that it does not preclude the existence or addition possibility of the operation, components, parts or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art. Terms such as those defined in a commonly used dictionary should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined herein. Does not.

The terms "image", "image", "image data" or "image data" used throughout the description and claims of the present invention are discrete image elements (eg, pixels in a 2D image, 3D images) Refers to multi-dimensional data composed of voxels, or, in other words, an object that can be seen with the eye (eg, displayed on a video screen) or a digital representation of the object (eg, a file corresponding to a pixel or voxel output). It is a term to refer to.

For example, the “image” may be an image of a subject collected according to any imaging method known in the art, such as imaging of a microscope image. However, the image is not necessarily provided in a medical context.

For convenience of description, some exemplary image modalities may be assumed, but a person skilled in the art will understand that the image formats used in various embodiments of the present invention are not limited to microscopic images.

In addition, throughout the detailed description and claims of the present disclosure, 'learning' or 'learning' is a term referring to performing machine learning through computing according to a procedure, and is a mental such as human educational activity. Those skilled in the art will understand that it is not intended to refer to action.

The user referred to in the present disclosure may not only refer to a singular number, but may also refer to a plurality of users for the purpose of acquiring richly and overlapping images, particularly pathological images and related data, including training of images (learning ) Or in actual use, there may be a purpose for securing the integrity of the image.

Moreover, the present invention encompasses all possible combinations of the embodiments indicated herein. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, the specific shapes, structures, and properties described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment. In addition, it should be understood that the location or arrangement of individual components within each disclosed embodiment can be changed without departing from the spirit and scope of the invention. Therefore, the following detailed description is not intended to be taken in a limiting sense, and the scope of the present invention, if appropriately described, is limited only by the appended claims, along with all ranges equivalent to those claimed. In the drawings, similar reference numerals refer to the same or similar functions throughout several aspects.

Unless otherwise indicated or clearly contradicted by context in this specification, items referred to as singular encompass a plurality of things unless otherwise required in that context. In addition, in describing the present invention, when it is determined that detailed descriptions of related well-known structures or functions may obscure the subject matter of the present invention, detailed descriptions thereof will be omitted.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable a person skilled in the art to easily implement the present invention.

1 is a conceptual diagram schematically illustrating an exemplary configuration of a computing device that performs a pathology report generation assist method according to an embodiment of the present invention.

Referring to FIG. 1, the computing device 100 according to an embodiment of the present invention includes a communication unit 110 and a processor 120, and directly or indirectly with an external computing device (not shown) through the communication unit 110. Can communicate with the enemy.

Specifically, the computing device 100 may include typical computer hardware (eg, a computer, a processor, memory, storage, an input device and an output device, other devices that may include components of an existing computing device; a router) , Electronic communication devices such as switches, electronic information storage systems such as network-attached storage (NAS) and storage area networks (SAN), and computer software (ie, computing devices in a particular way) It may be to achieve the desired system performance by using a combination of commands). The storage may include a storage device such as a hard disk and a universal serial bus (USB) memory, as well as a storage device based on a network connection such as a cloud server.

The communication unit 110 of the computing device can transmit and receive requests and responses between other interlocked computing devices. As an example, such requests and responses are performed by the same transmission control protocol (TCP) session. It may be, but is not limited to, for example, may be transmitted and received as a user datagram protocol (UDP) datagram.

Specifically, the communication unit 110 may be implemented in the form of a communication module including a communication interface. For example, the communication interface includes wireless LAN (WLAN), wireless fidelity (WiFi) Direct, digital living network alliance (DLNA), wireless broadband (Wibro), world interoperability for microwave access (Wimax), high speed downlink packet access (HSDPA), etc. Includes wireless internet interface and short-range communication interfaces such as Bluetooth ™, radio frequency identification (RFID), infrared data association (IrDA), ultra-wideband (UWB), ZigBee, and near field communication (NFC). can do. In addition, the communication interface may represent any interface (eg, a wired interface) capable of communicating with the outside.

For example, the communication unit 110 may acquire a pathology image or the like to be read from an external device through a suitable communication interface. In addition, in a broad sense, the communication unit 110 may include or be interlocked with a keyboard, a mouse, other external input devices, a printing device, a display, and other external output devices for receiving commands or instructions.

In addition, the processor 120 of the computing device is a micro processing unit (MPU), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU) or a tensor processing unit (TPU), a cache memory ), And a data bus. Also, it may further include a software configuration of an operating system and an application performing a specific purpose.

2 is a conceptual diagram exemplarily showing a hardware and software architecture of a computing device that performs a pathology report generation assist method according to an embodiment of the present invention.

Referring briefly to the configuration of the method and apparatus according to the present invention with reference to FIG. 2, the computing device 100 may include an image acquisition module 210 as a component thereof. The image acquisition module 210 is configured to acquire a pathological image to which the method according to the present invention is applied. The image may be, for example, obtained from another image capturing device (not shown) that is linked through the communication unit 110.

A part or all of the acquired image may be transmitted to the lesion region compartment module 220 and the anatomical object region compartment module 230, respectively. In one embodiment, the lesion region partition module 220 may be implemented by a processor that executes certain instructions. Specifically, the lesion region partition module 220 may include a first determination unit (or lesion determination module) configured to partition a predetermined lesion from the pathology image, for example, a trained first deep neural network module. As another example compatible with this embodiment, the anatomical object region partitioning module 230 may be implemented by a processor executing predetermined instructions. Specifically, the anatomical object region partitioning module 230 may include a second determination unit (or anatomical object determination module) configured to partition an anatomical object from the pathological image, for example, a trained second deep neural network module. have. Since the description of the processor implementing the lesion region compartment module 220 and the anatomical object region compartment module 230 described above may be applied to the processor 120 described with reference to FIG. 1, the duplicate description is It will be omitted.

For example, each of the first deep neural network module and the second deep neural network module may perform an image in a pixel or voxel unit through a deep neural network model as well as a feature value layer and feature value generating a feature value (feature vector) using the same. It may include a classifier classified as. The feature value layer may typically be a fully-connected layer. The segmentation of the pathology image by the lesion area segmentation module 220 and the anatomical object area segmentation module 230 will be described later in detail. In addition, the descriptions of the first deep neural network module and the second deep neural network module included in the lesion region compartment module 220 and the anatomical object region compartment module 230 are merely exemplary descriptions to help understanding, and other embodiments It should not be construed as limiting or limiting.

At least one of the lesion region partition module 220 and the anatomical object region partition module 230 may be interlocked with the user interface module 240, which is incorrect by the first determination unit in the case of the lesion region partition module 220 In order to correct the calculation of the partition of a predetermined lesion, the anatomical object region partition module 230 is for correcting the calculation of the wrong anatomical object by the second determination unit. The user interface module 240 provides a user with certain user interface elements to provide a function to assist the user in correcting errors in the lesion region compartment module 220 and / or the anatomical object region compartment module 230. can do. The user interface that can be provided by the predetermined user interface module 240 will be described later with reference to FIGS. 5A and 5B.

Next, the pathology report generation module 250 reflects the section of the predetermined lesion calculated by the lesion region section module 220 and the section of the anatomical object calculated by the anatomical object region section module 230, and the pathology report generation module 250 Analysis of pathology reports on lesions can be generated. For example, if the lesion corresponds to cancer, and the anatomical object is a lymph node, the stage of cancer may be determined using the number of lymph nodes and the number of lymph nodes determined to be cancer, and the analysis result of the pathology report shows that the lymph node It can be generated to include information on the number of lymph nodes, the number of lymph nodes that are determined to be cancer, the number of lymph nodes that are not, the ratio therebetween, and the cancer stage based thereon.

The result of analysis of the generated pathology report may be provided to an external entity by the output module 260, for which the output module 260 may include a display device (not shown). Here, the external entity includes a user, an administrator, and the like of the computing device 100, but it should be understood that any other subject that needs an analysis result of the pathology report is included. The external entity may be an artificial intelligence entity that uses the information, or a storage device for storing the information. When the external entity is a human, the output module 260 may provide the prediction result to the external entity through a predetermined output device, for example, a user interface displayed on a display.

In one embodiment, in addition to the components shown in FIG. 2, an update module and a learning module 270 (not shown) may be further included. The update module and the learning module may include a first determination unit based on manipulation such as modification by a user. And updating at least one of the second determination unit.

As described above, the modules illustrated in FIG. 2 may be implemented by, for example, a communication unit 110 or a processor 120 included in the computing device 100, or by interworking the communication unit 110 and the processor 120. Those skilled in the art will understand, and will be described in more detail with respect to its functions and effects below.

Now, a method for assisting in generating a pathology report according to the present invention will be described in detail with reference to FIG. 3. 3 is a flowchart exemplarily showing a method for assisting in generating a pathology report according to an embodiment of the present invention.

Referring to FIG. 3, in the method for assisting in generating a pathology report according to the present invention, first, the image acquisition module 210 implemented by the communication unit 110 of the computing device 100 acquires the pathology image to be read, or the computing And supporting another device (not shown) linked to the device to obtain the pathology image (S100).

Next, in the method for assisting in generating a pathology report according to the present invention, the lesion area partitioning module 220 implemented by the computing device 100 partitions a predetermined lesion from the pathology image or causes the other device to partition the predetermined lesion The process (S200a; not shown) and the anatomical object region partitioning module 230 implemented by the computing device 100 partition the anatomical object from the pathology image or allow the other device to detect the anatomical object. Further comprising the step of performing a process (S200b; not shown) to support the partition (S200).

Specifically, in the process S200a, a partition of the predetermined lesion may be calculated from the pathological image through a first determination unit configured to determine the predetermined lesion included in the pathological image. In addition, specifically, in the process S200b, a section of the anatomical object may be calculated from the pathological image through a second determination unit configured to determine the anatomical object included in the pathological image.

4 is a diagram conceptually illustrating a specific deep neural network architecture as an example of a deep neural network that can be used in the first and second determination units of the pathology report generation assist method according to the present invention.

In the present invention, a deep neural network may be used to partition at least one of a predetermined lesion and an anatomical object. It is known to those skilled in the art that the greater the number of layers constituting this, the better the effect. However, there has been a disadvantage in that the weight of the model is not properly reflected when performing the back propagation due to the vanishing gradient problem in the artificial neural network, but the solution to this recently constitutes a deep neural network to preserve the gradient. Is the way to do it.

In particular, the deep neural network architecture illustrated in FIG. 4 is a convolutional artificial neural network (CNN) commonly referred to as Inception V3. Feature values expressed in vectors (i.e., feature vectors) use 8x8x2048 as in the illustrated example, or global average pooling (GAP) that averages over 8x8-sized regions. You can use a 2048-dimensional vector. This deep neural network model is trained to solve the problem of performing various classifications on the input image patch (image fragment).

For example, it can be said that partitioning an area having a specific attribute in an input image is a class of any one of "{included, not included}" pixels or voxels belonging to the partition.

Accordingly, the deep neural network model for detecting a lesion such as cancer is trained to distinguish and classify pixels or voxels corresponding to the lesion from pixels or voxels that do not, and the deep neural network model for detecting anatomical objects such as lymph nodes It may be learned to distinguish and classify pixels or voxels corresponding to an anatomical object from pixels or voxels that are not. In this case, the number of detected lymph node segments may be determined by using pixel connectivity or voxel connectivity.

However, in addition to the Inception V3 as described above, the ordinary technician can classify the training input, that is, the pathology image and the corresponding labeling data, that is, the classification of the input image through training using the information on the partition, that is, various sections There may be a kind of deep neural network algorithm, and it will be understood that it is not limited to the deep neural network of the specified structure illustrated in FIG. 4. For example, other methods exist, such as using a U-Net-like structure.

On the other hand, when each of the lesion region partition module 220 and the anatomical object region partition module 230 uses the first deep neural network module and the second deep neural network module, there may be an error in incorrect partition calculation by these deep neural network modules. As it is, it needs to be corrected.

In addition to the need to correct the error, it is also necessary to configure the compartment to be made by the user directly. To this end, in one embodiment of step S200, the computing device 100, the predetermined lesion in the process (S200a) Provide user interface elements for supporting at least one of user input, change, and deletion of at least one of the sections of the anatomical object in the section and process S200b, or assist the other device to provide, and the user In response to manipulation of the interface element, at least one of the compartment of the predetermined lesion and the compartment of the anatomical object may be input, changed, or deleted, or may be supported by the other device to be input, changed, or deleted.

5A and 5B are conceptual diagrams exemplarily showing user interface elements provided according to embodiments of the method for generating a pathology report according to the present invention. 5A and 5B, cancer is illustrated as an example of a predetermined lesion, and lymph nodes are illustrated as an example of an anatomical object.

Referring to FIG. 5A, a lymph node slice image 510, a calculation result 520 of a lesion region section module, and a calculation result 530 of an anatomical object region section module are illustrated, and the lymph nodes 540, 550 and lymph nodes Non-objects 560 are shown, respectively. The calculation result 520 of the lesion region compartment module and the cancer region 540a are displayed on the lymph node 540, and the calculation result 530 of the anatomical object region compartment module 530, the lymph node 540, and the lymph node 550 are respectively shown. Sections 540b and 550b are shown, each of which corresponds to a lymph node.

5B, the lymph node regions 540b and 550b are normally partitioned, but the region 560b does not correspond to the lymph nodes and is incorrectly partitioned, and the lymph node regions 550b 'are partitioned smaller than they are, so correct them. There is a need. In other words, the wrongly partitioned ones should be deleted, and the smaller ones must be corrected.

In addition, the actual shape of the lymph node is similar to kidney beans, and even if it is divided into two sections, it may be one lymph node. That is, since the lymph nodes shown in the same image may be counted as one or two according to the reader, it is necessary to delete the incorrectly counted section. On the contrary, it is necessary to divide a section counted as one into two or more sections, or to newly divide an unpartitioned area.

To this end, the user may arbitrarily adjust the desired section using a user interface element on the provided user interface, for example, a brush or eraser or similar tool 570 as an example well known to those skilled in the art.

3, the pathology report generation assist method according to the present invention includes a pathology report generation module 250 implemented by the computing device 100 between the section of the predetermined lesion and the section of the anatomical object. Of the pathology report for the given lesion by reflecting at least one of the inclusion relationship, the dimensions of the compartments of the given lesion, the number of compartments of the given lesion, the dimensions of the compartments of the anatomical object, and the number of compartments of the anatomical object Further comprising the step (S300) of generating an analysis result or supporting the other device to generate.

For example, the inclusion relationship between the compartment of the predetermined lesion and the compartment of the anatomical object may be a relationship in which a specific predetermined lesion, such as a compartment of cancer, should be included in a compartment of a specific anatomical object, such as a lymph node. This is because it can serve as a basis for pathological reports on cancer metastasis to lymph nodes.

Specifically, in one embodiment of step S300, the pathology report generation module 250 analyzes the pathology report from the results of calculating the number of anatomical objects diagnosed as lesions and the number of anatomical objects other than lesions In other words, the analysis result of the pathology report may be generated based on a ratio in which the predetermined lesion is included in the anatomical object.

For example, referring to FIG. 5A, a section 520 calculated by a lesion (cancer) region section module from a pathology image (lymph node slice image) 510 and a section calculated by an anatomical object (lymph node) region section module ( Based on the results of 530, the pathology report generation module 250 may generate an analysis result of a pathology report called “one cancer (1/2) out of two total lymph nodes”.

Referring back to FIG. 3, in the method for generating a pathology report according to the present invention, the output module 260 implemented by the processor 120 provides the result of analyzing the generated pathology report to an external entity or Further comprising the step (S400) for supporting the other device to provide the pathology report analysis results to the external entity.

On the other hand, prior to performing the pathology report generation auxiliary method according to the present invention, it is necessary to go through the step (S050) in which the first and second determination units utilized, for example, deep neural network modules are previously learned (S050). An update and learning module for learning, or a learning module 270 (not shown) may be executed by the processor 120.

For example, in order to learn the first determination unit, the learning module 270 may use pathology images for a plurality of subjects and labeling data marking (marking) predetermined lesions shown in the pathology images as training data. In order to learn the second determination unit, the learning module 270 may use pathological images for a plurality of subjects and labeling data indicating anatomical objects represented in the pathological images as training data. It will be appreciated that training data for this can be prepared substantially by the user interface described above.

Components shown in the accompanying drawings in this disclosure are illustrated as being realized in one computing device for convenience of description, but the computing device 100 performing the method of the present invention may be configured as a plurality of devices interlocked with each other. It will be understood that this may be covered by the claims appended to this disclosure. Accordingly, it is obvious that each step of the above-described method of the present invention can be performed by directly performing one computing device or supporting another computing device that is connected to the one computing device to perform one computing device.

As described above, in the method for assisting in generating a pathology report according to the present invention, an analysis result of a pathology report can be generated based on the identified predetermined lesions and anatomical objects. If the deleted information and the like are used as data for updating or additional learning of the first and second judgment units again, there is an advantage that these judgment units can perform more accurate division, and thus the present invention for taking these advantages According to the method for generating a pathology report according to the method, the processor 120 may include a first determination unit and a pathology image trained to determine the predetermined lesion included in the pathology image based on manipulation of the user interface element. The method may further include updating at least one of the second determination units trained to determine the included anatomical object or supporting the other device to update (S500). At this time, the data that was not considered in the previous training or learning is additionally considered, and since the errors in the previous training can be corrected, the accuracy of the first and second determination units is improved. There is an advantage that the performance is continuously improved.

Here, an entity that performs manipulation on the user interface element may be an external entity such as the medical professional.

As described above, the present invention has an effect capable of more conveniently and efficiently counting lesions and related anatomical objects over all the above-described embodiments.

The advantages of the technique described herein as the above embodiments can greatly relieve the burden of medical professionals placed in a busy medical environment that must be accurately determined or predicted based on a lot of diagnostic data even a day, while anatomy, such as lymph node counting It is possible to provide a certain criterion regarding the counting of enemy objects, thereby lowering the degree of disagreement among readers, thereby enabling more accurate determination of related diseases. In short, it is possible to promote the task of displaying and dataating a region of a predetermined lesion and an anatomical object, thereby contributing positively to the development of medical-related AI technology.

As described above, based on the description of various embodiments of the present disclosure, a person skilled in the art, the method and / or processes of the present invention, and the steps of the hardware and software suitable for a specific application of hardware, software, or software It can be clearly understood that it can be realized in any combination. The hardware may include general purpose computers and / or dedicated computing devices or specific computing devices or special features or components of a particular computing device. The processes include one or more processors having internal and / or external memory, such as microprocessors, controllers, such as microcontrollers, embedded microcontrollers, microcomputers, arithmetic logic units (ALUs), digital signal processors (digital signal processors) For example, it may be realized by a programmable digital signal processor or other programmable device. In addition, or alternatively, the processes may include application specific integrated circuits (ASICs), programmable gate arrays (eg, field programmable gate arrays (FPGAs), programmable logic units (PLUs) or programmable array logics). (Programmable Array Logic; PAL) or any other device capable of executing and responding to any other instruction, or any other device or combination of devices that can be configured to process electronic signals. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and / or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored on one or more machine-readable recording media.

Moreover, the objects of the technical solution of the present invention or parts contributing to the prior art may be implemented in the form of program instructions that can be executed through various computer components and recorded in a machine-readable medium. The machine-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the machine-readable recording medium may be specially designed and configured for the embodiments or may be known and usable by those skilled in the computer software field. Examples of machine-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and Blu-rays, and floptical disks. And magneto-optical media such as), and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions are stored and compiled or interpreted to be executed on a machine capable of executing a heterogeneous combination of processors, processor architectures, or combinations of different hardware and software, or any other program instructions, as well as any of the devices described above. Can be made using structured programming languages such as C, object-oriented programming languages like C ++, or advanced or low-level programming languages (assemblies, hardware description languages and database programming languages and techniques), machine code, This includes not only bytecode, but also high-level language code that can be executed by a computer using an interpreter.

Thus, in one aspect according to the present invention, when the above-described method and combinations thereof are performed by one or more computing devices, the combination of the method and method may be implemented as executable code that performs each step. In another aspect, the method can be implemented as systems that perform the steps, and the methods can be distributed in various ways across devices or all functions can be integrated into one dedicated, standalone device or other hardware. In another aspect, means for performing the steps associated with the processes described above may include any hardware and / or software described above. All such sequential combinations and combinations are intended to fall within the scope of this disclosure.

For example, the hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa. The hardware device may include a processor such as an MPU, CPU, GPU, TPU, which is combined with a memory such as ROM / RAM for storing program instructions and is configured to execute instructions stored in the memory, and an external device and a signal. It may include a communication unit for sending and receiving. In addition, the hardware device may include a keyboard, a mouse, and other external input devices for receiving commands written by developers.

In the above, the present invention has been described by specific embodiments such as specific components and limited embodiments and drawings, but this is provided only to help the overall understanding of the present invention, and the present invention is not limited to the above embodiments, Any person having ordinary skill in the art to which the present invention pertains may seek various modifications and variations from these descriptions.

Therefore, the spirit of the present invention is not limited to the above-described embodiment, and should not be determined, and all claims that are equally or equivalently modified with the claims appended to the present disclosure are the spirit of the present invention. It would be said to belong to the category of. For example, the described techniques are performed in a different order than the described method, and / or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if substituted or substituted by equivalents, appropriate results can be achieved.

Such an equivalent or equivalent modification will include a method that is logically equivalent, which can produce the same results as, for example, the method according to the present invention. The scope should not be limited by the examples described above, but should be understood in the broadest sense acceptable by law.

Claims (10)

A method for assisting in the generation of a pathology report from a pathology image,
(a) the computing device acquiring a pathology image to be read, or assisting another device linked to the computing device to acquire the pathology image;
(c) the computing device includes an inclusion relationship between a section of a predetermined lesion shown in the pathology image and a section of an anatomical object shown in the pathology image, a dimension of the section of the predetermined lesion, the number of sections of the predetermined lesion, and the anatomy Generating an analysis result of a pathology report on the predetermined lesion by reflecting at least one of a dimension of a section of an enemy object and a number of sections of the anatomical object, or assisting the other device to generate the analysis result; And
(d) providing, by the computing device, the generated pathology report analysis result to an external entity or assisting the other device to provide the result.
Method for generating a pathology report comprising a. According to claim 1,
Before step (c),
(b1) the computing device partitioning the predetermined lesion from the pathological image or supporting the other device to partition
Method for generating a pathology report further comprising a. According to claim 2,
Step (b1) is,
Providing, by the computing device, at least one user interface element to support at least one of a user's input, change, and deletion of the partition of the predetermined lesion or assisting the other device to provide the user interface; And
In response to manipulation of the user interface element, the computing device inputting, changing, or deleting a section of the predetermined lesion or assisting the other device to input, change, or delete a section of the predetermined lesion.
A method for generating a pathology report, characterized in that it comprises a. According to claim 1,
Before step (c),
(b2) the computing device partitioning the anatomical object from the pathological image or supporting the other device to partition
Method for generating a pathology report further comprising a. The method of claim 4,
Step (b2) is,
Providing, by the computing device, at least one user interface element to support at least one of a user's input, change, and deletion of a section of the anatomical object, or assisting the other device to provide the user interface; ; And
In response to manipulation of the user interface element, the computing device entering, changing, or deleting a segment of the anatomical object, or assisting the other device to enter, change, or delete a segment of the anatomical object
A method for generating a pathology report, characterized in that it comprises a. According to claim 1,
Step (c) is,
A method of generating a pathology report, characterized in that an analysis result of the pathology report is generated based on a ratio in which the predetermined lesion is included in the anatomical object. A computer program stored on a medium comprising instructions implemented to cause a computing device to perform the method of claim 1. In the computing device to assist in the generation of a pathology report from the pathology image,
A communication unit that acquires a pathology image to be read; And
(ii) the inclusion relationship between a section of the lesion indicated in the pathological image and a section of the anatomical object indicated in the pathological image, the dimension of the section of the predetermined lesion, the number of sections of the predetermined lesion, and the section of the section of the anatomical object A process of generating an analysis result of a pathology report on the predetermined lesion by reflecting at least one of a dimension and the number of sections of the anatomical object, and (iii) providing the generated result of analysis of the pathology report to an external entity A processor that performs a process or supports other devices interlocked through the communication unit to perform
A pathology report generating device comprising a. The method of claim 8,
The processor,
Before the (ii) process,
(i1) Process of partitioning a predetermined lesion from the pathological image
Pathological report generating device characterized in that to further perform or to support the other device to perform more. The method of claim 8,
The processor,
Before the (ii) process,
(i2) Process of partitioning an anatomical object from the pathological image
Pathological report generating device characterized in that to further perform or to support the other device to perform more.

Images