KR102390347B1 - Method for assisting in generation of pathology report from pathology image and apparatus using the same - Google Patents

Abstract

Disclosed herein are a method of assisting in generation of a pathology report from a pathology image and a computing device using the same. Specifically, according to the method according to the present invention, a computing device acquires a pathological image to be read, and divides the predetermined lesion from the pathological image through a first determination unit configured to determine a predetermined lesion included in the pathological image On the other hand, the division of the anatomical object is calculated from the pathological image through a second determination unit configured to determine the anatomical object included in the pathological image, and includes between the division of the predetermined lesion and the division of the anatomical object. Analysis of the pathology report for the given lesion by reflecting at least one of a relationship, a dimension of a section of the given lesion, a number of sections of the given lesion, a dimension of a section of the anatomical object, and a number of sections of the anatomical object A result is generated, and the result of the analysis of the generated pathology report is provided to an external entity.

Description

Method for assisting in the generation of a pathology report from pathology images and a device using the same

Disclosed herein are a method of assisting in generation of a pathology report from a pathology image and a computing device using the same. Specifically, according to the method according to the present invention, a computing device acquires a pathological image to be read, and divides the predetermined lesion from the pathological image through a first determination unit configured to determine a predetermined lesion included in the pathological image On the other hand, the division of the anatomical object is calculated from the pathological image through a second determination unit configured to determine the anatomical object included in the pathological image, and includes between the division of the predetermined lesion and the division of the anatomical object. Analysis of the pathology report for the given lesion by reflecting at least one of a relationship, a dimension of a section of the given lesion, a number of sections of the given lesion, a dimension of a section of the anatomical object, and a number of sections of the anatomical object A result is generated, and the result of the analysis of the generated pathology report is provided to an external entity.

Pathological diagnosis through a microscope or a whole slide image of a section of a body organ including cells and tissues plays a decisive role in determining a treatment method for a patient and predicting the prognosis. These diagnoses are mainly made by considering the types of tissues and cells. Diagnosis of this pathology is one of the problems with a high degree of difficulty to the extent that there are differences between countries as well as among medical personnel within the same country. Detailed diagnosis is divided into various carcinomas and classes, and since there is a large variation in characteristics between samples, it is very difficult to find a target image that can be used as a reference for diagnosis. The difficulty of this diagnosis largely depends on the experience and intuition of the pathologist, and in actual clinical practice, it is often ambiguous for even an experienced pathologist to make a diagnosis alone. This is one of the main causes of inconsistency in diagnosis among doctors and waste of medical services such as consultation and re-reading.

In pathological diagnosis of lesions, particularly cancer, determining the extent to which cancer cells have spread is an important indicator in setting future treatment plans. This is called the stage of cancer, and the TNM method is generally used to determine this stage. In TNM, T (tumor) is the size and invasion of the primary tumor in the primary organ, N (node; lymph node) is how far it has spread from the primary tumor to the surrounding lymph nodes, and M (metastasis; metastasis) is other organs in the body. indicates whether the cancer has spread. When the cancer status is determined by the TNM method, the stages of progression to stage 1, 2, 3, and 4 can be simply summarized. Classification is also used.

In this way, in calculating the stage using the TNM method, it is necessary to calculate the number of lymph nodes containing cancer. Identifying cancer in lymph nodes and counting lymph nodes takes a lot of time, and generally there is a problem that there is not high agreement among readers.

Accordingly, in the present disclosure, a first determination unit, for example, a learned first deep neural network module, configured to determine whether a predetermined lesion is present, such as cancer, with respect to a local area of a pathological image such as a lymph node slice image with artificial intelligence technology; , a second determination unit capable of deriving the area and number of anatomical objects by finding anatomical objects such as lymph nodes in the pathological image, for example, using a second deep neural network module, anatomically corresponding to a predetermined lesion from the pathological image. The present invention proposes a method of calculating the number of objects and the number of anatomical objects that do not correspond to a predetermined lesion, and thereby assisting in the preparation of a pathology report.

KR 10-1874348 B

An object of the present invention is to make it possible to assist in the preparation of a pathology report, such as calculating the stage more conveniently and efficiently than the conventional method for determining whether a lesion exists.

In addition, the present invention is a method for overcoming the performance limitation of an algorithm for detecting an anatomical object by deforming the shape of an anatomical object due to a lesion. It is also intended to provide a user interface that allows

Specifically, an object of the present invention is to contribute to an appropriate treatment for a patient by accurately calculating a stage based on a pathological image.

The characteristic configuration of the present invention for achieving the object of the present invention as described above and for realizing the characteristic effects of the present invention to be described later is as follows.

According to one aspect of the present invention, there is provided a method of assisting in generation of a pathology report from a pathology image, the method comprising: (a) a computing device acquiring a pathology image to be read or another device interworking with the computing device assisting to acquire the pathological image; (b), by the computing device, (i) a process of segmenting a predetermined lesion from the pathological image; and (ii) performing a process of segmenting an anatomical object from the pathological image or supporting the other device to perform; (c) in the computing device, an inclusion relationship between a partition of the predetermined lesion and a partition of the anatomical object, a dimension of a partition of the predetermined lesion, the number of partitions of the predetermined lesion, a dimension of a partition of the anatomical object, and the generating an analysis result of a pathology report on the predetermined lesion by reflecting at least one of the number of sections of an anatomical object or supporting the other device to generate; and (d) providing, by the computing device, the generated pathology report analysis result to an external entity or supporting the other device to provide it.

According to another aspect of the present invention, there is also provided a computer program comprising instructions embodied for performing a method according to the present invention.

According to another aspect of the present invention, there is provided a computing device that assists in generating a pathology report from a pathology image, the computing device comprising: a communication unit configured to acquire a pathology image to be read; and (A) a process of segmenting a predetermined lesion from the pathology image; and (B) a processor that performs a process of segmenting an anatomical object from the pathological image or supports another device interlocked through the communication unit to perform, wherein the processor comprises: (i) segmentation of the predetermined lesion; By reflecting at least one of the inclusion relationship between the compartments of the anatomical object, the dimensions of the compartments of the predetermined lesion, the number of compartments of the predetermined lesion, the dimensions of the compartments of the anatomical object, and the number of compartments of the anatomical object. A process of generating an analysis result of a pathology report for a predetermined lesion, and (ii) a process of providing the generated pathology report analysis result to an external entity are performed or supported by the other device to perform.

*According to an exemplary embodiment of the present disclosure, it is possible to present certain criteria regarding counts of anatomical objects, such as counts of lymph nodes, which have conventionally been inconsistent even among readers, for example, medical professionals, so that the degree of discrepancy between readers has the effect of lowering

Therefore, according to an exemplary embodiment, it is possible to not only easily diagnose an anatomical object, but also to display the area of a predetermined lesion and the area of an anatomical object to facilitate the data conversion. There is an effect that can contribute as positive feedback to development.

Accordingly, the time of medical staff for diagnosing a specific disease is saved, and safety for more patients can be secured.

Such an effect according to an embodiment of the present invention can be applied not only to pathological images of a specific modality but also to 2D or 3D images of various types, so that the method of the present invention is dependent on a specific format or platform. of course it won't.

The accompanying drawings for use in the description of the embodiments of the present invention are only a part of the embodiments of the present invention, and for those of ordinary skill in the art (hereinafter referred to as "those of ordinary skill") to which the present invention pertains, Other drawings may be obtained based on these drawings without an effort to arrive at a novel invention.
1 is a conceptual diagram schematically illustrating an exemplary configuration of a computing device performing a method of assisting in generation of a pathology report from a pathology image (hereinafter referred to as a “pathology report generation assistance method”) according to an embodiment of the present invention. .
2 is a conceptual diagram exemplarily illustrating a hardware and software architecture of a computing device performing a pathology report generation assistance method according to an embodiment of the present invention.
3 is a flowchart exemplarily illustrating a pathology report generation assistance method according to an embodiment of the present invention.
4 is a diagram conceptually illustrating a specific deep neural network architecture as an example of a deep neural network that can be used in the first and second judgment units of the pathology report generation assistance method according to the present invention.
5A and 5B are conceptual views exemplarily illustrating user interface elements provided according to embodiments of the method for assisting pathology report generation according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description of the present invention refers to the accompanying drawings, which show by way of illustration a specific embodiment in which the present invention may be practiced, in order to clarify the objects, technical solutions and advantages of the present invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. In the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted.

Specific structural or functional descriptions of the embodiments are disclosed for purposes of illustration only, and may be changed and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosure form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various elements, these terms should be interpreted only for the purpose of distinguishing one element from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected to” another component, it may be directly connected or connected to the other component, but it should be understood that another component may exist in between.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features, number, step , it should be understood that it does not preclude the possibility of the existence or addition of an operation, a component, a part, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

The term "image", "image", "image data" or "image data" as used throughout the present description and claims refers to discrete image elements (eg, pixels in a two-dimensional image, a three-dimensional image refers to multidimensional data composed of voxels), in other words a visible object (eg displayed on a video screen) or a digital representation of that object (eg, a file corresponding to a pixel or voxel output). It is a term that refers to

For example, "image" may be an image of a subject collected according to any imaging method known in the art, such as taking a microscope image. However, the image does not necessarily have to be presented in a medical context.

For convenience of description, some exemplary image modality may be assumed, but a person skilled in the art will understand that the image formats used in various embodiments of the present invention are not limited to microscopic images.

In addition, throughout the detailed description and claims of the present disclosure, 'learning' or 'learning' is a term referring to performing machine learning through computing according to a procedure, It will be understood by those of ordinary skill in the art that it is not intended to refer to an action.

A user referred to in this disclosure does not refer to a singular, but may refer to a plurality of users for the purpose of acquiring images, particularly pathological images and related data, abundantly and superimposed, which includes training (learning) of images. ) or for the purpose of securing the integrity of the image in actual use.

Moreover, the invention encompasses all possible combinations of the embodiments indicated herein. It should be understood that various embodiments of the present invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention in relation to one embodiment. In addition, it should be understood that the position or arrangement of individual components in each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all scope equivalents as those claimed. Like reference numerals in the drawings refer to the same or similar functions throughout the various aspects.

In this specification, unless indicated otherwise or clearly contradicted by context, items referred to in the singular encompass the plural unless the context requires otherwise. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, in order to enable those skilled in the art to easily practice the present invention, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram schematically illustrating an exemplary configuration of a computing device performing a pathology report generation assistance method according to an embodiment of the present invention.

Referring to FIG. 1 , a computing device 100 according to an embodiment of the present invention includes a communication unit 110 and a processor 120 , and is directly or indirectly connected to an external computing device (not shown) through the communication unit 110 . can communicate effectively.

Specifically, the computing device 100 may include typical computer hardware (eg, a computer, a processor, a memory, storage, an input device and an output device, a device that may include other components of a conventional computing device; a router; , electronic communication devices such as switches, etc.; electronic information storage systems such as network-attached storage (NAS) and storage area networks (SANs)) and computer software (ie, enabling the computing device to operate in a specific way). to achieve the desired system performance by using a combination of instructions that make it function as The storage may include a storage device such as a hard disk and a universal serial bus (USB) memory, as well as a storage device based on a network connection such as a cloud server.

The communication unit 110 of such a computing device may transmit and receive a request and a response between the interworking computing device and the other computing device. As an example, such a request and a response are transmitted by the same transmission control protocol (TCP) session. It may be made, but is not limited thereto, and may be transmitted/received as, for example, a user datagram protocol (UDP) datagram.

Specifically, the communication unit 110 may be implemented in the form of a communication module including a communication interface. For example, the communication interface is wireless LAN (WLAN), wireless fidelity (WiFi) Direct, digital living network alliance (DLNA), wireless broadband (Wibro), world interoperability for microwave access (Wimax), high speed downlink packet access (HSDPA), etc. Including wireless Internet interface of wireless Internet interface and short-distance communication interface such as Bluetooth™, radio frequency identification (RFID), infrared data association (IrDA), ultra-wideband (UWB), ZigBee, and near field communication (NFC) can do. In addition, the communication interface may indicate any interface (eg, a wired interface) capable of performing communication with the outside.

For example, the communication unit 110 may acquire a pathological image to be read from an external device through an appropriate communication interface as described above. In addition, in a broad sense, the communication unit 110 may include or interwork with a keyboard, a mouse, other external input devices, printing devices, displays, and other external output devices for receiving commands or instructions.

In addition, the processor 120 of the computing device includes a micro processing unit (MPU), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU) or a tensor processing unit (TPU), and a cache memory. ), and may include a hardware configuration such as a data bus. In addition, the operating system may further include a software configuration of an application for performing a specific purpose.

2 is a conceptual diagram exemplarily illustrating a hardware and software architecture of a computing device performing a pathology report generation assistance method according to an embodiment of the present invention.

A brief overview of the configuration of the method and apparatus according to the present invention with reference to FIG. 2 , the computing device 100 may include an image acquisition module 210 as a component thereof. This image acquisition module 210 is configured to acquire a pathological image to which the method according to the present invention is applied. The image may be obtained from, for example, another image capturing apparatus (not shown) that is linked through the communication unit 110 .

Part or all of the acquired image may be transmitted to the lesion region partitioning module 220 and the anatomical object region partitioning module 230 , respectively. As an embodiment, the lesion region partitioning module 220 may be implemented by a processor executing predetermined instructions. Specifically, the lesion region partitioning module 220 may include a first determination unit (or a lesion determination module) configured to partition a predetermined lesion from the pathological image, for example, a trained first deep neural network module. As another embodiment compatible with this embodiment, the anatomical object region partitioning module 230 may be implemented by a processor executing predetermined instructions. Specifically, the anatomical object region partitioning module 230 may include a second determination unit (or anatomical object determination module) configured to partition an anatomical object from the pathological image, for example, a trained second deep neural network module. there is. The description of the processor implementing the lesion region partitioning module 220 and the anatomical object region partitioning module 230 described above can be applied as it is with the description of the processor 120 described with FIG. 1 , so the overlapping description is to be omitted.

Illustratively, each of the first deep neural network module and the second deep neural network module converts an image into a pixel or voxel unit through a feature value layer and feature values that generate a feature value (feature vector) by using the deep neural network model as well as the deep neural network model. It may include a classifier that classifies as The feature layer may typically be a fully-connected layer. The segmentation of the pathological image by the lesion area segmentation module 220 and the anatomical object area segmentation module 230 will be described later in detail. In addition, the description of the first deep neural network module and the second deep neural network module included in the lesion region partitioning module 220 and the anatomical object region partitioning module 230 are only exemplary descriptions for better understanding, and other embodiments It should not be construed as limiting or limiting.

At least one of the lesion region partitioning module 220 and the anatomical object region partitioning module 230 may be linked with the user interface module 240 , which may be erroneous by the first determination unit in the case of the lesion zone partitioning module 220 . This is to correct the division calculation of a predetermined lesion, while in the case of the anatomical object region division module 230 , it is to correct the division calculation of an erroneous anatomical object by the second determination unit. The user interface module 240 provides a user with predetermined user interface elements to provide a function to support the user to correct errors of the lesion area segmentation module 220 and/or the anatomical object area segmentation module 230 . can do. A user interface that can be provided by a predetermined user interface module 240 will be described later with reference to FIGS. 5A and 5B .

Next, the pathology report generation module 250 reflects the division of the predetermined lesion calculated by the lesion area division module 220 and the division of the anatomical object calculated by the anatomical object area division module 230 to generate the predetermined It is possible to generate an analysis result of a pathology report regarding the lesion. For example, when the lesion corresponds to cancer and the anatomical object is a lymph node, the stage of cancer may be determined using the number of lymph nodes and the number of lymph nodes determined to be cancerous, and the analysis result of the pathology report includes such lymph nodes may be generated to include information about the number of lymph nodes, the number of lymph nodes determined to be cancerous, the number of non-cancerous lymph nodes, the ratio therebetween, and the stage of cancer based thereon.

The analysis result of the generated pathology report may be provided to an external entity by the output module 260 . For this purpose, the output module 260 may include a display device (not shown). Here, the external entity includes a user, an administrator, and the like of the computing device 100 , but should be understood to include any subject as long as it requires an analysis result of the pathology report in addition to this. The external entity may be an artificial intelligence entity using the information, or a storage device for storing the information. When the external entity is a human, the output module 260 may provide the prediction result to the external entity through a predetermined output device, for example, a user interface displayed on a display.

In one embodiment, an update module and a learning module 270 (not shown) may be further included in addition to the components shown in FIG. 2 , and the update module and the learning module are based on a manipulation such as modification by the user, the first determination unit and updating at least one of the second determination unit.

As such, the modules shown in FIG. 2 may be implemented by, for example, the communication unit 110 or the processor 120 included in the computing device 100, or the interworking of the communication unit 110 and the processor 120. Those of ordinary skill in the art will be able to understand, and will be described in more detail with respect to its functions and effects, etc. below.

A method of assisting pathology report generation according to the present invention will now be described in detail with reference to FIG. 3 . 3 is a flowchart exemplarily illustrating a pathology report generation assistance method according to an embodiment of the present invention.

Referring to FIG. 3 , in the pathology report generation assistance method according to the present invention, first, the image acquisition module 210 implemented by the communication unit 110 of the computing device 100 acquires a pathological image to be read or the computing device 100 . and a step (S100) of supporting another device (not shown) interworking with the device to acquire the pathological image.

Next, in the pathology report generation assistance method according to the present invention, the lesion region partitioning module 220 implemented by the computing device 100 partitions a predetermined lesion from the pathological image or allows the other device to partition the predetermined lesion The anatomical object region partitioning module 230 implemented by the process (S200a; not shown) and the computing device 100 for supporting It further includes a step (S200) of performing a process (S200b; not shown) to support partitioning.

Specifically, in the process S200a, the division of the predetermined lesion may be calculated from the pathological image through the first determination unit configured to determine the predetermined lesion included in the pathological image. Also, specifically, in the process S200b, the division of the anatomical object may be calculated from the pathological image through the second determination unit configured to determine the anatomical object included in the pathological image.

4 is a diagram conceptually illustrating a specific deep neural network architecture as an example of a deep neural network that can be used in the first and second decision units of the pathology report generation assistance method according to the present invention.

In the present invention, a deep neural network may be used to partition at least one of a predetermined lesion and an anatomical object, and it is known to those skilled in the art that the effect of the deep neural network increases as the number of layers forming the deep neural network increases. However, due to the vanishing gradient problem in the conventional artificial neural network, the model weights were not properly reflected when performing backpropagation. is a way to

In particular, the deep neural network architecture illustrated in FIG. 4 is a convolutional artificial neural network (CNN) commonly referred to as Inception V3. A feature value expressed as a vector {i.e., a feature vector} uses 8x8x2048 as in the illustrated example, or global average pooling (GAP) that takes an average over an 8x8 sized area. 2048 dimensional vector can be used. Such a deep neural network model is trained to solve the problem of performing various classifications on input image patches (image fragments).

For example, partitioning a region having a specific property in an input image may be classified as one of "{included, not included}" class of pixels or voxels belonging to the partition.

Therefore, the deep neural network model for detecting a lesion such as cancer is trained to distinguish and classify pixels or voxels corresponding to the lesion from pixels or voxels that do not, and the deep neural network model for detecting an anatomical object such as lymph nodes is It may be learned to distinguish and classify pixels or voxels corresponding to anatomical objects from pixels or voxels that do not. In this case, the number of detected lymph node divisions may be determined using pixel or voxel connectivity or the like.

However, in addition to Inception V3, a person of ordinary skill in the art can classify an image input through training using training data, that is, a pathological image and corresponding labeling data, that is, information about the partition, that is, a variety of methods that can perform partitioning. It will be appreciated that there may be many kinds of deep neural network algorithms, and are not limited to the deep neural network of the specific structure illustrated in FIG. 4 . For example, there are other methods such as using a U-Net-like structure.

On the other hand, when the lesion region partitioning module 220 and the anatomical object region partitioning module 230 each use the first deep neural network module and the second deep neural network module, there may be errors in the calculation of incorrect partitions by these deep neural network modules. There is, it needs to be rectified.

In addition to the need to correct the error, it is also necessary to configure the division by the user directly. For this purpose, in an embodiment of step S200 , the computing device 100 performs Provide user interface elements for supporting at least one of input, change, and deletion of a user for at least one of a section and a section of an anatomical object in the section and process S200b, or support the other device to provide, and the user In response to manipulation of the interface element, at least one of the partition of the predetermined lesion and the partition of the anatomical object may be input, changed, or deleted, or the other device may support input, change, or deletion.

5A and 5B are conceptual diagrams exemplarily illustrating user interface elements provided according to embodiments of the method for assisting pathology report generation according to the present invention. 5A and 5B illustrate cancer as an example of a predetermined lesion, and lymph nodes as an example of an anatomical object.

Referring to FIG. 5A , a lymph node slice image 510 , a calculation result 520 of the lesion region division module, a calculation result 530 of the anatomical object region division module, etc. are exemplified, and lymph nodes 540 and 550 and lymph nodes are illustrated. Objects 560 that are not each are shown. The cancer region 540a is displayed in the calculation result 520 and the lymph node 540 of the lesion area division module, and the calculation result 530 of the anatomical object area division module, the lymph node 540 and the lymph node 550, respectively. Sections 540b and 550b are marked, each showing that it corresponds to a lymph node.

Referring to FIG. 5B , the lymph node regions 540b and 550b are normally partitioned, but the region 560b does not correspond to a lymph node and is incorrectly partitioned. There is a need. That is, erroneously partitioned ones should be deleted and those partitioned smaller than actual ones should be corrected.

In addition, the actual shape of the lymph node is similar to that of a kidney bean, so even if it is sectioned into two, it may actually be one lymph node. That is, since lymph nodes appearing in the same image may be counted as one or two depending on the reader, it is also necessary to delete the incorrectly counted section. Conversely, it may be necessary to divide a partition counted by one into two or more partitions, or to newly partition an undivided area.

To this end, the user may arbitrarily adjust the desired section by using a user interface element on the provided user interface, for example, an element 570 corresponding to a brush or an eraser or a similar tool as an example well known to those skilled in the art.

Continuing to refer to FIG. 3 , in the pathology report generation assistance method according to the present invention, the pathology report generation module 250 implemented by the computing device 100 is configured to provide a space between a section of the predetermined lesion and a section of the anatomical object. of the pathology report on the predetermined lesion by reflecting at least one of the inclusion relationship, the dimensions of the compartments of the predetermined lesion, the number of compartments of the predetermined lesion, the dimensions of the compartments of the anatomical object, and the number of compartments of the anatomical object. The method further includes generating an analysis result or supporting the other device to generate it (S300).

For example, the inclusion relationship between the compartment of the predetermined lesion and the compartment of the anatomical object may be a relationship in which a compartment of a specific predetermined lesion, eg, cancer, should be included within a compartment of a specific anatomical object, eg, a lymph node. , because it can serve as a basis for pathological reports on metastasis of cancer to lymph nodes.

Specifically, in an embodiment of step S300 , the pathology report generation module 250 analyzes the pathology report from the result of calculating the number of anatomical objects diagnosed as lesions and the number of anatomical objects that are not lesions. can be generated, that is, the analysis result of the pathology report may be generated based on a ratio in which the predetermined lesion is included in the anatomical object.

For example, referring to FIG. 5A , a partition 520 calculated by the lesion (cancer) area partitioning module from a pathological image (lymph node slice image) 510 and a partition calculated by the anatomical object (lymph node) area partitioning module ( Based on the result of 530), the pathology report generation module 250 may generate an analysis result of the pathology report, "cancer 1 out of 2 lymph nodes (1/2)".

Referring back to FIG. 3 , in the pathology report generation assistance method according to the present invention, the output module 260 implemented by the processor 120 provides the generated pathology report analysis result to an external entity or The method further includes supporting the other device to provide the pathology report analysis result to the external entity (S400).

On the other hand, prior to performing the pathology report generation assistance method according to the present invention, it is necessary to go through a step (S050) in which the utilized first and second determination units, for example, deep neural network modules, are trained in advance. An update and learning module or a learning module 270 (not shown) for this purpose may be executed by the processor 120 .

For example, for the learning of the first judgment unit, the learning module 270 uses, as training data, pathological images of a plurality of subjects and labeling data indicating (marking) predetermined lesions appearing in the pathological images. In addition, for learning of the second determination unit, the learning module 270 may use, as training data, pathological images of a plurality of subjects and labeling data indicating anatomical objects appearing in the pathological images. It will be appreciated that training data for this may be substantially prepared by the user interface described above.

Although the components shown in the accompanying drawings in the present disclosure are exemplified as being realized in one computing device for convenience of description, the computing device 100 for performing the method of the present invention may be configured as a plurality of devices interworking with each other. It is to be understood that this may be encompassed by the claims appended to this disclosure. Therefore, it is obvious that each step of the method of the present invention described above can be performed by one computing device directly or by supporting another computing device linked to the one computing device to perform the one computing device.

As described above, the pathology report generation assistance method according to the present invention can generate the analysis result of the pathology report based on the identified predetermined lesion and anatomical object, and can add, edit, If the deleted information is used again as data for updating or additional learning for the first and second determination units, there is an advantage in that these determination units perform more accurate division, so the present invention for taking these advantages In the pathology report generation assistance method according to The method may further include updating at least one of the second determination units trained to determine the included anatomical object or supporting the other device to update (S500). At this time, since data not taken into account during previous training or learning is additionally considered, and errors made during previous learning can be corrected, the accuracy of the first and second determination units is improved. It has the advantage of continuously improving its performance.

Here, the subject performing the manipulation on the user interface element may be an external entity such as the medical professional.

As such, the present invention has the effect of more conveniently and efficiently counting lesions and related anatomical objects across all the above-described embodiments.

The advantage of the technology described herein as the above embodiments is that it can greatly relieve the burden of medical professionals in a busy medical environment who have to make accurate decisions or predictions based on a lot of diagnostic data every day, while anatomy such as counting lymph nodes The point is that it is possible to suggest a certain standard for the count of enemy objects, thereby reducing the degree of inconsistency among readers and making more accurate judgments about related diseases. In other words, it is possible to promote the data conversion by displaying the area of a predetermined lesion and the area of an anatomical object, thereby contributing as positive feedback to the development of medical-related artificial intelligence technology.

Above, based on the description of various embodiments of the present disclosure, those of ordinary skill in the art will recognize that the method and/or processes of the present invention, and the steps thereof, are hardware, software, or hardware and software suitable for a specific application. It can be clearly understood that they can be realized in any combination. The hardware may include general purpose computers and/or dedicated computing devices or specific computing devices or special features or components of specific computing devices. The processes may include one or more processors, eg, microprocessors, controllers, eg, microcontrollers, embedded microcontrollers, microcomputers, arithmetic logic units (ALUs), digital signal processors, with internal and/or external memory. , for example, by a programmable digital signal processor or other programmable device. In addition, or as an alternative, the processes may include application specific integrated circuits (ASICs), programmable gate arrays, such as field programmable gate arrays (FPGAs), programmable logic units (PLUs) or programmable array logic. (Programmable Array Logic; PAL) or any other device capable of executing and responding to instructions, any other device or combination of devices that can be configured to process electronic signals. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more machine-readable recording media.

Moreover, the objects of the technical solution of the present invention or parts contributing to the prior arts may be implemented in the form of program instructions that can be executed through various computer components and recorded in a machine-readable medium. The machine-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the machine-readable recording medium may be specially designed and configured for the embodiment, or may be known and used by those skilled in the art of computer software. Examples of the machine-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs, DVDs, and Blu-rays, and floppy disks. ), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include any one of the devices described above, as well as heterogeneous combinations of processors, processor architectures, or combinations of different hardware and software, or stored and compiled or interpreted for execution on a machine capable of executing any other program instructions. machine code, which may be created using a structured programming language such as C, an object-oriented programming language such as C++ This includes not only bytecode, but also high-level language code that can be executed by a computer using an interpreter or the like.

Accordingly, in one aspect according to the present invention, when the above-described method and combinations thereof are performed by one or more computing devices, the methods and combinations of methods may be implemented as executable code for performing respective steps. In another aspect, the method may be implemented as systems performing the steps, the methods may be distributed in various ways across devices or all functions may be integrated into one dedicated, standalone device or other hardware. In another aspect, the means for performing the steps associated with the processes described above may include any of the hardware and/or software described above. All such sequential combinations and combinations are intended to fall within the scope of this disclosure.

For example, the hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa. The hardware device may include a processor, such as an MPU, CPU, GPU, TPU, coupled with a memory such as ROM/RAM for storing program instructions and configured to execute instructions stored in the memory, an external device and a signal It may include a communication unit that can send and receive. In addition, the hardware device may include a keyboard, a mouse, and other external input devices for receiving commands written by developers.

In the above, the present invention has been described with specific matters such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments, Those of ordinary skill in the art to which the present invention pertains can devise various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims appended to the present disclosure, but also all modifications equivalently or equivalently to the claims attached to the present disclosure are the spirit of the present invention will fall into the category of For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Such equivalent or equivalent modifications will include, for example, logically equivalent methods capable of producing the same results as practiced by the methods according to the present invention, the spirit and spirit of the present invention. The scope should not be limited by the above examples, and should be understood in the broadest sense permitted by law.

Claims (10)

A method of assisting in generating a pathology report from pathology images, comprising:
acquiring, by the computing device, a pathological image to be read;
performing, by the computing device, segmentation of a predetermined lesion and segmentation of an anatomical object from the pathological image based on a deep neural network;
The computing device determines, by the computing device, an inclusion relationship between a partition of the predetermined lesion displayed in the pathological image and a partition of the anatomical object displayed in the pathological image, a dimension of a partition of the predetermined lesion, the number of partitions of the predetermined lesion, and the anatomical generating an analysis result of a pathology report on the predetermined lesion by reflecting at least one of a dimension of a compartment of an object and a number of compartments of the anatomical object; and
providing, by the computing device, an analysis result of the generated pathology report to an external entity
including,
The analysis result of the pathology report is,
information on the stage of the predetermined lesion determined using the number of divisions of the predetermined lesion and the number of divisions of the anatomical object; and
including a ratio in which the predetermined lesion is included in the anatomical object based on an inclusion relationship between the partitions of the predetermined lesion and the partitions of the anatomical object,
Auxiliary method for generating pathology reports. delete The method of claim 1,
providing, by the computing device, at least one user interface element for supporting at least one of a user's input, change, and deletion of the partition of the predetermined lesion; and
In response to the manipulation of the user interface element, the computing device inputting, changing, or deleting a section of the predetermined lesion.
An auxiliary method for generating a pathology report further comprising: delete The method of claim 1,
providing, by the computing device, at least one user interface element for supporting at least one of a user's input, change, and deletion of the partition of the anatomical object; and
in response to manipulation of the user interface element, inputting, changing, or deleting, by the computing device, a section of the anatomical object;
An auxiliary method for generating a pathology report further comprising: delete A computer program stored on a medium comprising instructions implemented to cause a computing device to perform the method of any one of claims 1, 3 and 5. A computing device that assists in generating a pathology report from a pathology image, comprising:
a communication unit that acquires a pathological image to be read; and
A segmentation of a predetermined lesion and a segmentation of an anatomical object are performed from the pathology image, and an inclusion relationship between the segmentation of the prescribed lesion shown in the pathological image and the segment of the anatomical object shown in the pathology image, the segmentation of the prescribed lesion a process of generating an analysis result of a pathology report regarding the predetermined lesion by reflecting at least one of a dimension, a number of compartments of the predetermined lesion, a dimension of a compartment of the anatomical object, and a number of compartments of the anatomical object; and A processor performing a process of providing an analysis result of the generated pathology report to an external entity
including,
The analysis result of the pathology report is,
information on the stage of the predetermined lesion determined using the number of divisions of the predetermined lesion and the number of divisions of the anatomical object; and
including a ratio in which the predetermined lesion is included in the anatomical object based on an inclusion relationship between the partitions of the predetermined lesion and the partitions of the anatomical object,
Pathology report generation device.
delete delete

Images