KR20230023568A - A method and an apparatus for outputting information related to a pathological slide image - Google Patents

Abstract

According to one aspect of the present invention, a computing device comprises at least one memory and at least one processor. The processor analyzes a pathological slide image to generate quantitative information of at least one cell included in a region of interest of the pathological slide image, analyzes the pathological slide image to generate qualitative information of at least one tissue included in the qualitative information, and controls a display device to output at least one between the quantitative information and the qualitative information on the pathological slide image in accordance with an operation of a user.

Description

A method and an apparatus for outputting information related to a pathological slide image {A method and an apparatus for outputting information related to a pathological slide image}

The present disclosure relates to a method and apparatus for outputting information related to a pathology slide image.

The field of digital pathology is a field of obtaining histological information or predicting a prognosis of a subject using a whole slide image generated by scanning a pathological slide image.

A pathology slide image can be obtained from a stained tissue sample of a subject. For example, tissue samples can be tested for hematoxylin and eosin, trichrome, periodic acid schiff, autoradiogrphy, enzyme histochemistry, It can be stained by various staining methods such as immuno-fluorescence and immunohistochemistry. Stained tissue samples can be used for histology and biopsy evaluation, thereby providing a basis for determining whether to proceed to molecular profile analysis to understand disease status.

In general, a pathology slide image is displayed together with tissue segmentation results and cell detection results. However, since the user performs reading while enlarging or reducing the pathology slide image, information output together with the pathology slide image must be changed in response to the user's manipulation.

An object of the present invention is to provide a method and apparatus for outputting information related to a pathology slide image. In addition, it is to provide a computer-readable recording medium recording a program for executing the method on a computer. The technical problem to be solved is not limited to the technical problems described above, and other technical problems may exist.

A computing device according to one aspect includes at least one memory; and at least one processor, wherein the processor analyzes the pathology slide image to generate quantitative information of at least one cell included in a region of interest of the pathology slide image, and generates the pathology slide image a display device to analyze and generate qualitative information of at least one tissue included in the pathology slide image, and to output at least one of the quantitative information and qualitative information on the pathology slide image according to a user's manipulation. Control.

A method of outputting information on a pathology slide image according to another aspect includes generating quantitative information of at least one cell included in a region of interest of the pathology slide image by analyzing the pathology slide image; generating qualitative information of at least one tissue included in the pathology slide image by analyzing the pathology slide image; and outputting at least one of the quantitative information and the qualitative information on the pathology slide image according to a user's manipulation.

A computer-readable recording medium according to another aspect includes a recording medium recording a program for executing the above-described method on a computer.

1 is a diagram for explaining an example of a system for outputting information on a pathology slide image according to an exemplary embodiment.
2 is a block diagram of a system and network for preparing, processing, and reviewing slide images of tissue specimens using machine learning according to one embodiment.
3A is a configuration diagram illustrating an example of a user terminal according to an embodiment.
3B is a configuration diagram illustrating an example of a server according to an embodiment.
4 is a flowchart illustrating an example in which a processor outputs information about a pathology slide image, according to an exemplary embodiment.
5 is a flowchart illustrating an example in which a processor generates quantitative information according to an exemplary embodiment.
6 is a flowchart illustrating another example in which a processor generates quantitative information according to an exemplary embodiment.
7 is a diagram for explaining an example in which a processor defines a kernel according to an embodiment.
8 is a diagram for explaining an example of a convolution operation according to an embodiment.
9 is a flowchart illustrating an example in which a processor generates qualitative information according to an exemplary embodiment.
10 is a flowchart illustrating another example in which a processor generates qualitative information according to an exemplary embodiment.
11 and 12 are diagrams illustrating examples of outputting quantitative information according to an exemplary embodiment.
13 and 14 are diagrams illustrating examples of outputting quantitative information and qualitative information according to an embodiment.
15 is a flowchart illustrating an example in which a processor outputs information about a pathology slide image, according to an exemplary embodiment.

The terms used in the embodiments have been selected as general terms that are currently widely used as much as possible, but they may vary according to the intention of a person skilled in the art or a precedent, the emergence of new technologies, and the like. In addition, in a specific case, there are also terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the corresponding description. Therefore, terms used in the specification should be defined based on the meaning of the term and the overall content of the specification, not simply the name of the term.

When it is said that a certain part "includes" a certain component throughout the specification, it means that it may further include other components without excluding other components unless otherwise stated. In addition, terms such as “~ unit” and “~ module” described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software.

Also, terms including ordinal numbers such as “first” or “second” used in the specification may be used to describe various components, but the components should not be limited by the terms. The terms may be used for the purpose of distinguishing one component from another.

According to an embodiment, a "pathology slide image" may refer to an image obtained by taking a pathology slide fixed and stained through a series of chemical treatment processes for tissue, etc., removed from a human body. In addition, the pathology slide image may refer to a whole slide image (WSI) including a high-resolution image of the entire slide, and may refer to a part of the entire slide image, for example, one or more patches. . For example, a pathology slide image may refer to a digital image photographed or scanned through a scanning device (eg, a digital scanner, etc.), and may refer to a specific protein, cell, tissue, and/or structure ( structure) may be included. Also, the pathology image may include one or more patches, and histological information may be applied (eg, tagging) to the one or more patches through an annotation operation.

According to one embodiment, "medical information" may refer to any medically meaningful information that can be extracted from a medical image, for example, the area, location, size, and location of tumor cells in a medical image, It may include, but is not limited to, diagnostic information, information associated with the subject's possibility of developing cancer, and/or medical conclusions associated with cancer treatment. In addition, medical information may include not only quantified numerical values obtained from medical images, but also information visualizing numerical values, prediction information according to numerical values, image information, statistical information, and the like. The medical information generated in this way may be provided to a user terminal or output or transferred to a display device and displayed.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, embodiments may be implemented in many different forms and are not limited to the examples described herein.

1 is a diagram for explaining an example of a system for outputting information on a pathology slide image according to an exemplary embodiment.

Referring to FIG. 1 , a system 1 includes a user terminal 10 and a server 20 . For example, the user terminal 10 and the server 20 may be connected in a wired or wireless communication method to transmit and receive data (eg, image data, etc.) between them.

For convenience of description, FIG. 1 shows that the system 1 includes the user terminal 10 and the server 20, but is not limited thereto. For example, the system 1 may include other external devices (not shown), and the operation of the user terminal 10 and the server 20 to be described below may be performed by a single device (eg, the user terminal 10). Alternatively, it may be implemented by the server 20) or more devices.

The user terminal 10 may be a computing device including a display device and a device (eg, a keyboard, mouse, etc.) that receives a user input, and includes a memory and a processor. For example, the user terminal 10 may include a notebook PC, a desktop PC, a laptop, a tablet computer, a smart phone, and the like, but is not limited thereto.

The server 20 may be a device that communicates with an external device (not shown) including the user terminal 10 . As an example, the server 20 includes a pathology slide image, a bitmap image corresponding to the pathology slide image, and information generated by analyzing the pathology slide image (eg, including quantitative information, qualitative information, etc.) It may be a device for storing various data. Alternatively, the server 20 may be a computing device including a memory and a processor and having its own computing capability. When the server 20 is a computing device, the server 20 may perform at least some of the operations of the user terminal 10 to be described later with reference to FIGS. 1 to 16 . For example, the server 20 may be a cloud server, but is not limited thereto.

The user terminal 10 outputs a pathology slide image and/or an image 40 representing at least one of quantitative information and qualitative information generated through analysis of the pathology slide.

Here, the quantitative information includes numerical information on various aspects of at least one cell or tissue included in a region of interest set on the pathology slide image. For example, the quantitative information includes the number of total cells (or specific types of cells) included in the region of interest, the density of total cells, the density of specific types of cells, the ratio between different cell types, and a specific number among all cells. It may include the ratio of cells of a type, the area of total tissue (or a specific type of tissue), the ratio between different tissue types, the ratio of a specific type of tissue out of the total tissue area, and the like. Quantitative information may include information in text form or graph form.

Meanwhile, the qualitative information includes information about the state of at least one tissue or cell included in the pathology slide image.

As an example, the qualitative information may include characteristics of at least one tissue included in the pathology slide image. The qualitative information may include information representing characteristics of a tissue in corresponding colors. For example, the characteristics of the tissue may indicate whether a ratio of PD-L1 positive cancer cells to PD-L1 negative cancer cells within a predetermined size circle centered on each pixel is equal to or greater than a predetermined threshold value. For example, tissue characteristics may indicate information about an immune phenotype (IP) corresponding to a partial region of a pathology slide image. For example, depending on which range a value calculated based on at least one of the density of immune cells in a cancer region and/or the density of immune cells in a cancer stroma is included, the IP may be classified as immune inflamed, immune excluded ( immune exclusion), or at least one of the immune desert.

As another example, the qualitative information may include at least one tissue type included in the pathology slide image. Qualitative information may include information representing tissue types in corresponding colors. For example, the type of tissue may include one of cancer, cancer stroma, necrosis, or background.

As another example, the qualitative information may include a type or characteristic of at least one cell included in the pathology slide image. Qualitative information may include information representing cell types in corresponding colors. For example, the type of cell may include tumor cells, immune cells such as lymphocytes and macrophages, or stromal cells such as fibroblasts and adipocytes constituting the matrix around the tumor. (stromal cell) may represent at least one.

In addition, the qualitative information may include information representing cell characteristics in corresponding colors. For example, cell characteristics may include information on whether a cell expresses a specific protein. For example, qualitative information indicates that the cell is PD-L1 positive tumor cell (PD-L1 (programmed death-ligand 1) positive tumor cell), PD-L1 negative tumor cell (PD-L1 negative tumor cell), PD-L1 Information on whether the cell is a PD-L1 positive lymphocyte or a PD-L1 positive macrophage may be included.

The pathology slide image may refer to an image obtained by taking a pathology slide fixed and stained through a series of chemical treatments in order to observe a tissue removed from the human body under a microscope. As an example, a pathology slide image may refer to a whole slide image including a high-resolution image of the entire slide. As another example, a pathology slide image may refer to a portion of such a high-resolution, full-slide image.

Meanwhile, the pathology slide image may refer to a patch region divided in patch units from an entire slide image. For example, a patch may have a certain area size. Alternatively, a patch may refer to a region including each of included objects within the entire slide.

Also, the pathology slide image may refer to a digital image taken using a microscope, and may include information about cells, tissues, and/or structures in the human body.

The user terminal 10 analyzes the pathology slide image to generate quantitative information of at least one cell included in the ROI of the pathology slide image and qualitative information of at least one tissue included in the pathology slide image. At this time, the user terminal 10 may generate quantitative information and qualitative information by analyzing the pathology slide image by itself, and the server 20 may generate quantitative and qualitative information by analyzing the pathology slide image. ) may be received from

In general, information output on a pathology slide image includes tissue segmentation results, cell detection results, and the like. In order to effectively deliver these results to the user 30, it may be necessary to output them together with the pathology slide image through a predetermined process. For example, it is preferable that information output on a display device is adaptively changed as the user manipulates the pathology slide image (eg, changing the output position, enlarging/reducing the image, etc.).

According to an embodiment, the user terminal 10 outputs quantitative information or qualitative information corresponding to a region of interest set according to the user's 30 manipulation or at least a portion of the pathology slide image output on the display device. Accordingly, the user 30 can read the image output on the display device and provide effective assistance in diagnosing the subject.

Meanwhile, for convenience of description, the user terminal 10 analyzes the pathology slide image throughout the specification to generate quantitative information and qualitative information, and outputs at least one of the quantitative information and qualitative information according to a user's operation. Although described as such, it is not limited thereto. For example, at least some of the operations performed by the user terminal 10 may be performed by the server 20 .

In other words, at least some of the operations of the user terminal 10 described with reference to FIGS. 1 to 15 may be performed by the server 20 . As an example, the server 20 may generate various information (eg, quantitative information and qualitative information) related to tissues and cells by analyzing a pathology slide image. And, the server 20 may transmit the generated information to the user terminal 10 . As another example, the server 20 may generate various information related to tissues and cells, and transmit information to be output to the display device according to a user's manipulation among the generated information to the user terminal 10 . However, the operation of the server 20 is not limited to the above.

2 is a block diagram of a system and network for preparing, processing, and reviewing slide images of tissue specimens using machine learning according to one embodiment.

Referring to FIG. 2, the system 2 includes user terminals 11 and 12, a scanner 50, an image management system 61, an AI-based biomarker analysis system 62, a laboratory information system 63, and a server ( 70). In addition, the components 11, 12, 50, 61, 62, 63, and 70 included in the system 2 may be connected to each other through a network 80. For example, the network 80 may be a network in which components 11, 12, 50, 61, 62, 63, and 70 may be connected to each other through wired or wireless communication. For example, the system 2 shown in FIG. 2 may include a network that can be connected to servers in hospitals, laboratories, laboratories, and/or user terminals of doctors or researchers.

According to various embodiments of the present disclosure, a method to be described later with reference to FIGS. 3A to 15 includes user terminals 11 and 12, an image management system 61, an AI-based biomarker analysis system 62, and a laboratory information system. (63) and/or hospital or laboratory server 70.

The scanner 50 may acquire a digitized image from a tissue sample slide generated using a tissue sample of the subject 90 . For example, the scanner 50, the user terminals 11 and 12, the image management system 61, the AI-based biomarker analysis system 62, the laboratory information system 63 and / or the hospital or laboratory server 70 Is connected to a network 80 such as the Internet through one or more computers, servers, and/or mobile devices, respectively, or communicates with the user 30 and/or the subject 90 through one or more computers and/or mobile devices. can do.

The user terminals 11 and 12, the image management system 61, the AI-based biomarker analysis system 62, the laboratory information system 63, and/or the hospital or laboratory server 70 are configured to monitor the tissue of one or more subjects 90. A sample, tissue sample slide, digitized images of a tissue sample slide, or any combination thereof may be created or otherwise obtained from another device. In addition, the user terminals 11 and 12, the image management system 61, the AI-based biomarker analysis system 62, and the laboratory information system 63 determine the subject's age, medical history, cancer treatment history, family history, Any combination of subject-specific information such as past biopsy records or disease information of the subject 90 may be obtained.

The scanner 50, the user terminals 11 and 12, the image management system 61, the laboratory information system 63, and/or the hospital or laboratory server 70 transmit digitized slide images and/or data through the network 80. Alternatively, the subject-specific information may be transmitted to the AI-based biomarker analysis system 62. The AI-based biomarker analysis system 62 is configured from at least one of the scanner 50, the user terminals 11 and 12, the image management system 61, the laboratory information system 63, and/or the hospital or laboratory server 70. It may include one or more storage devices (not shown) for storing received images and data. In addition, AI-based biomarker analysis system 62 may include an AI model repository that stores AI models trained to process received images and data. For example, the AI-based biomarker analysis system 62 may obtain information about at least one cell, information about at least one region, information related to a biomarker, medical diagnosis information, and/or information about at least one cell from a slide image of the subject 90. It may include an AI model learned and trained to predict at least one of the medical treatment information.

The scanner 50, the user terminals 11 and 12, the AI-based biomarker analysis system 62, the laboratory information system 63, and/or the hospital or laboratory server 70 transmit digitized slide images through the network 80. , subject-specific information and/or a result of analyzing the digitized slide image may be transmitted to the image management system 61 . The image management system 61 may include a storage for storing received images and a storage for storing analysis results.

In addition, according to various embodiments of the present disclosure, information about at least one cell, information about at least one region, information related to a biomarker, medical diagnosis information, and/or medical treatment from a slide image of the subject 90 The AI model learned and trained to predict at least one of the pieces of information may be stored and operated in the user terminals 11 and 12 and/or the image management system 61 .

According to various embodiments of the present disclosure, a slide image processing method, a subject information processing method, a subject group selection method, a clinical trial design method, a biomarker selection method, and/or a reference value setting method for a specific biomarker are AI-based biomarkers. It may be performed in the user terminals 11 and 12, the image management system 61, the laboratory information system 63, and/or the hospital or laboratory server 70, as well as the marker analysis system 62.

3A is a configuration diagram illustrating an example of a user terminal according to an embodiment.

Referring to FIG. 3A , the user terminal 100 includes a processor 110, a memory 120, an input/output interface 130 and a communication module 140. For convenience of description, only components related to the present invention are shown in FIG. 3A. Accordingly, in addition to the components shown in FIG. 3A , other general-purpose components may be further included in the user terminal 100 . In addition, the processor 110, the memory 120, the input/output interface 130, and the communication module 140 shown in FIG. 3A may be implemented as independent devices with ordinary knowledge in the technical field related to the present invention. be self-evident

The processor 110 may process commands of a computer program by performing basic arithmetic, logic, and input/output operations. Here, the command may be provided from the memory 120 or an external device (eg, the server 20). Also, the processor 110 may overall control operations of other components included in the user terminal 100 .

The processor 110 analyzes the pathology slide image and generates quantitative information of at least one cell included in the ROI of the pathology slide image. For example, the processor 110 may analyze the pathology slide image using a predetermined image processing technique or analyze the pathology slide image using a machine learning model.

As an example, the processor 110 generates a data structure using coordinate information corresponding to each of the cells included in the pathology slide image, and corresponds to at least one cell included in the region of interest from the data structure. coordinate information can be checked. Here, the data structure may be an efficient tree for searching cells in a pathology slide image, and may be implemented as, for example, a K-D tree or a ball tree, but is not limited thereto. Also, the processor 110 may generate quantitative information of cells based on the number corresponding to the coordinate information. An example of generating quantitative information by the processor 110 using a data structure will be described later with reference to FIG. 5 .

As another example, the processor 110 may define a kernel based on the region of interest and generate quantitative information through a pathology slide image and a convolution operation. Here, at least one kernel may be defined according to the type of quantitative information. An example in which the processor 110 generates quantitative information through a convolution operation will be described later with reference to FIGS. 6 to 8 .

The processor 110 analyzes the pathology slide image to generate qualitative information on at least one cell or tissue included in the pathology slide image. For example, the processor 110 may identify a label corresponding to a corresponding cell or tissue based on an analysis result of at least one cell or tissue. Also, the processor 110 may convert a bitmap image corresponding to the pathology slide image based on the label.

For example, the processor 110 divides the bitmap image into a plurality of tiles based on resources of a device (eg, the user terminal 100) that converts the bitmap image, and identifies the labels. Each of the tiles may be transformed based on . Also, the processor 110 may complete conversion of the bitmap image by combining the converted tiles. An example of how the processor 110 generates qualitative information will be described later with reference to FIGS. 9 and 10 .

The processor 110 outputs at least one of quantitative information and qualitative information on the pathology slide image according to a user's manipulation. For example, the processor 110 may change and output at least one of static information and qualitative information based on the output portion or output magnification of the pathology slide image changed according to a user's manipulation. In addition, the processor 110 may change and output at least one of quantitative information and qualitative information by comparing an output magnification of the pathology slide image changed according to a user manipulation with a threshold magnification. An example in which the processor 110 controls the output of the display device based on the user's manipulation will be described later with reference to FIGS. 13 and 14 .

The processor 110 may be implemented as an array of a plurality of logic gates, or may be implemented as a combination of a general-purpose microprocessor and a memory in which programs executable by the microprocessor are stored. For example, the processor 110 may include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, the processor 110 may include an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), or the like. For example, processor 110 may be a combination of a digital signal processor (DSP) and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors coupled with a digital signal processor (DSP) core, or any other such combination. It may also refer to a combination of processing devices, such as a combination of configurations.

Memory 120 may include any non-transitory computer readable recording medium. As an example, the memory 120 may include a permanent mass storage device such as random access memory (RAM), read only memory (ROM), a disk drive, a solid state drive (SSD), and flash memory. device) may be included. As another example, a non-perishable mass storage device such as a ROM, SSD, flash memory, or disk drive may be a separate permanent storage device from memory. In addition, the memory 210 may store an operating system (OS) and at least one program code (eg, a code for the processor 110 to perform an operation to be described later with reference to FIGS. 4 to 15 ).

These software components may be loaded from a computer-readable recording medium separate from the memory 120 . The recording medium readable by such a separate computer may be a recording medium that can be directly connected to the user terminal 100, and for example, a floppy drive, a disk, a tape, a DVD/CD-ROM drive, a memory card, etc. A readable recording medium may be included. Alternatively, the software components may be loaded into the memory 120 through the communication module 140 rather than a computer-readable recording medium. For example, at least one program is a computer program installed by files provided by developers or a file distribution system that distributes application installation files through the communication module 140 (eg, FIGS. 4 to 15 It may be loaded into the memory 120 based on a computer program for the processor 110 to perform an operation to be described later with reference to .

The input/output interface 130 may be a means for interface with a device (eg, keyboard, mouse, etc.) for input or output that may be connected to the user terminal 100 or included in the user terminal 100 . In FIG. 3A , the input/output interface 130 is shown as a separate component from the processor 110, but is not limited thereto, and the input/output interface 130 may be included in the processor 110.

The communication module 140 may provide a configuration or function for the server 20 and the user terminal 100 to communicate with each other through a network. Also, the communication module 140 may provide a configuration or function for the user terminal 100 to communicate with other external devices. For example, control signals, commands, data, etc. provided under the control of the processor 110 may be transmitted to the server 20 and/or an external device via the communication module 140 and a network.

Meanwhile, although not shown in FIG. 3A , the user terminal 100 may further include a display device. Alternatively, the user terminal 100 may be connected to an independent display device through a wired or wireless communication method to transmit and receive data between them. For example, a pathology slide image, quantitative information, qualitative information, etc. may be provided to the user 30 through a display device.

3B is a configuration diagram illustrating an example of a server according to an embodiment.

Referring to FIG. 3B , the server 20 includes a processor 210 , a memory 220 and a communication module 230 . For convenience of description, only components related to the present invention are shown in FIG. 3B. Accordingly, other general-purpose components may be further included in the server 200 in addition to the components shown in FIG. 3B. In addition, it is apparent to those skilled in the art that the processor 210, memory 220, and communication module 230 shown in FIG. 3B may be implemented as independent devices.

The processor 210 may acquire a pathology slide image from at least one of the internal memory 220, an external memory (not shown), the user terminal 10, or an external device. The processor 210 analyzes the pathology slide image to generate quantitative information on at least one cell included in the region of interest, to generate qualitative information on at least one tissue included in the pathology slide image, or according to a user's manipulation of the pathology slide. At least one of quantitative information and qualitative information is output on the image. In other words, at least one of the operations of the processor 110 described above with reference to FIG. 3A may be performed by the processor 210 . In this case, the user terminal 10 may output the information transmitted from the server 20 through the display device.

Meanwhile, since an implementation example of the processor 210 is the same as the implementation example of the processor 110 described above with reference to FIG. 3A, a detailed description thereof will be omitted.

The memory 220 may store various data such as pathology slide images and data generated according to the operation of the processor 210 . Also, an operating system (OS) and at least one program (eg, a program required for the processor 210 to operate) may be stored in the memory 220 .

Meanwhile, since an implementation example of the memory 220 is the same as the implementation example of the memory 220 described above with reference to FIG. 3A, a detailed description thereof will be omitted.

The communication module 230 may provide a configuration or function for the server 200 and the user terminal 100 to communicate with each other through a network. Also, the communication module 140 may provide a configuration or function for the server 200 to communicate with other external devices. For example, control signals, commands, data, etc. provided under the control of the processor 210 may be transmitted to the user terminal 100 and/or an external device via the communication module 230 and a network.

4 is a flowchart illustrating an example in which a processor outputs information about a pathology slide image, according to an exemplary embodiment.

Referring to FIG. 4 , a method of outputting information on a pathology slide image includes steps sequentially processed by the user terminal 10 or 100 or the processor 110 shown in FIGS. 1 to 3B. Therefore, even if the content is omitted below, the information described above regarding the user terminals 10 and 100 or the processor 110 shown in FIGS. can be applied

Also, as described above with reference to FIGS. 1 to 3B , at least one of the steps of the flowchart shown in FIG. 4 may be processed by the server 20 or 200 or the processor 210 .

In step 410, the processor 110 analyzes the pathology slide image to generate quantitative information of at least one cell included in the ROI of the pathology slide image.

As an example, the processor 110 may analyze the pathology slide image using a predetermined image processing technique, detect regions corresponding to tissues from the pathology slide image, and separate layers representing the tissues. As another example, the processor 110 may perform detection of regions corresponding to tissues and separation of layers representing tissues from a pathology slide image by using a machine learning model. In this case, the machine learning model detects regions corresponding to tissues in the reference pathology slide images using training data including a plurality of reference pathology slide images and a plurality of reference label information, and creates layers representing the tissues. Can be taught to separate. Accordingly, the processor 110 may obtain various information from the pathology slide image.

As an example, the processor 110 may obtain information about a region. The area may be one of a cancer area, a cancer stroma area, a necrosis area, or a background area. Here, the background area may include an area representing biological noise and/or an area representing technical noise. For example, a region representing biological noise may include a normal region, and a region representing technical noise may include a degraded region.

As another example, the processor 110 may obtain information about cells. The cells may be either tumor cells, lymphocytes cells or other cells. For example, the processor 110 may include cells expressing a specific protein (e.g. PD-L1 positive cells), cells not expressing a specific protein (e.g. PD-L1 negative cells), and intratumoral infiltrating lymphocytes (Intratumoral Tumor- Information on infiltrating lymphocytes cells or stromal tumor-infiltrating lymphocytes cells in the stroma can be obtained.

As another example, processor 110 may obtain other information of a cell, region, or tissue. For example, the processor 110 determines nuclei size, cell density, number of cells, cell cluster, cell heterogeneity, and spatial distances between cells. cells), interactions between cells, and the like can be confirmed.

As another example, the processor 110 may obtain information 424 about a tumor microenvironment. Here, the tumor microenvironment means information about the environment surrounding the tumor, for example, blood vessels, immune cells, fibroblasts, signaling molecules, and extracellular matrix around the tumor. ECM) includes information such as existence, location, type, quantity, and area.

The processor 110 generates quantitative information of at least one cell included in the ROI based on the analysis result of the pathology slide image.

Here, the region of interest may be a region requiring observation by the user 30 in the pathology slide image, a selected region in the pathology slide image, or an entire cell region in the pathology slide image. For example, the region of interest may be, but is not limited to, an important region to be observed in pathology interpretation of a disease. For example, when diagnosis of lung cancer is required, a region in which lung cancer cells exist within a pathology slide image may be a region of interest, and when mitosis is evaluated in breast cancer, mitosis is expected to be high in the pathology slide image. A region can be a region of interest. For example, the quantitative information may include at least one of a tumor proportion score (TPS), a combined positive score (CPS), and a density of intratumoral tumor-infiltrating lymphocytes within the region of interest.

For example, the region of interest may be set according to the manipulation of the user 30 . In addition, an arbitrary region of interest may be set on the pathology slide image output on the display device and then adjusted by the user 30 . Meanwhile, the region of interest may be displayed in a circular or rectangular shape on the pathology slide image, but is not limited thereto.

As an example, the processor 110 may generate quantitative information about a region of interest using a data structure configured based on coordinate information of cells. As another example, the processor 110 may generate quantitative information through a convolution operation. Hereinafter, examples in which the processor 110 generates quantitative information will be described with reference to FIGS. 5 to 8 .

5 is a flowchart illustrating an example in which a processor generates quantitative information according to an exemplary embodiment.

In the pathology slide image, there may be coordinate values corresponding to each of hundreds of thousands of cells. In order for the user 30 to read the pathology slide image effectively, the processor 110 may generate quantitative information about how many cells of a specific type are included in the region of interest.

In step 510, the processor 110 acquires a data structure based on coordinate information corresponding to each of the cells included in the pathology slide image. For example, the processor 110 may fetch a data structure generated and stored in advance using coordinate information from a memory, or may directly create a data structure using coordinate information.

As an example, the processor 110 may load a data structure generated based on coordinate values corresponding to each of all cells expressed in the pathology slide image. According to the manipulation of the user 30, the size or location of the region of interest may be changed in real time. In order for the processor 110 to provide quantitative information in real time according to a change in the size or position of the region of interest, it must be able to efficiently search cells at a specific location from hundreds of thousands of cells. Therefore, the data structure acquired through step 510 may be a structure with high efficiency in searching for data rather than a structure with high efficiency in inserting or removing data.

For example, the data structure may be an efficient tree for searching for cells in a pathology slide image, and may be implemented as, for example, a K-D tree or a ball tree, but is not limited thereto. This data structure can also be used when visualizing the positions of cells within an image area output to a display device.

As another example, the processor 110 checks coordinate values corresponding to each of all cells represented in the pathology slide image. Then, the processor 110 creates a data structure using the checked coordinate values.

According to the manipulation of the user 30, the size or location of the region of interest may be changed in real time. In order for the processor 110 to provide quantitative information in real time according to a change in the size or position of the region of interest, it must be able to efficiently search cells at a specific location from hundreds of thousands of cells. Accordingly, a data structure generated through step 510 is more suitable for a structure with high efficiency for searching data rather than a structure with high efficiency for inserting or removing data.

For example, the data structure may be an efficient tree for searching for cells in a pathology slide image, and may be implemented as, for example, a K-D tree or a ball tree, but is not limited thereto. The tree created by the processor 110 can also be used to visualize the positions of cells within an image area output to a display device.

Meanwhile, in the same manner as in step 510, the processor 110 may generate a data structure for grids set in the pathology slide image by using coordinate values of the grids.

In step 520, the processor 110 checks coordinate information corresponding to at least one cell included in the region of interest from the data structure.

The processor 110 checks the position and size of the region of interest and searches for coordinate values of cells included in the region of interest through a data structure. For example, assuming that the data structure is a predetermined tree, the processor 110 searches for coordinate values of cells in the tree.

In step 530, the processor 110 generates quantitative information based on the number corresponding to the identified coordinate information.

The processor 110 generates quantitative information by counting the number of coordinate values retrieved through step 520 . For example, the quantitative information may include the total number of cells included in the region of interest or the number of specific cells included in the region of interest.

6 is a flowchart illustrating another example in which a processor generates quantitative information according to an exemplary embodiment.

The processor 110 may generate quantitative information by calculating the number of cells in the region of interest or specific statistical values. Since a corresponding region of interest is set for an arbitrary pixel in the pathology slide image, when the user 30 designates a specific pixel through an input device (eg, mouse), the processor 110 executes the corresponding pixel. The number of cells in the region of interest corresponding to or a specific statistic may be read. Referring to FIG. 6 , an example in which the processor 110 calculates the number of cells or specific statistical values for cells in a region of interest will be described.

In step 610, the processor 110 defines a kernel based on the region of interest.

Here, at least one kernel is defined according to the type of quantitative information. As described above with reference to FIG. 4 , the size or shape of the region of interest may be defined by the user 30 . The processor 110 determines the type of quantitative information to be obtained from the region of interest based on the input of the user 30 . For example, the type of quantitative information may include the number of total cells (or cells of a specific type), the density of total cells (or cells of a specific type), the ratio between different cells, and the ratio of cells of a specific type to total cells. etc. may apply. The processor 110 defines a kernel according to the size and shape of the region of interest and the type of quantitative information. Hereinafter, an example in which the processor 110 defines a kernel will be described with reference to FIG. 7 .

7 is a diagram for explaining an example in which a processor defines a kernel according to an embodiment.

As a first example, it is assumed that the type of quantitative information is the number of cancer cells. Also, at a specific magnification of the pathology slide image, it is assumed that the region of interest 710 is a circle with radius R. In addition, it is assumed that a channel is an image in which '1' is recorded in a pixel corresponding to a cancer cell and '0' is recorded in the remaining pixels. For example, '1' may be recorded in a pixel corresponding to the center of a cancer cell.

The processor 110 sets a kernel 720 including the entire region of interest 710 . For example, the processor 110 may set a rectangle having a side length of 2R as the kernel 720 . Accordingly, the kernel 720 may be divided into a region 721 included in the region of interest 710 and a region 722 not included in the region of interest 710 . The processor 110 may define the kernel 720 by recording '1' in the area 721 and '0' in the area 722 .

As a second example, it is assumed that the type of quantitative information is the ratio of the number of cancer cells to the number of all types of cells. Also, at a specific magnification of the pathology slide image, it is assumed that the region of interest 710 is a circle with radius R. In addition, it is assumed that the first channel is an image in which '1' is recorded in a pixel corresponding to the center of the cancer cell and '0' is recorded in the remaining pixels. In addition, it is assumed that the second channel is an image in which '1' is recorded in a pixel corresponding to the center of all cells and '0' is recorded in the remaining pixels.

The processor 110 sets a first kernel and a second kernel including the entire region of interest. For example, the processor 110 may set the first kernel and the second kernel as a rectangle having a side length of 2R. Accordingly, the first kernel and the second kernel may be divided into a region included in the region of interest and a region not included in the region of interest. The processor 110 may define the first kernel and the second kernel by recording '1' in an area included in the ROI and '0' in an area not included in the ROI.

Referring back to FIG. 6 , in step 620, the processor 110 generates quantitative information through a convolution operation using a pathology slide image and a kernel. Hereinafter, an example in which the processor 110 performs a convolution operation will be described with reference to FIG. 8 .

8 is a diagram for explaining an example of a convolution operation according to an embodiment.

8 shows a channel 810 and a kernel 820 used for convolution operation. In addition, FIG. 8 shows a result 830 of performing a convolution operation by the channel 810 and the kernel 820 . 8 shows that the size of the channel 810 is 4*4 and the size of the kernel 820 is 2*2, but is not limited thereto.

The processor 110 overlaps the channel 810 and the kernel 820 to perform a sum of product operation, and after the operation, moves the kernel by one cell in a specific direction to perform the sum of product operation again. In this way, the processor 110 derives a result 830 by performing a convolution operation using the channel 810 and the kernel 820 .

In the first example of FIG. 7 , the processor 110 may calculate the number of cancer cells in the ROI corresponding to all pixels included in the channel by performing a convolution operation on the channel and the kernel. Accordingly, the processor 110 may generate the number of cancer cells in the region of interest as quantitative information.

In the second example of FIG. 7 , the processor 110 may calculate the number of cancer cells in the ROI by performing a convolution operation on the first channel and the first kernel. Also, the processor 110 may calculate the total number of cells in the ROI by performing a convolution operation on the second channel and the second kernel. Accordingly, the processor 110 may generate a ratio of the number of cancer cells to the number of all types of cells in the region of interest as quantitative information.

Referring back to FIG. 4 , in step 420, the processor 110 analyzes the pathology slide image to generate qualitative information of at least one tissue included in the pathology slide image.

A method of analyzing the pathology slide image by the processor 110 is the same as described above with reference to step 410 . Meanwhile, the qualitative information may include information on whether cells included in the tissue express PD-L1 (programmed death-ligand 1). For example, qualitative information indicates that the corresponding cells are PD-L1 positive tumor cells, PD-L1 negative tumor cells, and PD-L1 positive lymphocytes. lymphocyte) or PD-L1 positive macrophage. For example, which type of cell is the above-mentioned cells can be expressed in a predetermined color, and qualitative information can be output in a predetermined color to a display device. Hereinafter, examples in which a processor generates qualitative information will be described with reference to FIGS. 9 and 10 .

9 is a flowchart illustrating an example in which a processor generates qualitative information according to an exemplary embodiment.

In the case of tissue analysis results (e.g., PD-L1 TPS (Tumor Proportion Score), PD-L1 CPS (Combined Positive Score), cancer area, cancer stroma, etc. tissue segmentation results), the resolution is higher than the original resolution of the pathology slide image. It is created as a bitmap image having a low resolution, and each pixel of the bitmap image has a label ID of a tissue existing at a corresponding position as a pixel value. At this time, the label ID may be expressed as a number. The processor 110 may generate qualitative information using label information according to the following process.

In step 910, the processor 110 checks label information corresponding to at least one cell included in at least one tissue based on the analysis result of the at least one tissue.

First, the processor 110 checks label information for each cell or body tissue. Here, label information includes ID, color, and title of each label. And, the processor 110 determines the cell or tissue type by analyzing the pathology slide image. Then, the processor 110 searches for label information corresponding to at least one cell or tissue included in the pathology slide image from among the checked label information.

In step 920, the processor 110 converts a bitmap image corresponding to the pathology slide image based on the checked label information.

The processor 110 sets a pixel value corresponding to each cell in the bitmap image as a color value of the label color by using the color (ie, label color) in the label information retrieved through step 910 . Accordingly, each pixel of each cell expressed in the bitmap image may be set to a label color suitable for tissue characteristics.

Meanwhile, even if the resolution of the bitmap image is lower than that of the pathology slide image, the size of the pathology slide image is so large that the processor 110 may not be able to load the entire bitmap image due to the large size of the bitmap image. . In this case, the processor 110 may convert the bitmap image in another method based on resources of a device (eg, the user terminal 100) that converts the bitmap image. For example, the processor 110 divides the bitmap image into a plurality of tiles based on resources of a device that converts the bitmap image. Then, the processor 110 converts each of the tiles based on the identified label information. Then, the processor 110 combines the converted tiles.

For example, in order for the processor 110 to convert a bitmap image into a label color (step 920), WebGL that can utilize a GPU built into the device may be used. Specifically, the processor 110 loads a bitmap image into WebGL as a 2DTexture, samples the corresponding texture in a fragment shader, and renders the texture using a color value corresponding to the sampled label ID. However, even if the resolution of the bitmap image is lower than that of the original pathology slide image, if the size of the original pathology slide image is very large, it may not be possible to load all the bitmap images as 2DTexture to the GPU. In this case, the processor 110 may divide the bitmap image into a plurality of tiles and perform conversion of the bitmap image based on the tiles. An example in which the processor 110 performs conversion of a bitmap image based on tiles will be described with reference to FIG. 10 .

10 is a flowchart illustrating another example in which a processor generates qualitative information according to an exemplary embodiment.

In step 1010, the processor 110 obtains a bitmap image.

In step 1020, the processor 110 determines whether the GPU can load the bitmap image as a texture. Here, the texture may be 2DTexture.

If the GPU can load the bitmap image as a texture, it proceeds to step 1030, otherwise it proceeds to step 1040.

In step 1030, the processor 110 converts the bitmap image into a displayable image. For example, the processor 110 converts the bitmap image according to the method described above with reference to step 920 . Then, the processor 110 converts the converted bitmap image into an image capable of being output to a display device.

In step 1040, the processor 110 divides the bitmap image into tiles. For example, the processor 110 may divide the bitmap image into ¼ of the maximum size of an image that can be loaded as a texture into the GPU, but is not limited thereto.

In step 1050, the processor 110 converts the divided tiles into displayable images. For example, the processor 110 transforms each of the tiles according to the method described above with reference to step 920 . Then, the processor 110 converts the converted tiles into images that can be output to the display device.

In step 1060, the processor 110 determines whether conversion of all tiles has been completed. When the conversion of all tiles is completed, step 1070 is performed. Otherwise, step 1050 is additionally performed.

In step 1070, the processor 110 combines the transformed tiles. Specifically, the processor 110 connects the converted tiles to generate an output corresponding to the entire bitmap image.

Referring back to FIG. 4 , in step 430 , the processor 110 outputs at least one of quantitative information and qualitative information on the pathology slide image according to the manipulation of the user 30 .

For example, the processor 110 may output qualitative information corresponding to at least a portion of the pathology slide image and quantitative information corresponding to a region of interest. Part or all of the pathology slide image may be output on the display device. Also, a region of interest may be set to a portion of an image output to a display device. In this case, the processor 110 outputs qualitative information about the entire image output on the display device (eg, outputting a color corresponding to a tissue) and outputs quantitative information about a region of interest (eg, interest). The number of cells of a specific type included in the area, TPS, CPS, or TIL density) can be output.

Meanwhile, the processor 110 may adaptively output quantitative information and/or qualitative information in response to a change in an image output on the display device according to the manipulation of the user 30 .

As an example, the processor 110 may change and output at least one of quantitative information and qualitative information based on the output portion or output magnification of the pathology slide image changed according to the manipulation of the user 30 . For example, whenever the user 30 enlarges or reduces the pathology slide image, an analysis result (quantitative information and/or qualitative information) suitable for that area is provided, and analysis is performed for areas other than the area output on the display device. may not provide results.

As another example, the processor 110 may change and output at least one of quantitative information and qualitative information by comparing an output magnification of the pathology slide image changed according to a user's manipulation with a threshold magnification. For example, a specific analysis result may be provided only when the user 30 enlarges the pathology slide image at a predetermined magnification or higher. Alternatively, analysis results that were provided before the user 30 enlarged the pathology slide image may not be provided when the pathology slide image is enlarged beyond a certain magnification.

11 and 12 are diagrams illustrating examples of outputting quantitative information according to an exemplary embodiment.

11 and 12 show portions 1110 and 1210 of the pathology slide image output on the display device. Regions of interest 1120 and 1220 may be set on the pathology slide image according to the manipulation of the user 30 . In addition, the size and location of the regions of interest 1120 and 1220 may be changed according to the manipulation of the user 30 .

When the regions of interest 1120 and 1220 are set, the processor 110 generates and outputs quantitative information 1130 and 1230 of cells included in the regions of interest 1120 and 1220 . Quantitative information 1130 and 1230 may include the number of cells 1140 and 1240 of a specific type included in the region of interest, but is not limited thereto.

13 and 14 are diagrams illustrating examples of outputting quantitative information and qualitative information according to an embodiment.

Referring to FIG. 13 , the location of cancer cells and the number of cancer cells may not be output in the image 1310 before magnification (ie, the entire pathology slide image). Only qualitative information (eg, information representing tissue characteristics in colors) may be output to the image 1310 before magnification. For example, the characteristics of the tissue may indicate whether a ratio of PD-L1 positive cancer cells to PD-L1 negative cancer cells within a predetermined size circle centered on each pixel is equal to or greater than a predetermined threshold value. If the user 30 enlarges the image 1310 above a threshold magnification, the location of cancer cells may be output on the enlarged image 1320 . For example, the position of each cell may be displayed as a dot having a different color depending on whether the cell type is a PD-L1 positive tumor cell or a PD-L1 negative tumor cell. Also, a region of interest 1321 may be set in the image 1320, and quantitative information 1322 of the region of interest 1321 may be output. As quantitative information 1322 for the region of interest 1321, the TPS for the region of interest, the total number of cells in the region of interest, the width of the region of interest, the number of PD-L1 positive tumor cells in the region of interest, or the number of PD-L1 positive tumor cells in the region of interest At least one of the number of L1 negative tumor cells may be output.

Meanwhile, thumbnail images 1311 and 1323 representing the pathology slide images 1310 and 1320 may be output on one side of the screen, and images 1312 and 1324 representing the current magnification of the pathology slide images 1310 and 1320 may also be displayed. can be output.

In addition, a control panel (not shown) is output on one side of the screen, and the positions of cancer cells can be displayed or removed by selecting and/or deselecting a check box in the control panel (not shown). In addition, the PD-L1 TPS Map may be displayed, displayed, or removed by selecting and/or deselecting a check box in a control panel (not shown).

Meanwhile, referring to FIG. 14 , according to a user's manipulation (eg, enlargement of a pathology slide image, etc.), previously output first qualitative information may be changed to other second qualitative information and output. In addition, predetermined quantitative information that was not output before the user's manipulation is received may be additionally output.

For example, the first qualitative information may be an immune phenotype (IP) corresponding to at least a portion 1410 of the pathology slide image output before the user's manipulation is received, and the second qualitative information may be the user's It may be at least one characteristic of tissues and cells included in at least one portion 1420 of the pathology slide image output according to the manipulation of . However, the first qualitative information and the second qualitative information are not limited to those described above. In addition, the predetermined quantitative information may be the density of immune cells included in at least a portion 1420 of the pathology slide image output according to a user's manipulation, but is not limited thereto.

For example, only qualitative information (eg, information representing tissue characteristics in colors) may be output to the image 1410 before magnification. For example, the characteristics of an organization may indicate IP. IP can be determined based on quantitative information. For example, the IP may be determined based on at least one of the number, distribution, and density of immune cells in at least a partial area of the pathology slide.

The IP may be displayed in various classification schemes. As an example, the IP for each grid may be determined based on at least one of a density of immune cells in a cancer region and a density of immune cells in a cancer stroma on the grid. For example, depending on what range the value calculated based on at least one of the density of immune cells in the cancer region and the density of immune cells in the cancer stroma is included, the IP is immune inflamed, immune excluded (immune inflamed), excluded) and an immune desert.

In the image 1410 before magnification, a map of cells and tissues may not be output so that the IP map is better identified.

If the user 30 enlarges the image 1410 to a threshold magnification or higher, a map of characteristics of cells and tissues may be output instead of the IP map. In other words, the processor 110 may output the location of at least one cell (eg, Lymphoplasma Cell) on the screen. The processor 110 may output information on each region on the screen instead of the IP map. For example, the processor 110 may identify each region on the screen as one of a cancer region, a cancer stroma region, a necrotic region, and a background region, and overlay a corresponding color.

Also, a region of interest 1421 may be set in the image 1420, and quantitative information 1422 of the region of interest 1421 may be output. As quantitative information 1422 for the region of interest 1421, at least one of the density of immune cells in a cancer region included in the region of interest, the density of immune cells in a cancer stroma included in the region of interest, or the number of immune cells in the region of interest One can be output.

Meanwhile, thumbnail images 1411 and 1423 representing the pathology slide images 1410 and 1420 may be output on one side of the screen, and images 1412 and 1413 representing the current magnification of the pathology slide images 1410 and 1420 are also displayed. can be output.

In addition, a control panel (not shown) is output on one side of the screen, and at least one location may be displayed or removed by selecting and/or deselecting a check box in the control panel (not shown). In addition, the IP map and/or histological feature map may be displayed or removed by selecting and/or deselecting a check box in a control panel (not shown).

15 is a flowchart illustrating an example in which a processor outputs information about a pathology slide image, according to an exemplary embodiment.

In generating and outputting quantitative information and/or qualitative information by analyzing the pathology slide image, the processor 110 may process the above-described process through a plurality of threads. Accordingly, the processor 110 can reduce the time required for processing, and can output related information to the display device in real time according to the manipulation of the user 30 .

For example, referring to FIG. 15 , the first thread may receive a manipulation of the user 30 and request a task corresponding to the manipulation of the user 30 to the second thread. The second thread may transmit the processing result of the requested task to the first thread, and the first thread may output an image and related information to a display device using the received result.

In step 1510, a visualization process request is received through the first thread. Here, the visualization process request includes enlarging or reducing the pathology slide image, changing the position of the region of interest, or enlarging or reducing the size of the region of interest according to the manipulation of the user 30 .

In step 1520, the first thread waits until processing of the second thread is finished.

Through steps 1530 to 1550, the second thread acquires, analyzes, and renders the bitmap image. Steps 1530 to 1550 correspond to the operation of the processor 110 described above with reference to FIGS. 4 to 10 . Accordingly, detailed descriptions of steps 1530 to 1550 are omitted.

In step 1560, the second thread transmits the rendered bitmap image and cell coordinate information to the first thread. Then, in step 1570, the first thread creates a binary tree using the information transmitted from the second thread.

In steps 1580 and 1590, the first thread receives the user's 30 manipulation and outputs an image on the display device in response to the user's 30 manipulation. Here, the image represents an image displaying at least one portion of the pathology slide image as well as at least one of quantitative information and qualitative information.

As described above, the processor 110 outputs quantitative information or qualitative information corresponding to the region of interest set according to the manipulation of the user 30 or at least a portion of the pathology slide image output on the display device. Accordingly, the user 30 can read the image output on the display device and provide effective assistance in diagnosing the subject.

On the other hand, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. In addition, the structure of data used in the above-described method can be recorded on a computer-readable recording medium through various means. The computer-readable recording medium includes storage media such as magnetic storage media (eg, ROM, RAM, USB, floppy disk, hard disk, etc.) and optical reading media (eg, CD-ROM, DVD, etc.) do.

Those skilled in the art related to the present embodiment will be able to understand that it may be implemented in a modified form within a range that does not deviate from the essential characteristics of the above description. Therefore, the disclosed methods should be considered from an explanatory point of view rather than a limiting point of view, and the scope of rights is shown in the claims rather than the foregoing description, and should be construed to include all differences within the equivalent range.

1: system
10: user terminal
20: server
30: user
40: output image

Claims (20)

at least one memory; and
at least one processor;
the processor,
The pathology slide image is analyzed to generate quantitative information of at least one cell included in the region of interest of the pathology slide image, and the pathology slide image is analyzed to generate quantitative information of at least one tissue included in the pathology slide image. A computing device that controls a display device to generate qualitative information and output at least one of the quantitative information and the qualitative information on the pathology slide image according to a user's manipulation. According to claim 1,
the processor,
A data structure is created using coordinate information corresponding to each of the cells included in the pathology slide image, and coordinate information corresponding to at least one cell included in the region of interest is identified from the data structure. , A computing device generating the quantitative information based on a number corresponding to the identified coordinate information. According to claim 1,
the processor,
A computing device that defines a kernel based on the region of interest and generates the quantitative information through a convolution operation using the pathology slide image and the kernel. According to claim 3,
The computing device wherein at least one kernel is defined according to the type of the quantitative information. According to claim 1,
the processor,
Based on the analysis result of the at least one tissue, label information corresponding to at least one cell included in the at least one tissue is identified, and a bitmap corresponding to the pathology slide image is determined based on the identified label information. A computing device that converts images. According to claim 5,
the processor,
The bitmap image is divided into a plurality of tiles based on a resource of a device that converts the bitmap image, each of the tiles is converted based on the identified label information, and the converted tiles are Combining computing devices. According to claim 1,
the processor,
A computing device that controls the display device to output qualitative information corresponding to at least a portion of the pathology slide image and quantitative information corresponding to the region of interest. According to claim 1,
the processor,
A computing device that controls the display device to change and output at least one of the quantitative information and the qualitative information based on an output portion or an output magnification of the pathology slide image changed according to the user's operation. According to claim 1,
the processor,
A computing device that controls the display device to change and output at least one of the quantitative information and qualitative information by comparing an output magnification of the pathology slide image changed according to the user's operation with a critical magnification. generating quantitative information of at least one cell included in a region of interest of the pathology slide image by analyzing the pathology slide image;
generating qualitative information of at least one tissue included in the pathology slide image by analyzing the pathology slide image; and
and outputting at least one of the quantitative information and the qualitative information on the pathology slide image according to a user's manipulation. According to claim 10,
Generating the quantitative information,
generating a data structure using coordinate information corresponding to each of the cells included in the pathology slide image;
checking coordinate information corresponding to at least one cell included in the region of interest from the data structure; and
Generating the quantitative information based on the number corresponding to the confirmed coordinate information; method comprising a. According to claim 10,
Generating the quantitative information,
defining a kernel based on the region of interest; and
and generating the quantitative information through a convolution operation using the pathology slide image and the kernel. According to claim 12,
The method of claim 1, wherein at least one kernel is defined according to the type of the quantitative information. According to claim 10,
Generating the qualitative information,
verifying label information corresponding to at least one cell included in the at least one tissue based on an analysis result of the at least one tissue; and
and converting a bitmap image corresponding to the pathology slide image based on the checked label information. 15. The method of claim 14,
Converting the bitmap image,
dividing the bitmap image into a plurality of tiles based on resources of a device that converts the bitmap image; and
transforming each of the tiles based on the identified label information; and
and combining the converted tiles. According to claim 10,
The outputting step is
A method of outputting qualitative information corresponding to at least a portion of the pathology slide image and quantitative information corresponding to the region of interest. According to claim 10,
The outputting step is
A method of changing and outputting at least one of the quantitative information and the qualitative information based on an output portion or output magnification of the pathology slide image changed according to the user's operation. According to claim 10,
The outputting step is
A method of comparing output magnification and threshold magnification of the pathology slide image changed according to the user's operation to change and output at least one of the quantitative information and the qualitative information. According to claim 10,
The outputting step is
Based on the user's operation, the previously output first qualitative information is changed into second qualitative information and predetermined quantitative information is output;
The first qualitative information includes an immune phenotype (IP) corresponding to at least a portion of the pathology slide image output before the user's manipulation is received,
The second qualitative information includes at least one characteristic of tissues and cells included in at least a portion of the pathology slide image output according to the user's manipulation;
The method of claim 1 , wherein the predetermined quantitative information includes a density of immune cells included in at least a portion of the pathology slide image output according to the user's manipulation. A computer-readable recording medium on which a program for executing the method of claim 10 is recorded on a computer.

Images