CN111950544A - A method and device for determining a region of interest in a pathological image - Google Patents

Abstract

The present invention discloses a method and device for determining a region of interest in a pathological image, a computer device, and a computer-readable storage medium. The method includes segmenting the pathological image to obtain a plurality of sub-regions. Acquire at least one associated area of each sub-area, wherein the associated area partially coincides with the sub-area. The sub-regions and their associated regions are input into a classification model to obtain confidence that the sub-regions are regions of interest. The region of interest in the pathological image is determined based on the confidence that the sub-region is a region of interest and a preset threshold. The solution of the present invention, on the one hand, improves the reading efficiency and the reading accuracy, and on the other hand, also improves the accuracy and speed of determining whether a sub-region is a region of interest.

Description

A method and device for determining a region of interest in a pathological image

technical field

The present invention relates to the field of medical technology, and in particular, to a method and apparatus for determining a region of interest in a pathological image, computer equipment, and a computer-readable storage medium.

Background technique

Cancer is usually diagnosed by pathological examination. After sectioning the tissue to be examined, perform staining operation to obtain different staining images, such as immunohistochemical staining images. Doctors in the pathology department complete the diagnosis of a case of pathology by observing the region of interest in the stained image as a whole and locally under a microscope. However, manual observation of stained images, such as searching for cancer nests under a microscope, is time-consuming and labor-intensive, with low image reading efficiency, and there is greater subjectivity, which may lead to misjudgment.

Therefore, how to determine the region of interest in the pathological image and improve the reading efficiency and the reading accuracy has become one of the problems to be solved urgently.

SUMMARY OF THE INVENTION

The present invention provides a method, apparatus, computer equipment and computer-readable storage medium for determining a region of interest in a pathological image, so as to determine the region of interest in a pathological image. On the one hand, the efficiency of interpreting pathological images is improved, and on the other hand, the accuracy of interpreting pathological images is also improved.

The present invention provides a method for determining a region of interest in a pathological image, comprising:

Segment the pathological image to obtain multiple sub-regions;

Acquiring at least one associated area of each sub-area, wherein the associated area partially overlaps with the sub-area;

inputting the sub-region and its associated region into a classification model to obtain the confidence that the sub-region is a region of interest;

The region of interest in the pathological image is determined based on the confidence that the sub-region is the region of interest and a preset threshold.

Optionally, the sub-region is included in its associated region.

Optionally, the centers of the sub-regions and their associated regions are the same.

Optionally, the sub-regions and their associated regions have the same shape.

Optionally, there are two associated areas of the sub-areas.

Optionally, the region of interest is a cancer nest.

The present invention also provides a device for determining a region of interest in a pathological image, comprising:

A segmentation unit for segmenting the pathological image to obtain multiple sub-regions;

an acquisition unit, configured to acquire at least one associated area of each sub-area, wherein the associated area partially overlaps with the sub-area;

a classification model for inputting the sub-region and its associated region, and outputting the confidence that the sub-region is a region of interest;

A determination unit, configured to determine the region of interest in the pathological image based on the confidence that the sub-region is the region of interest and a preset threshold.

The present invention also provides a computer device, comprising at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, enables the processor to execute the above A method for determining regions of interest in pathological images.

The present invention also provides a computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor in the device, enables the device to execute the above-mentioned method for determining a region of interest in a pathological image.

Compared with the prior art, the technical scheme of the present invention has the following beneficial effects:

The pathological image is segmented to obtain a plurality of sub-regions, and at least one associated region of each sub-region is obtained, wherein the associated region partially coincides with the sub-region. The sub-regions and their associated regions are input into a classification model to obtain confidence that the sub-regions are regions of interest. The region of interest in the pathological image is determined based on the confidence that the sub-region is the region of interest and a preset threshold. Since it is no longer necessary to read the pathological images manually, the reading efficiency is improved to a certain extent, and the accuracy of the reading is also improved. Further, when judging whether a sub-area is a region of interest, not only the sub-area is input to the classification model, but the associated area of the sub-area is also input to the classification model. Therefore, the input of the classification model The regional information of multiple scales including the sub-region is fused, thereby improving the classification accuracy, that is, improving the accuracy of determining whether the sub-region is the region of interest. In addition, when determining the region of interest in the pathological image, the pathological image is divided into multiple sub-regions, and by judging whether each sub-region is a region of interest, and then determining the region of interest in the pathological image, also to a certain extent Improves the speed of determining regions of interest in pathological images. In addition, the above scheme is used to determine the region of interest in the pathological image, which is simple and efficient.

Other features and advantages of the present invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description, claims, and drawings.

The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention. In the attached image:

1 is a schematic diagram of a pathological image according to an embodiment of the present invention;

2 is a schematic diagram of a method for determining a region of interest in a pathological image according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a classification model according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

As mentioned in the background art, in the prior art, the overall and local observation of pathological sections under a microscope is used to find regions of interest, such as cancer nests (mainly composed of cancer cells). However, it is time-consuming to manually search for regions of interest. It is laborious and there may be some misjudgment. Therefore, in the embodiment of the present invention, the pathological image can be obtained by enlarging the pathological slice by a certain multiple, such as 20 times or 40 times, and then scanning to obtain the pathological image, and then determining the region of interest in the pathological image by means of artificial intelligence, so as to improve the interpretation of the slice. Efficiency and accuracy of reading. FIG. 1 is a schematic diagram of a pathological image according to an embodiment of the present invention. The pathological image in FIG. 1 includes tumor cells (positive tumor cells, negative tumor cells), immune cells, and other cells. Generally speaking, whether a cell in a pathological image is a cancer cell is mainly judged by the size of the stained cell nucleus and the color of the cell nucleus, and the nucleus is large and light in color, which is usually a cancer cell. The inventors considered that when the cancer nests in the pathological images are identified, when the sizes of the regions to be identified are different, the relative sizes and relative colors of the cell nuclei are also different. For example, when the area to be identified is small, the nucleus is relatively large and the staining is relatively deep, but as the area to be identified increases, the nucleus is relatively small and the staining is relatively shallow. Therefore, the inventor proposes that, when identifying cancer cells through a neural network, not only a sub-region to be identified is input, but multiple associated regions associated with the sub-region to be identified are obtained based on the sub-region to be identified. The sub-region to be identified and its associated region are input to the neural network together, so as to determine whether the input to-be-identified region is a cancer nest through the neural network.

The technical solution of the present invention is described in detail below with the region of interest as the cancer nest, but the technical solution of the present invention can also identify other regions of interest, such as regions composed of other cells, therefore, the region of interest is cancer The nest should not be taken as a limitation on the technical solution of the present invention.

Referring to FIG. 2, FIG. 2 is a schematic diagram of a method for determining a region of interest in a pathological image according to an embodiment of the present invention. As shown in FIG. 2 , the method for determining a region of interest in a pathological image according to an embodiment of the present invention includes:

S101: Segment the pathological image to obtain multiple sub-regions.

S102: Acquire at least one associated area of each sub-area, wherein the associated area partially overlaps with the sub-area.

S103: Input the sub-region and its associated region into a classification model to obtain a confidence that the sub-region is a region of interest.

S104: Determine a region of interest in the pathological image based on the confidence that the sub-region is a region of interest and a preset threshold.

Step S101 is performed, and the pathological image is segmented to obtain multiple sub-regions. In this embodiment, the pathological image may be segmented into multiple sub-regions with a size of 512╳512, or may be segmented into larger or smaller sub-regions. sub-regions, those skilled in the art can divide the pathological image into sub-regions of different sizes according to actual needs. The shape of the sub-region can be square, rectangle, circle, etc.

Execute S102: Acquire at least one associated area of each sub-area. In this embodiment, the associated area associated with the sub-area partially overlaps with the sub-area, which may be a larger area than the sub-area, or may be a smaller area than the sub-area, and the shape of the associated area and the shape of the sub-area may be The same or different, for example, the sub-area is a square, and the associated area is a rectangle. Subregions are rectangles, and associated regions are squares. In addition, the center of the associated area and its sub-area may be the same or different. In this embodiment, in order to facilitate the acquisition of the associated region of the sub-region and improve the accuracy of classifying the sub-region, the associated region and the sub-region have the same center and the same shape. Specifically, in this embodiment, considering the final classification accuracy and classification speed of the sub-regions, two associated regions are selected for each sub-region. Specifically, taking the sub-region as an area of 512╳512 as an example, the associated regions may be two associated regions of 1024╳1024 and 2048╳2048 with the same center and shape as the sub-region. Of course, in other embodiments, the sub-regions may be 2048╳2048, and the associated regions thereof may be two associated regions with the same center and shape, respectively 1024╳1024 and 512╳512.

Perform S103: Input the sub-region and its associated region into a classification model to obtain a confidence that the sub-region is a region of interest. In this embodiment, the classification model includes: a feature extraction network and a classification network, and the output of the feature extraction network is used as the input of the classification network. Referring to FIG. 3, FIG. 3 is a schematic diagram of a classification model according to an embodiment of the present invention. Taking the sub-region as the region of 512╳512 and the associated regions as the regions of 1024╳1024 and 2048╳2048 as examples, the classification model of the embodiment of the present invention is analyzed. describe.

As shown in Figure 3, the sub-region and its associated regions are used as three-way inputs of the classification model to the feature extraction network. For each input, it can be passed through several consecutive convolution modules to output the corresponding feature map. In this embodiment, the size of the output feature map is 256╳256 for description. In other embodiments, the size of the output feature map may also be 128╳128, 64╳64. Those skilled in the art can select the number of convolution modules to output feature maps of different sizes according to actual requirements. As shown in Figure 3, in this embodiment, the sub-region of 512╳512 outputs a feature map with a size of 256╳256 through one convolution module, and the associated region of 1024╳1024 outputs a size of 256╳ through two convolution modules The feature map of 256, the associated region of 2048╳2048 outputs a feature map of size 256╳256 through three convolution modules. Each convolution module can include a 3╳3 2D convolutional layer, a batch normalization layer (BN, BatchNormalization), an activation layer and a 2╳2 max pooling (max pooling) layer. The activation function can be a linear rectification function (ReLU, Recified Linear Unit). The three 256╳256 feature maps output by the feature network are combined into a 256╳256╳96 feature map and then input to the classification network to output the final classification result, that is, the confidence that the sub-region is a cancer nest. In this embodiment, the classification network may include two consecutive fully connected layers, and between the fully connected layer and the fully connected layer may be a dropout layer with a pass rate of 0.5. The second fully connected layer outputs the confidence that the sub-region is a cancer nest, and through the softmax operation, the sum of the confidence that the sub-region is a cancer nest and the confidence that the sub-region is not a cancer nest is 1.

It should be noted that FIG. 3 only shows a schematic diagram of a classification model according to an embodiment of the present invention, and those skilled in the art can select different types of classification models according to actual needs, as long as the classification of sub-regions can be realized.

Step S104 is performed, and a region of interest in the pathological image is determined based on the confidence that the sub-region is a region of interest and a preset threshold. In this embodiment, the preset threshold may be 0.5, that is, when the confidence level that the sub-region output by the classification model is a cancer nest is greater than 0.5, the sub-region is a cancer nest.

So far, each sub-region in the pathological image is judged through the above process to determine whether the sub-region is a cancer nest. It can help doctors to observe and diagnose cancer nests. In this embodiment, when determining whether a sub-region is a cancer nest, the information of the sub-region and its associated regions is fused to classify the sub-region, which improves the accuracy of classifying the sub-region, that is, improves the determination of the sub-region. The accuracy of whether it is a cancer nest. Using the method of dividing the pathological image into multiple sub-regions in this embodiment and determining whether the multiple sub-regions are cancer nests one by one also improves the speed of determining cancer nests in the pathological image to a certain extent, and the above method is adopted. Determining cancer nests in pathological images is fast, simple, and highly accurate.

The present invention also provides a device for determining a region of interest in a pathological image, and the device for determining a region of interest in a pathological image includes:

The segmentation unit is used to segment the pathological image to obtain multiple sub-regions.

an acquisition unit, configured to acquire at least one associated area of each sub-area, wherein the associated area and the sub-area are partially coincident.

The classification model is used to input the sub-region and its associated region, and output the confidence that the sub-region is a region of interest.

A determination unit, configured to determine the region of interest in the pathological image based on the confidence that the sub-region is the region of interest and a preset threshold.

For the implementation of the apparatus for determining a region of interest in a pathological image in this embodiment, reference may be made to the implementation of the above-mentioned method for determining a region of interest, which will not be repeated here.

Based on the same technical concept, an embodiment of the present invention provides a computer device, including at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, The processor is enabled to perform the above-described method of determining a region of interest in a pathological image.

Based on the same technical concept, an embodiment of the present invention provides a computer-readable storage medium, when the instructions in the storage medium are executed by a processor in the device, the device can perform the above-mentioned determination of the pathological image in the pathological image. area of interest method.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, or as a computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flows of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiment and all changes and modifications that fall within the scope of the present invention.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (9)

1. a method for determining a region of interest in a pathological image, comprising: Segment the pathological image to obtain multiple sub-regions; Acquiring at least one associated area of each sub-area, wherein the associated area partially overlaps with the sub-area; inputting the sub-region and its associated region into a classification model to obtain the confidence that the sub-region is a region of interest; The region of interest in the pathological image is determined based on the confidence that the sub-region is the region of interest and a preset threshold. 2. The method of claim 1, wherein the sub-region is contained in its associated region. 3. The method of claim 1, wherein the centers of the sub-regions and their associated regions are the same. 4. The method of claim 1, wherein the sub-regions and their associated regions have the same shape. 5. The method of claim 1, wherein the number of associated regions of the sub-region is two. 6. The method of claim 1, wherein the region of interest is a cancer nest. 7. A device for determining a region of interest in a pathological image, comprising: A segmentation unit for segmenting the pathological image to obtain multiple sub-regions; an acquisition unit, configured to acquire at least one associated area of each sub-area, wherein the associated area partially overlaps with the sub-area; a classification model for inputting the sub-region and its associated region, and outputting the confidence that the sub-region is a region of interest; A determination unit, configured to determine the region of interest in the pathological image based on the confidence that the sub-region is the region of interest and a preset threshold. 8. A computer device comprising at least one processor and at least one memory, wherein the memory stores a computer program which, when executed by the processor, enables the processor to execute claim 1 The method of any one of ~6. 9 . A computer-readable storage medium, when instructions in the storage medium are executed by a processor in a device, enabling the device to perform the method of any one of claims 1 to 6 .

Images