WO2021157405A1

WO2021157405A1 - Analysis device, analysis method, analysis program, and diagnosis assistance system

Info

Publication number: WO2021157405A1
Application number: PCT/JP2021/002442
Authority: WO
Inventors: 山根　健治; 真司渡辺; 長谷川　寛; 一真高橋; 威國弘
Original assignee: ソニーグループ株式会社
Priority date: 2020-02-04
Filing date: 2021-01-25
Publication date: 2021-08-12
Also published as: JP2021124861A

Abstract

Proposed are an analysis device, an analysis method, an analysis program, and a diagnosis assistance system that make it possible to verify the effects of parameters on analysis results. An analysis device 100 that comprises a reception unit 130c that receives parameters to be used in first analysis processing of an image that relates to a pathology and an output unit 130e that visually outputs first results from the first analysis processing as performed on the basis of the parameters and second results from second analysis processing performed on the basis of the first results.

Description

Analytical equipment, analysis method, analysis program and diagnostic support system

This disclosure relates to an analysis device, an analysis method, an analysis program, and a diagnostic support system.

There is a system that photographs the observation object stored on the glass slide with a microscope, generates a digitized pathological image, and performs various image analysis on the pathological image. For example, the object to be observed is a tissue or cell collected from a patient, and corresponds to a piece of meat, saliva, blood, or the like of an organ.

In the conventional system, when the user specifies an area to be analyzed for the pathological image, the specified area is analyzed by a predetermined analysis procedure using a predetermined parameter, and the final analysis is performed. Output the result.

U.S. Pat. No. 7,602,524 U.S. Pat. No. 8,1993,358

Depending on the analysis content, the analysis procedure and parameters for the pathological image have not been established, in which case the user needs to adjust the analysis procedure and parameters.

However, it was difficult to understand which analysis procedure / parameter affected the final analysis result, and there was room for improvement.

Therefore, this disclosure proposes an analysis device, an analysis method, an analysis program, and a diagnostic support system that can confirm the influence of the analysis result by the parameter.

In order to solve the above-mentioned problems, the one-form analysis apparatus according to the present disclosure includes a reception unit that receives parameters used in the first analysis process of images related to pathology, and the first one that is executed based on the parameters. It includes a first result of the analysis process and an output unit that outputs the second result of the second analysis process executed based on the first result in a visually recognizable state.

It is a figure which shows the diagnosis support system which concerns on 1st Embodiment. It is a figure for demonstrating the imaging process which concerns on 1st Embodiment. It is a figure for demonstrating the imaging process which concerns on 1st Embodiment. It is a figure for demonstrating the generation process of a partial image (tile image). It is a figure for demonstrating the pathological image which concerns on 1st Embodiment. It is a figure for demonstrating the pathological image which concerns on 1st Embodiment. It is a figure which shows an example of the viewing mode by a viewer of a pathological image. It is a figure which shows an example of the browsing history storage part which a server has. It is a figure which shows the diagnostic information storage part which a medical information system has. It is a figure which shows the diagnostic information storage part which a medical information system has. It is a figure which shows the diagnostic information storage part which a medical information system has. It is a figure which shows an example of the analysis apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the data structure of the pathological image DB. It is a figure which shows an example of the data structure of the analysis module table. It is a figure which shows an example of the data structure of the analysis module configuration information. It is a figure which shows the connection relation of the analysis module based on the analysis module configuration information shown in FIG. It is a figure which shows an example of the data structure of a parameter table. It is a figure which shows an example of the data structure of an intermediate data table. Outline It is a figure which shows an example of the data structure of a parameter conversion table. It is a figure for demonstrating the process of the analysis target designation part which concerns on this Example 1. It is a figure which shows an example of the analysis result screen which concerns on 1st Embodiment. Outline It is a figure for demonstrating the change process of a parameter. It is a flowchart which shows the analysis processing procedure which concerns on 1st Embodiment. It is a flowchart which shows the parameter display processing procedure which concerns on 1st Embodiment. It is a figure for demonstrating an example of a parameter. It is a figure which shows an example of the histogram generated by the analysis module. It is a figure which shows an example of the analysis apparatus which concerns on 2nd Embodiment. It is a figure for demonstrating the process of the analysis target designation part which concerns on this Example 2. It is a figure which shows an example of the analysis result screen which concerns on 2nd Embodiment. It is a figure which shows an example of the HE-stained image and the IHC-stained image. It is a figure which shows an example of the analysis result screen using the HE-stained image and the IHC-stained image. It is a figure which shows an example of the HE-stained image and the fluorescence IHC-stained image. It is a figure which shows an example of the inter-cell distance distribution. It is a figure which shows an example of the analysis result screen using the HE-stained image and the fluorescence IHC-stained image. It is a figure which shows an example of an HE-stained image and a CT image. It is a figure which shows an example of the analysis result screen using a HE-stained image and a CT image. It is a hardware block diagram which shows an example of the computer which realizes the function of an analyzer.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each of the following embodiments, the same parts are designated by the same reference numerals, so that duplicate description will be omitted.

In addition, the present disclosure will be described according to the order of items shown below.
<First Embodiment>
1. 1. Configuration of the system according to the first embodiment 2. Various information 2-1. Pathological image 2-2. Browsing history information 2-3. Diagnostic information 3. Analytical apparatus according to the first embodiment 4. Processing procedure 4-1. Analysis processing procedure according to the first embodiment 4-2. 5. Parameter display processing procedure according to the first embodiment. Example of parameters 6. Effect of analysis device according to the first embodiment <Second embodiment>
7. Analytical apparatus according to the second embodiment 8. Effect of analyzer according to the second embodiment 9. Analysis combining multiple types of pathological images 9-1. Analysis example using HE-stained image and IHC-stained image 9-2. Analysis example using HE-stained image and fluorescent IHC-stained image 9-3. Analysis example using images other than HE-stained images and pathological images 10. Hardware configuration 11. Conclusion

(First Embodiment)
[1. System configuration according to the embodiment]
First, the diagnosis support system 1 according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram showing a diagnosis support system 1 according to the first embodiment. As shown in FIG. 1, the diagnosis support system 1 includes a pathology system 10 and an analysis device 100.

The pathological system 10 is a system mainly used by pathologists, and is applied to, for example, laboratories and hospitals. As shown in FIG. 1, the pathology system 10 includes a microscope 11, a server 12, a display control device 13, and a display device 14.

The microscope 11 is an imaging device that has the function of an optical microscope, images an observation object stored on a glass slide, and acquires a pathological image (an example of a medical image) that is a digital image. The observation object is, for example, a tissue or cell collected from a patient, such as a piece of meat, saliva, or blood of an organ.

The server 12 is a device that stores and stores the pathological image captured by the microscope 11 in a storage unit (not shown). When the server 12 receives a browsing request from the display control device 13, the server 12 searches for a pathological image from a storage unit (not shown) and sends the searched pathological image to the display control device 13. Further, when the server 12 receives the request for acquiring the pathological image from the analysis device 100, the server 12 searches for the pathological image from the storage unit and sends the searched pathological image to the analysis device 100.

The display control device 13 sends a viewing request for the pathological image received from the user to the server 12. Then, the display control device 13 controls the display device 14 so as to display the pathological image received from the server 12.

The display device 14 has a screen on which, for example, a liquid crystal, an EL (Electro-Luminescence), a CRT (Cathode Ray Tube), or the like is used. The display device 14 may correspond to 4K or 8K, or may be formed by a plurality of display devices. The display device 14 displays a pathological image controlled to be displayed by the display control device 13. Although details will be described later, the server 12 stores browsing history information regarding the area of the pathological image observed by the pathologist via the display device 14.

The analysis device 100 is a device that sends a pathological image acquisition request to the server 12 and analyzes the pathological image received from the server 12.

[2. About various information]
[2-1. Pathological image]
As described above, the pathological image is generated when the observation object is imaged by the microscope 11. First, the imaging process by the microscope 11 will be described with reference to FIGS. 2 and 3. 2 and 3 are diagrams for explaining the imaging process according to the first embodiment. The microscope 11 described below has a low-resolution imaging unit for imaging at a low resolution and a high-resolution imaging unit for imaging at a high resolution.

FIG. 2 includes a glass slide G10 in which the observation object A10 is housed in the imaging region R10, which is a region that can be photographed by the microscope 11. The glass slide G10 is placed on, for example, a stage (not shown). The microscope 11 captures the imaging region R10 with a low-resolution imaging unit to generate an overall image, which is a pathological image in which the observation object A10 is entirely imaged. The label information L10 shown in FIG. 2 describes identification information (for example, a character string or a QR code (registered trademark)) for identifying the observation object A10. By associating the identification information described in the label information L10 with the patient, it is possible to identify the patient corresponding to the entire image. In the example of FIG. 2, "# 001" is described as the identification information. In the label information L10, for example, a brief description of the observation object A10 may be described.

Subsequently, the microscope 11 identifies the region where the observation object A10 exists from the whole image after generating the whole image, and divides the region where the observation object A10 exists into each predetermined size into high resolution. Images are sequentially taken by the imaging unit. For example, as shown in FIG. 3, the microscope 11 first images the region R11 and generates a high-resolution image I11 which is an image showing a part of the observation target A10. Subsequently, the microscope 11 moves the stage to image the region R12 by the high-resolution imaging unit, and generates a high-resolution image I12 corresponding to the region R12. Similarly, the microscope 11 produces high resolution images I13, I14, ... Corresponding to regions R13, R14, .... Although only the area R18 is shown in FIG. 3, the microscope 11 sequentially moves the stage to image all the divided areas corresponding to the observation object A10 by the high-resolution imaging unit, and corresponds to each divided area. Generate high resolution images.

By the way, when moving the stage, the glass slide G10 may move on the stage. When the glass slide G10 moves, an unphotographed region of the observation object A10 may occur. As shown in FIG. 3, the microscope 11 captures images with a high-resolution imaging unit so that adjacent divided regions partially overlap, so that even when the glass slide G10 moves, an unphotographed region is generated. Can be prevented.

The low-resolution imaging unit and the high-resolution imaging unit described above may have different optical systems or the same optical system. In the case of the same optical system, the microscope 11 changes the resolution according to the image pickup target. Further, in the above, an example in which the imaging region is changed by moving the stage is shown, but the imaging region may be changed by moving the optical system (high-resolution imaging unit or the like) of the microscope 11. Further, FIG. 3 shows an example in which the microscope 11 takes an image from the central portion of the observation object A10. However, the microscope 11 may image the observation object A10 in an order different from the imaging order shown in FIG. For example, the microscope 11 may take an image from the outer peripheral portion of the observation object A10. Further, in the above, an example in which only the region where the observation object A10 exists is imaged by the high-resolution imaging unit is shown. However, since the region where the observation object A10 exists may not be accurately detected, the microscope 11 divides the entire region of the imaging region R10 or the glass slide G10 shown in FIG. 2 and images the image with the high-resolution imaging unit. You may.

Subsequently, each high-resolution image generated by the microscope 11 is divided into predetermined sizes. As a result, a partial image (hereinafter referred to as a tile image) is generated from the high-resolution image. This point will be described with reference to FIG. FIG. 4 is a diagram for explaining a process of generating a partial image (tile image). FIG. 4 shows a high-resolution image I11 corresponding to the region R11 shown in FIG. In the following description, it is assumed that the server 12 generates a partial image from the high-resolution image. However, the partial image may be generated by a device other than the server 12 (for example, an information processing device mounted inside the microscope 11).

In the example shown in FIG. 4, the server 12 generates 100 tile images T11, T12, ... By dividing one high-resolution image I11. For example, when the resolution of the high resolution image I11 is 256 0 × 256 [pixel: pixel], the server 12 has 100 tile images T11 having a resolution of 256 × 256 [pixel: pixel] from the high resolution image I11. T12, ... Is generated. Similarly, the server 12 generates tile images by dividing other high-resolution images into similar sizes.

In the example of FIG. 4, the regions R111, R112, R113, and R114 are regions that overlap with other adjacent high-resolution images (not shown in FIG. 4). The server 12 performs stitching processing on high-resolution images adjacent to each other by aligning the overlapping areas by a technique such as template matching. In this case, the server 12 may generate a tile image by dividing the high-resolution image after the stitching process. Alternatively, the server 12 generates a tile image of an area other than the areas R111, R112, R113, and R114 before the stitching process, and generates a tile image of the area R111, R112, R113, and R114 after the stitching process. May be good.

In this way, the server 12 generates a tile image which is the minimum unit of the captured image of the observation object A10. Then, the server 12 generates tile images having different hierarchies by sequentially synthesizing the tile images of the smallest unit. Specifically, the server 12 generates one tile image by synthesizing a predetermined number of adjacent tile images. This point will be described with reference to FIGS. 5 and 6. 5 and 6 are diagrams for explaining the pathological image according to the first embodiment.

The upper part of FIG. 5 shows a tile image group of the smallest unit generated from each high-resolution image by the server 12. In the upper example of FIG. 5, the server 12 generates one tile image T110 by synthesizing four tile images T111, T112, T211 and T212 adjacent to each other among the tile images. For example, when the resolutions of the tile images T111, T112, T211 and T212 are 256 × 256, respectively, the server 12 generates the tile image T110 having a resolution of 256 × 256. Similarly, the server 12 generates the tile image T120 by synthesizing the four tile images T113, T114, T213, and T214 adjacent to each other. In this way, the server 12 generates a tile image in which a predetermined number of tile images of the smallest unit are combined.

Further, the server 12 generates a tile image obtained by further synthesizing tile images adjacent to each other among the tile images after synthesizing the tile images of the smallest unit. In the example of FIG. 5, the server 12 generates one tile image T100 by synthesizing four tile images T110, T120, T210, and T220 adjacent to each other. For example, when the resolution of the tile images T110, T120, T210, and T220 is 256 × 256, the server 12 generates the tile image T100 having the resolution of 256 × 256. For example, the server 12 uses a 4-pixel average, a weighting filter (a process that reflects close pixels more strongly than a distant pixel), and 1/2 thinning out of an image having a resolution of 512 × 512, which is a composite of four tile images adjacent to each other. By performing processing or the like, a tile image having a resolution of 256 × 256 is generated.

By repeating such a compositing process, the server 12 finally generates one tile image having the same resolution as the resolution of the tile image of the smallest unit. For example, as in the above example, when the resolution of the minimum unit tile image is 256 × 256, the server 12 repeats the above-mentioned composition process to finally obtain one tile image having a resolution of 256 × 256. Generate T1.

FIG. 6 schematically shows the tile image shown in FIG. In the example shown in FIG. 6, the tile image group of the lowest layer is the tile image of the smallest unit generated by the server 12. The tile image group in the second layer from the bottom is a tile image after the tile image group in the lowest layer is combined. Then, the tile image T1 of the uppermost layer indicates that it is one tile image finally generated. In this way, the server 12 generates a tile image group having a hierarchy like the pyramid structure shown in FIG. 6 as a pathological image.

Note that the area D shown in FIG. 5 shows an example of an area displayed on the display screen of the display device 14 or the like. For example, it is assumed that the resolution that can be displayed by the display device is a tile image for three vertical tiles and a tile image for four horizontal tiles. In this case, as in the area D shown in FIG. 5, the degree of detail of the observation object A10 displayed on the display device changes depending on the hierarchy to which the tile image to be displayed belongs. For example, when the tile image of the lowest layer is used, a narrow area of the observation object A10 is displayed in detail. Further, the wider the area of the observation object A10 is displayed coarser as the upper tile image is used.

The server 12 stores the tile images of each layer as shown in FIG. 6 in a storage unit (not shown). For example, the server 12 stores each tile image together with tile identification information (an example of partial image information) that can uniquely identify each tile image. In this case, when the server 12 receives a request for acquiring a tile image including the tile identification information from another device (for example, the display control device 13), the server 12 transmits the tile image corresponding to the tile identification information to the other device. .. Further, for example, the server 12 may store each tile image together with the layer identification information for identifying each layer and the tile identification information that can be uniquely identified within the same layer. In this case, when the server 12 receives a request for acquiring a tile image including the hierarchy identification information and the tile identification information from another device, the server 12 corresponds to the tile identification information among the tile images belonging to the hierarchy corresponding to the hierarchy identification information. Send the tile image to be sent to another device.

Note that the server 12 may store the tile images of each layer as shown in FIG. 6 in a storage device other than the server 12. For example, the server 12 may store tile images of each layer in a cloud server or the like. Further, the tile image generation process shown in FIGS. 5 and 6 may be executed by a cloud server or the like.

Further, the server 12 does not have to store the tile images of all layers. For example, the server 12 may store only the tile image of the lowest layer, may store only the tile image of the lowest layer and the tile image of the uppermost layer, or may store only a predetermined layer (for example, an odd-numbered layer). , Even-numbered layers, etc.) tile images may be stored only. At this time, when the server 12 requests a tile image of a layer that is not stored from another device, the server 12 dynamically synthesizes the stored tile image to obtain the tile image requested by the other device. Generate an image. In this way, the server 12 can prevent the storage capacity from being compressed by thinning out the tile images to be stored.

Although the imaging conditions were not mentioned in the above example, the server 12 may store tile images of each layer as shown in FIG. 6 for each imaging condition. An example of the imaging condition is a focal length with respect to a subject (observation object A10 or the like). For example, the microscope 11 may take an image of the same subject while changing the focal length. In this case, the server 12 may store tile images of each layer as shown in FIG. 6 for each focal length. The reason for changing the focal length is that some observation objects A10 are translucent, so that the focal length is suitable for imaging the surface of the observation object A10 and the inside of the observation object A10 is imaged. This is because there is a suitable focal length. In other words, the microscope 11 can generate a pathological image of the surface of the observation object A10 and a pathological image of the inside of the observation object A10 by changing the focal length.

Further, as another example of the imaging condition, there is a staining condition for the observation object A10. Specifically, in the pathological diagnosis, a luminescent substance may be stained on a specific portion (for example, the nucleus of a cell) of the observation object A10. The luminescent material is, for example, a substance that emits light when irradiated with light having a specific wavelength. Then, different luminescent substances may be stained on the same observation object A10. In this case, the server 12 may store a tile image of each layer as shown in FIG. 6 for each dyed luminescent material.

Also, the number and resolution of the tile images mentioned above are examples and can be changed as appropriate depending on the system. For example, the number of tile images synthesized by the server 12 is not limited to four. For example, the server 12 may repeat the process of synthesizing 3 × 3 = 9 tile images. Further, in the above example, the resolution of the tile image is 256 × 256, but the resolution of the tile image may be other than 256 × 256.

The display control device 13 uses software that employs a system that can handle the tile image group having a hierarchical structure described above, and is desired from the tile image group having a hierarchical structure in response to an input operation via the display control device 13 of the user. The tile image is extracted and output to the display device 14. Specifically, the display device 14 displays an image of an arbitrary portion selected by the user among images having an arbitrary resolution selected by the user. By such a process, the user can obtain the feeling of observing the observation object while changing the observation magnification. That is, the display control device 13 functions as a virtual microscope. The virtual observation magnification here actually corresponds to the resolution.

[2-2. Browsing history information]
Next, the browsing history information of the pathological image stored in the server 12 will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of a viewing mode of a pathological image by a viewer. In the example shown in FIG. 7, it is assumed that a viewer such as a pathologist browses the pathological image I10 in the order of regions D1, D2, D3, ..., D7. In this case, the display control device 13 first acquires the pathological image corresponding to the area D1 from the server 12 according to the browsing operation by the viewer. In response to a request from the display control device 13, the server 12 acquires one or more tile images forming a pathological image corresponding to the area D1 from the storage unit, and transfers the acquired one or more tile images to the display control device 13. Send. Then, the display control device 13 displays the pathological image formed from one or more tile images acquired from the server 12 on the display device 14. For example, when there are a plurality of tile images, the display control device 13 displays the plurality of tile images side by side. Similarly, each time the display control device 13 changes the display area by the viewer, the display control device 13 outputs a pathological image corresponding to the display target area (areas D2, D3, ..., D7, etc.) from the server 12. It is acquired and displayed on the display device 14.

In the example of FIG. 7, the viewer first browses the relatively wide area D1 and moves the browsing area to the area D2 because there is no area to be carefully observed in the area D1. Then, since the viewer has an area in the area D2 that he / she wants to observe carefully, he / she is browsing the area D3 by enlarging a part of the area D2. Then, the viewer further moves to the area D4, which is a part of the area D2. Then, since the viewer has an area in the area D4 that he / she wants to observe more carefully, he / she is viewing the area D5 by enlarging a part of the area D4. In this way, the viewer is also browsing the areas D6 and D7. For example, the pathological image corresponding to the regions D1, D2, and D7 is a 1.25-magnification display image, the pathological image corresponding to the regions D3 and D4 is a 20-magnification display image, and the pathological image corresponding to the regions D5 and D6. The image is a 40-magnification display image. The display control device 13 acquires and displays the tile images of the hierarchy corresponding to each magnification among the tile images of the hierarchical structure stored in the server 12. For example, the layer of the tile image corresponding to the areas D1 and D2 is higher than the layer of the tile image corresponding to the area D3 (that is, the layer close to the tile image T1 shown in FIG. 6).

While the pathological image is being browsed as described above, the display control device 13 acquires the browsing information at a predetermined sampling cycle. Specifically, the display control device 13 acquires the center coordinates and the display magnification of the viewed pathological image at predetermined timings, and stores the acquired viewing information in the storage unit of the server 12.

This point will be described with reference to FIG. FIG. 8 is a diagram showing an example of the browsing history storage unit 12a included in the server 12. As shown in FIG. 8, the browsing history storage unit 12a stores information such as “sampling”, “center coordinates”, “magnification”, and “time”. "Sampling" indicates the order of timing for storing browsing information. The "center coordinates" indicate the position information of the viewed pathological image. In this example, the center coordinates are the coordinates indicated by the center position of the viewed pathological image, and correspond to the coordinates of the coordinate system of the tile image group in the lowest layer. "Magnification" indicates the display magnification of the viewed pathological image. "Time" indicates the elapsed time from the start of browsing. In the example of FIG. 8, it is shown that the sampling period is 30 seconds. That is, the display control device 13 stores the browsing information in the browsing history storage unit 12a every 30 seconds. However, the present invention is not limited to this example, and the sampling period may be, for example, 0.1 to 10 seconds, or may be outside this range.

In the example of FIG. 8, sampling “1” indicates browsing information of region D1 shown in FIG. 7, sampling “2” indicates browsing information of region D2, and samplings “3” and “4” indicate browsing information of region D3. , The sampling "5" indicates the browsing information of the area D4, and the samplings "6", "7" and "8" indicate the browsing information of the area D5. That is, in the example of FIG. 8, the area D1 is browsed for about 30 seconds, the region D2 is browsed for about 30 seconds, the region D3 is browsed for about 60 seconds, the region D4 is browsed for about 30 seconds, and the region D5 is browsed for about 90 seconds. Indicates that it has been viewed. In this way, the browsing time of each area can be extracted from the browsing history information.

Also, the number of times each area has been browsed can be extracted from the browsing history information. For example, it is assumed that the number of times each pixel of the displayed pathological image is displayed increases by one each time the display area is changed (for example, the display area is moved or the display size is changed). For example, in the example shown in FIG. 7, when the area D1 is displayed first, the number of times each pixel included in the area D1 is displayed is one. Next, when the area D2 is displayed, the number of times each pixel included in both the area D1 and the area D2 is displayed is twice, and the number of times each pixel not included in the area D1 but included in the area D2 is displayed is. It will be once. Since the display area can be specified by referring to the center coordinates and the magnification of the browsing history storage unit 12a, each pixel of the pathological image (each) can be analyzed by analyzing the browsing history information stored in the browsing history storage unit 12a. It is possible to extract the number of times (which can be said to be coordinates) is displayed.

The display control device 13 may interrupt the storage process of the browsing information when the viewer does not perform the operation of changing the display position for a predetermined time (for example, 5 minutes). Further, in the above example, an example in which the pathological image browsed by the center coordinates and the magnification is stored as browsing information is shown, but the browsing information is not limited to this example, and the browsing information is information that can identify the area of the browsed pathological image. Any information may be used as long as it is. For example, the display control device 13 stores the tile identification information for identifying the tile image corresponding to the browsed pathological image and the information indicating the position of the tile image corresponding to the browsed pathological image as the browsing information of the pathological image. You may. Further, although not shown in FIG. 8, information for identifying a patient, a medical record, or the like is stored in the browsing history storage unit 12a. That is, the browsing history storage unit 12a shown in FIG. 8 stores the browsing information so as to be associated with the patient, the medical record, and the like.

[2-3. Diagnostic information]
Next, the diagnostic information stored in the medical information system 30 will be described with reference to FIGS. 9A to 9C. 9A to 9C are diagrams showing a diagnostic information storage unit included in the medical information system 30. 9A to 9C show an example in which diagnostic information is stored in a different table for each organ to be inspected. For example, FIG. 9A shows an example of a table that stores diagnostic information related to a breast cancer test, FIG. 9B shows an example of a table that stores diagnostic information related to a lung cancer test, and FIG. 9C shows an example of a table that stores diagnostic information related to a colon test. An example of the table is shown.

The diagnostic information storage unit 30A shown in FIG. 9A includes "patient ID", "pathological image", "diagnosis result", "grade", "tissue type", "genetic test", "ultrasonography", and "medication". Memorize information. "Patient ID" indicates identification information for identifying a patient. A "pathological image" indicates a pathological image saved by a pathologist at the time of diagnosis. In the "pathological image", position information (center coordinates, magnification, etc.) indicating an image area to be saved with respect to the entire image may be stored instead of the image itself. The "diagnosis result" is a diagnosis result by a pathologist, and indicates, for example, the presence or absence of a lesion site and the type of the lesion site. "Grade" indicates the degree of progression of the diseased area. "Histological type" indicates the type of diseased part. "Genetic test" indicates the result of the genetic test. "Ultrasonography" indicates the result of an ultrasonic examination. Dosing provides information about dosing to the patient.

The diagnostic information storage unit 30B shown in FIG. 9B stores information related to the "CT examination" performed in the lung cancer examination instead of the "ultrasonic examination" stored in the diagnostic information storage unit 30A shown in FIG. 9A. The diagnostic information storage unit 30C shown in FIG. 9C stores information related to the "endoscopy" performed in the large intestine examination instead of the "ultrasound examination" stored in the diagnostic information storage unit 30A shown in FIG. 9A. ..

In the examples of FIGS. 9A to 9C, when "normal" is stored in the "diagnosis result", it indicates that the pathological diagnosis result is negative, and information other than "normal" is shown in the "diagnosis result". If is remembered, it indicates that the result of the pathological diagnosis was positive. In addition, in FIGS. 9A to 9C, the case where each item (pathological image, diagnosis result, grade, histological type, genetic test, ultrasonography, medication) is stored in association with each other for the patient ID has been described. Information related to the examination may be stored in association with the patient ID, and not all items are required.

[3. Analytical apparatus according to the first embodiment]
Next, the analysis device 100 according to the first embodiment will be described. FIG. 10 is a diagram showing an example of the analysis device according to the first embodiment. As shown in FIG. 10, the analysis device 100 includes a communication unit 110, an input unit 111, a display unit 112, a storage unit 120, and a control unit 130.

The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 110 is connected to a network (not shown) by wire or wirelessly, and transmits / receives information to / from the pathology system 10 or the like via the network. The control unit 130, which will be described later, transmits / receives information to / from these devices via the communication unit 110.

The input unit 111 is an input device that inputs various information to the analysis device 100. The input unit 111 corresponds to a keyboard, a mouse, a touch panel, and the like.

The display unit 112 is a display device that displays information output from the control unit 130. The display unit 112 corresponds to a liquid crystal display, an organic EL (Electro Luminescence) display, a touch panel, and the like.

The storage unit 120 has a pathological image DB (Data Base) 120a, an analysis module table 120b, an analysis module configuration information 120c, a parameter table 120d, an intermediate data table 120e, and an outline parameter conversion table 120f. The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.

The pathological image DB 120a is a database that stores a plurality of pathological images. FIG. 11 is a diagram showing an example of the data structure of the pathological image DB. As shown in FIG. 11, the pathological image DB 120a has a "patient ID" and a "pathological image". The patient ID is information that uniquely identifies the patient. The pathological image shows a pathological image saved by the pathologist at the time of diagnosis. The pathological image is transmitted from the server 12. The pathological image DB 120a includes the patient ID and the pathological image, as well as the "diagnosis result", "grade", "histological type", "genetic test", "ultrasonography", and "medication" described in FIGS. 9A-9C. Information may be retained.

The analysis module table 120b is a table that holds a plurality of analysis modules that execute analysis processing for pathological images. FIG. 12 is a diagram showing an example of the data structure of the analysis module table. As shown in FIG. 12, the analysis module table 120b has a "module ID", a "module type", and an "analysis module". The module ID is information that uniquely identifies the analysis module. The module type is information indicating the analysis content executed by the analysis module. The analysis module is the data (program) of the corresponding analysis module.

For example, the analysis module with module ID "M1" is a module that executes staining separation. Staining separation is to separate a designated region into a positive region and a negative region based on the color of the pathological image.

The analysis module of module ID "M2" is a module that executes positive nucleus detection. Positive nuclei detection is the detection of cell nuclei from regions that belong to the positive. The analysis module of module ID "M3" is a module that executes negative nucleus detection. Negative nuclei detection is the detection of cell nuclei from regions that belong to the negative.

The analysis module of the module ID "M4" is a module that executes the calculation of the positive rate of the nucleus. In the calculation of the positive rate of nuclei, the positive rate of nuclei is calculated from the number of positive nuclei contained in the designated region and the number of negative nuclei.

The analysis module of module ID "M5" is a module that executes positive membrane detection. Positive membrane detection is the detection of cell membranes from regions that belong to the positive. The analysis module of module ID "M6" is a module that executes negative film detection. Negative membrane detection is the detection of cell membranes from regions that belong to the negative.

The analysis module of module ID "M7" is a module that executes the calculation of the positive rate of the membrane. In the calculation of the positive rate of the membrane, the positive rate of the membrane is calculated from the number of positive membranes contained in the designated region and the number of negative membranes.

The analysis module of module ID "M8" is a module that executes carcinoma identification. In carcinoma identification, carcinoma is identified from the positive rate of the membrane and the positive rate of the nucleus. For example, the analysis module of module ID "M8" is a non-patent document "HER2 inspection guide, [online], 3rd edition, September 2009, [Search on November 8, 1991", Internet (URL: http: http: Cancer tumors may be identified according to the protocol described in (/pathology.or.jp/news/pdf/HER2_guideline_090828.pdf).

Return to the explanation of FIG. The analysis module configuration information 120c is information indicating the connection relationship of the analysis modules. FIG. 13 is a diagram showing an example of the data structure of the analysis module configuration information 120c. As shown in FIG. 13, the analysis module configuration information 120c has a “first module ID” and a “second module ID”. The first module ID corresponds to the module ID of the analysis module of the connection source. The second module ID corresponds to the module ID of the analysis module to be connected. The analysis module configuration information 120c is an example of "order definition information".

FIG. 14 is a diagram showing a connection relationship of analysis modules based on the analysis module configuration information 120c shown in FIG. In the following description, the analysis module of the module ID "Mn (n is a natural number)" is referred to as an analysis module Mn.

As shown in FIG. 14, the analysis modules M2, M3, M5, and M6 are connected under the analysis module M1. The analysis module M4 is connected under the analysis module M2. The analysis module M4 is connected under the analysis module M3. The analysis module M7 is connected under the analysis module M5. The analysis module M7 is connected under the analysis module M6. The analysis module M8 is connected under the analysis module M4. The analysis module M8 is connected under the analysis module M7.

Return to the explanation of FIG. The parameter table 120d is a table that stores parameters used when each analysis module executes analysis processing. FIG. 15 is a diagram showing an example of the data structure of the parameter table 120d. As shown in FIG. 15, the parameter table 120d associates the module ID with the parameter set. The module ID is information that uniquely identifies the analysis module. A plurality of parameters are associated with the parameter set. In the example shown in FIG. 15, the first parameter, the second parameter, and the third parameter are shown, but the parameter set may further include other parameters.

Return to the explanation of FIG. The intermediate data table 120e is a table that stores the processing results of each analysis module. FIG. 16 is a diagram showing an example of the data structure of the intermediate data table 120e. As shown in FIG. 16, the intermediate data table 120e associates the module ID with the intermediate data. The module ID is information that uniquely identifies the analysis module. The intermediate data indicates the processing result data of the analysis module identified by the module ID.

For example, the intermediate data of the module ID "M1" is the data of the processing result of staining separation. The intermediate data of the module ID "M2" is the data of the processing result of the positive nucleus detection. The intermediate data of the module ID "M3" is the data of the processing result of the negative nucleus detection. The intermediate data of the module ID "M4" is the data of the processing result of the positive rate calculation of the nucleus. The intermediate data of the module ID "M5" is the data of the processing result of the positive film detection. The intermediate data of the module ID "M6" is the data of the processing result of the negative film detection. The intermediate data of the module ID "M7" is the data of the processing result of the positive rate calculation of the membrane. The intermediate data of the module ID "M8" is the data of the scoring processing result.

Return to the explanation of FIG. The summary parameter conversion table 120f is a table that is referred to when converting a parameter into a summary parameter. FIG. 17 is a diagram showing an example of the data structure of the outline parameter conversion table 120f. As shown in FIG. 17, the module ID and the outline parameter conversion formula are associated with the outline parameter conversion table 120f. The module ID is information that uniquely identifies the analysis module. The summary parameter conversion formula is a conversion formula used when converting a parameter into a summary parameter.

For example, as described with reference to FIG. 15, there are three parameters of the module ID "M1", that is, the first parameter, the second parameter, and the third parameter. Will be integrated.

Return to the explanation of FIG. The control unit 130 includes an acquisition unit 130a, an analysis target designation unit 130b, a reception unit 130c, an analysis unit 130d, and an output unit 130e. In the control unit 130, for example, a program (an example of an analysis program) stored inside the analysis device 100 is executed by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) using a RAM (Random Access Memory) or the like as a work area. It is realized by being done. Further, the control unit 130 may be executed by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The acquisition unit 130a is a processing unit that sends a pathological image acquisition request to the server 12 and acquires the pathological image from the server 12. The acquisition unit 130a registers the acquired pathological image in the pathological image DB 120a. In addition to the patient ID, information such as "diagnosis result", "grade", "tissue type", "genetic test", "ultrasonography", and "medication" described in FIGS. 9A to 9C is attached to the pathological image. It may have been done.

The analysis target designation unit 130b is a processing unit that designates an area to be analyzed in the pathological image based on the user's operation. In the following description, the area to be analyzed is referred to as "annotation".

FIG. 18 is a diagram for explaining the processing of the analysis target designation unit 130b according to the first embodiment. For example, the analysis target designating unit 130b acquires the information of the pathological image I1 from the pathological image DB 120a and displays it on the display unit 112. The observation object A10 is entirely visualized on the pathological image I1. The user operates the input unit 111 while referring to the display unit 112 to select the annotation RI 21. The annotation for the pathological image may be specified in advance by the pathologist of the pathological system 10.

The analysis target designation unit 130b outputs the pathological image and annotation information to the analysis unit 130d.

The reception unit 130c is a processing unit that receives parameters used in the analysis processing of the pathological image. The reception unit 130c cooperates with the output unit 130e, which will be described later, to receive changes in the parameters used by each analysis module.

The analysis unit 130d is a processing unit that executes analysis processing by a plurality of analysis modules for the annotation designated for the pathological image. The analysis process by one analysis module corresponds to the "partial analysis process". Further, the partial analysis process corresponds to the first analysis process and the second analysis process. The analysis unit 130d loads the analysis modules in order based on the analysis module configuration information 120c, and executes the partial analysis process using the parameters corresponding to the analysis modules. The analysis unit 130d receives the result of the upstream partial analysis process as an input and executes the downstream partial analysis process.

The processing of the analysis unit 130d will be described with reference to FIG. The analysis unit 130d loads the “analysis module M1” from the analysis module table 120b. The analysis unit 130d acquires the parameters for the analysis module M1. The analysis unit 130d inputs the information of the annotation of the pathological image into the analysis module M1 and executes the partial analysis process (staining separation). The analysis unit 130d registers the result (intermediate data) of the partial analysis process (staining separation) in the intermediate data table 120e.

The analysis unit 130d loads the "analysis module M2" from the analysis module table 120b. The analysis unit 130d acquires the parameters for the analysis module M2. The analysis unit 130d acquires the intermediate data of the analysis module M1 from the intermediate data table 120e. The analysis unit 130d inputs the intermediate data of the analysis module M1 to the analysis module M2 and executes a partial analysis process (positive nucleus detection). The analysis unit 130d registers the result (intermediate data) of the partial analysis process (positive nucleus detection) in the intermediate data table 120e.

The analysis unit 130d loads the "analysis module M3" from the analysis module table 120b. The analysis unit 130d acquires the parameters for the analysis module M3. The analysis unit 130d acquires the intermediate data of the analysis module M1 from the intermediate data table 120e. The analysis unit 130d inputs the intermediate data of the analysis module M1 to the analysis module M3 and executes a partial analysis process (negative nucleus detection). The analysis unit 130d registers the result (intermediate data) of the partial analysis process (negative nucleus detection) in the intermediate data table 120e.

The analysis unit 130d loads the "analysis module M5" from the analysis module table 120b. The analysis unit 130d acquires the parameters for the analysis module M5. The analysis unit 130d acquires the intermediate data of the analysis module M1 from the intermediate data table 120e. The analysis unit 130d inputs the intermediate data of the analysis module M1 into the analysis module M5 and executes a partial analysis process (positive film detection). The analysis unit 130d registers the result (intermediate data) of the partial analysis process (positive film detection) in the intermediate data table 120e.

The analysis unit 130d loads the "analysis module M6" from the analysis module table 120b. The analysis unit 130d acquires the parameters for the analysis module M6. The analysis unit 130d acquires the intermediate data of the analysis module M1 from the intermediate data table 120e. The analysis unit 130d inputs the intermediate data of the analysis module M1 to the analysis module M6 and executes a partial analysis process (negative film detection). The analysis unit 130d registers the result (intermediate data) of the partial analysis process (positive film detection) in the intermediate data table 120e.

The analysis unit 130d loads the "analysis module M4" from the analysis module table 120b. The analysis unit 130d acquires the parameters for the analysis module M4. The analysis unit 130d acquires the intermediate data of the analysis modules M2 and M3 from the intermediate data table 120e. The analysis unit 130d inputs the intermediate data of the analysis modules M2 and M3 into the analysis module M4, and executes a partial analysis process (calculation of the positive rate of the nucleus). The analysis unit 130d registers the result (intermediate data) of the partial analysis process (calculation of the positive rate of the nucleus) in the intermediate data table 120e.

The analysis unit 130d loads the "analysis module M7" from the analysis module table 120b. The analysis unit 130d acquires the parameters for the analysis module M7. The analysis unit 130d acquires the intermediate data of the analysis modules M5 and M6 from the intermediate data table 120e. The analysis unit 130d inputs the intermediate data of the analysis modules M5 and M6 into the analysis module M7, and executes a partial analysis process (calculation of the positive rate of the membrane). The analysis unit 130d registers the result (intermediate data) of the partial analysis process (calculation of the positive rate of the membrane) in the intermediate data table 120e.

The analysis unit 130d loads the "analysis module M8" from the analysis module table 120b. The analysis unit 130d acquires the parameters for the analysis module M8. The analysis unit 130d acquires the intermediate data of the analysis modules M4 and M7 from the intermediate data table 120e. The analysis unit 130d inputs the intermediate data of the analysis modules M4 and M7 into the analysis module M8 and executes a partial analysis process (carcinoma identification). The analysis unit 130d registers the result (intermediate data) of the partial analysis process (carcinoma identification) in the intermediate data table 120e.

By the way, the analysis unit 130d uses the parameter received by the reception unit 130c as a parameter for the analysis module Mn (n = 1 to 8). When the analysis unit 130d does not accept the parameter, the analysis unit 130d uses the initial value of the parameter set in advance for the analysis module Mn. For example, the initial value of the parameter is set as the parameter of the parameter set "Set1" of the parameter table 120d.

When the analysis unit 130d receives the update of the parameter from the reception unit 130c, the analysis unit 130d executes each analysis module Mn using the updated parameter. When the parameters of the specific analysis module are updated, the analysis unit 130d may execute the partial analysis process of all the analysis modules Mn, or only the partial analysis process of the analysis module located downstream of the specific analysis module. May be executed.

Further, when the analysis unit 130d receives the outline parameter from the reception unit 130c, the analysis unit 130d executes each analysis module Mn using the outline parameter. It is assumed that each analysis module Mn defines a simple algorithm for partial analysis processing that is executed based on summary parameters, in addition to the algorithm for partial analysis processing that is executed based on parameters.

The output unit 130e is a processing unit that outputs information on the progress of the analysis process executed based on the parameters in a visually recognizable state. For example, the output unit 130e generates an analysis result screen as shown in FIG. 19 and displays it on the display unit 112.

FIG. 19 is a diagram showing an example of the analysis result screen according to the first embodiment. As shown in FIG. 19, icons IC1 to IC8 are arranged on the analysis result screen 40. The icons IC1 to IC8 are icons corresponding to the analysis modules M1 to M8. The output unit 130e specifies the connection relationship between the analysis modules M1 to M8 based on the analysis module configuration information 120c, and arranges and connects the icon ICs 1 to IC8 based on the connection relationship.

Areas A41 to A48 for displaying the result (intermediate data) of the partial analysis process of the analysis module Mn are arranged in association with the icon ICn (n = 1 to 8). For example, the output unit 130e acquires the result (intermediate data) of the partial analysis process (staining separation) from the intermediate data table 120e and visually arranges it in the area A41 of the icon IC1. When the result (intermediate data) of the partial analysis process (staining separation) is updated, the output unit 130e visually arranges the updated intermediate data in the area A41.

The output unit 130e acquires the result (intermediate data) of the partial analysis process (positive nucleus detection) from the intermediate data table 120e and visually arranges it in the area A42 of the icon IC2. When the result (intermediate data) of the partial analysis process (positive nucleus detection) is updated, the output unit 130e visually arranges the updated intermediate data in the region A42.

The output unit 130e acquires the result (intermediate data) of the partial analysis process (negative nucleus detection) from the intermediate data table 120e and visually arranges it in the area A43 of the icon IC3. When the result (intermediate data) of the partial analysis process (negative nucleus detection) is updated, the output unit 130e visually arranges the updated intermediate data in the area A43.

The output unit 130e acquires the result (intermediate data) of the partial analysis process (positive film detection) from the intermediate data table 120e and visually arranges it in the area A45 of the icon IC5. When the result (intermediate data) of the partial analysis process (positive film detection) is updated, the output unit 130e visually arranges the updated intermediate data in the area A45.

The output unit 130e acquires the result (intermediate data) of the partial analysis process (negative film detection) from the intermediate data table 120e and visually arranges it in the area A46 of the icon IC6. When the result (intermediate data) of the partial analysis process (negative film detection) is updated, the output unit 130e visually arranges the updated intermediate data in the area A46.

The output unit 130e acquires the result (intermediate data) of the partial analysis process (calculation of the positive rate of the nucleus) from the intermediate data table 120e and visually arranges it in the area A44 of the icon IC4. When the result (intermediate data) of the partial analysis process (calculation of the positive rate of the nucleus) is updated, the output unit 130e visually arranges the updated intermediate data in the region A44.

The output unit 130e acquires the result (intermediate data) of the partial analysis process (calculation of the positive rate of the membrane) from the intermediate data table 120e and visually arranges it in the area A47 of the icon IC7. When the result (intermediate data) of the partial analysis process (calculation of the positive rate of the membrane) is updated, the output unit 130e visually arranges the updated intermediate data in the area A47.

The output unit 130e acquires the result (intermediate data) of the partial analysis process (carcinoma identification) from the intermediate data table 120e and visually arranges it in the area A48 of the icon IC8. When the result (intermediate data) of the partial analysis process (carcinoma identification) is updated, the output unit 130e visually arranges the updated intermediate data in the area A48.

Although not shown in FIG. 19, the output unit 130e may visually display the parameters used by the analysis module Mn in the partial analysis process on the icon ICn (n = 1 to 8). The output unit 130e may acquire the parameters set in the analysis module Mn from the parameter table 120d or from the reception unit 130c.

Here, change buttons bu1 to bu8 for changing parameters are arranged on the icon ICn (n = 1 to 8), respectively. When changing the parameters used in the analysis module M1, the user operates the input unit 111 and presses the change button bu1. Similarly, when changing the parameters used in the analysis module Mn, the user presses the change button bun.

The processing related to the parameter change is executed by the reception unit 130c described above. When the change button bun (n = 1 to 8) is pressed by the user, the reception unit 130c sets each parameter (first to third parameters) of the parameter sets Set1 to Set3 corresponding to the module ID "Mn". Obtained from the parameter table 120d.

The reception unit 130c causes the display unit 112 to display each parameter of the acquired parameter sets Set1 to Set3. The user operates the input unit 111 to select one of the parameter sets, and the reception unit 130c accepts the parameter change. After selecting any of the parameter sets, the user may operate the input unit 111 to directly change the values of the first to third parameters.

When the reception unit 130c receives the parameter change by the above process, the reception unit 130c associates the changed parameter with the module ID in which the parameter is changed and outputs the changed parameter to the analysis unit 130d.

When the analysis unit 130d receives the update of the parameter from the reception unit 130c, the analysis unit 130d executes each analysis module Mn using the updated parameter. When the parameters of the specific analysis module are updated, the analysis unit 130d may execute the partial analysis process of all the analysis modules Mn, or the part of some analysis modules located downstream of the specific analysis module. Only the analysis process may be executed. When the analysis unit 130d executes such a process, new intermediate data is generated and the intermediate data in the intermediate data table 120e is updated. By updating the intermediate data table 120e in this way, the intermediate data of the area A4n of the analysis result screen shown in FIG. 19 is also updated. The output unit 130e updates the analysis result screen 40 every time the intermediate data is updated.

By the way, when the reception unit 130c accepts the pressing of the parameter change button bun (n = 1 to 8), the reception unit 130c may generate an outline parameter based on the parameter and display it on the display unit 112. Whether or not to generate the outline parameter may be set in advance, or the user may operate the input unit 111 to set it.

The reception unit 130c generates an outline parameter based on the outline parameter conversion table 120f. For example, the reception unit 130c calculates the summary parameters of the first to third parameters of the parameter sets Set1 to Set3, and displays the summary parameters of the parameter sets Set1 to Set3 on the display unit 112 in a visually observable state.

The user operates the input unit 111 to select one of the parameter sets, and the reception unit 130c receives the summary parameters corresponding to the selected parameter set. After selecting any of the parameter sets, the user may operate the input unit 111 to directly change the value of the summary parameter. When the reception unit 130c receives the selection of the outline parameter by the above process, the reception unit 130c associates the outline parameter with the module ID and outputs the output to the analysis unit 130d.

Further, the reception unit 130c may convert the summary parameter into a visible image and display it on the icon ICn of the analysis result screen 40, and accept the change of the summary parameter. FIG. 20 is a diagram for explaining a process of changing the outline parameters. In FIG. 20, the icon IC2 will be used as an example. The icon IC 2 includes an outline parameter change area A52. The reception unit 130c converts the summary parameter as a selection candidate into an image, and arranges the converted image in the summary parameter change area A52. For example, the reception unit 130c arranges the luminance images corresponding to the outline parameters in the outline parameter change area A52. Each image and each summary parameter are associated with each other, and when any image is selected by the user, the reception unit 130c acquires the summary parameter corresponding to the received image.

[4. Processing procedure]
[4-1. Analysis processing procedure according to the first embodiment]
Next, the processing procedure according to the first embodiment will be described with reference to FIGS. 21 and 22. FIG. 21 is a flowchart showing an analysis processing procedure according to the first embodiment. As shown in FIG. 21, the analysis unit 130d of the analysis device 100 acquires the analysis module configuration information 120c (step S101). The analysis target designation unit 130b of the analysis device 100 acquires the pathology image from the pathology image DB 120a and displays it on the display unit 112 (step S102).

The analysis target designation unit 130b accepts the annotation designation from the user who operates the input unit 111 (step S103). The analysis unit 130d of the analysis device 100 loads the analysis module from the analysis module table 120b (step S104). The analysis unit 130d acquires the parameters for the loaded analysis module (step S105).

The analysis unit 130d executes the partial analysis process of the analysis module (step S106). The analysis unit 130d stores the processing result of the partial analysis process in the intermediate data table 120e (step S107).

The output unit 130e generates an analysis result screen based on the intermediate data of the intermediate data table 120e and displays it on the display unit 112 (step S108). If the analysis unit 130d does not have the next analysis module (steps S109, No), the analysis unit 130d proceeds to step S111. On the other hand, if there is a next analysis module (step S109, Yes), the analysis unit 130d loads the next analysis module (step S110) and proceeds to step S105.

The reception unit 130c determines whether or not the parameter change has been accepted (step S111). If the reception unit 130c has not received the parameter change (steps S111, No), the reception unit 130c ends the analysis process. On the other hand, when the reception unit 130c accepts the parameter change (step S111, Yes), the analysis unit 130d loads the analysis module related to the changed parameter (step S112) and proceeds to step S105.

[4-2. Parameter display processing procedure according to the first embodiment]
FIG. 22 is a flowchart showing a parameter display processing procedure according to the first embodiment. As shown in FIG. 22, the output unit 130e of the analysis device 100 displays the analysis result screen (step S201). If the reception unit 130c of the analysis device 100 does not receive the parameter change button bun (steps S202, No), the process ends.

When the receiving unit 130c of the analysis device 100 receives the pressing of the parameter change button bun (steps S202 and No), the process proceeds to step S203. When the reception unit 130c displays the summary parameter (step S203, Yes), the reception unit 130c calculates the summary parameter based on the summary parameter conversion table 120f (step S204). The reception unit 130c displays the summary parameter (step S205). On the other hand, when the reception unit 130c does not display the summary parameter (step S203, No), the reception unit 130c displays the parameter (step S206).

[5. Parameter example]
An example of the parameters used in the analysis module Mn will be described below. In the partial analysis process for dyeing separation as in the analysis module M1, two vectors are arranged in the RGB (Red Green Blue) space, and these two vectors are used as parameters. The analysis unit 130d executes the analysis module M1 and uses two vectors (first vector and second vector) to annotate the region of staining for detecting the nucleus and the staining for detecting the membrane. Separate into areas. For example, when the color of the region is close to the first vector, the analysis unit 130d sets such a region as a staining region for detecting the nucleus. When the color of the region is close to the second vector, the analysis unit 130d sets such a region as a staining region for detecting the film.

In the analysis process for detecting the cell nucleus as in the analysis modules M2 and M3, the circularity, irregularity, roundness, etc. of the nucleus are specified as parameters. The analysis modules M2 and M3 execute a partial analysis process for detecting nuclear cells from annotations using at least one of the circularity, deformity, roundness, and the like of the nucleus. Further, in the analysis modules M2 and M3 (other analysis modules Mn), the score regarding the CR ratio of cells and the degree of dyeing of stains may be used as parameters.

When detecting a cell corresponding to annotation, a certain analysis module Mn may execute a process of detecting a cell in which the distance between the annotation and the cell nucleus is less than the threshold value and the closest cell is detected as the annotation cell. The threshold value used here is used as a parameter. FIG. 23 is a diagram for explaining an example of parameters. In the example shown in FIG. 23, the nuclei n1, n2, and n3 exist in the vicinity of the annotation A21, but the nucleus n1 which is the shortest distance from the annotation A21 and is less than the threshold value (parameter) is attached to the annotation A21. Selected as the nucleus of the corresponding cell.

A certain analysis module Mn (distance module) classifies a plurality of cells into a first cell group and a second cell group by executing clustering when a plurality of cells are detected, and the first cell group and the first cell group. A process of calculating the distance from the second cell group may be executed. When calculating the distance between the first cell group and the second cell group, information indicating how to determine the cell to be compared is specified as a parameter. For example, it is specified by the parameter that the cell close to the center of gravity of the first cell group and the cell close to the center of gravity of the second cell group are to be compared.

For example, in such an analysis module (distance module), a histogram is generated based on the relationship between the distance and frequency between the first cell group and the second cell group. FIG. 24 is a diagram showing an example of a histogram generated by the analysis module. The vertical axis of the histogram h1 corresponds to the frequency, and the horizontal axis corresponds to the distance. The output unit 130e may visually display the histogram h1 as intermediate data in association with the icon of the analysis module.

[6. Effect of Analytical Device According to First Embodiment]
According to the analysis device 100 according to the first embodiment, the reception unit 130c receives the parameters used in the analysis processing of the pathological image, and the output unit 130e is in the middle of the analysis processing executed based on the parameters. Output information in a visible state. As a result, the user can confirm the influence of the analysis result by the parameter.

According to the analysis device 100, the reception unit 130c further accepts the parameter change, and the output unit 130e can visually check the progress of the analysis process executed based on the changed parameter each time the parameter is changed. Output in the state. As a result, the user can confirm the information on the progress of the analysis result caused by the parameter change.

According to the analysis device 100, the output unit 130e outputs the information on the progress of the analysis process in a visually recognizable state in association with the parameters. This allows the user to easily check the parameters used by the analysis module.

According to the analysis device 100, the output unit 130e can visually recognize the information on the progress of the analysis process executed based on the summary parameter in which a plurality of parameters are combined and the summary parameter in association with each other. Output in the state. As a result, the user can confirm the influence of the analysis result by the summary parameter. In addition, the user can roughly grasp a plurality of parameters with one summary parameter.

In the analysis device 100, the analysis unit 130d reads out each analysis module stored in the analysis module table 120b in order and executes the partial analysis process, and the output unit 130e makes the results of the partial analysis process visible. Output. As a result, the user can confirm the result of each partial analysis process affected by the parameter change.

The analysis device 100 stores the result (intermediate data) of the analysis unit 130d executing the upstream partial analysis process in the intermediate data table 120e, and executes the downstream partial analysis process of the intermediate data of the intermediate data table 120e. Used as a case input. As a result, the downstream partial analysis process can be executed using the result of the upstream partial analysis process.

The analysis device 100 reads out each analysis module based on the analysis module configuration information 120c, and sequentially executes the partial analysis process of each analysis module. As a result, the partial analysis processing of each analysis module can be executed in the order according to the analysis module configuration information.

The partial analysis process executed by the analyzer 100 includes a process of detecting the cell nucleus, and the parameters of the process of detecting the cell nucleus include circularity, irregularity, and roundness. The analysis unit 130d executes a partial analysis process for detecting a cell nucleus from a pathological image using at least one of circularity, irregularity, and roundness. Thereby, the cell nucleus can be detected by executing the partial analysis process using parameters such as circularity, irregularity, and roundness.

The partial analysis process executed by the analysis device 100 includes a process for separating the dyeing, and the parameters of the process for separating the dyeing include a vector in RGB space for performing the separation. The analysis unit 130d can separate the annotation region into a staining region for detecting the nucleus and a staining region for detecting the film by using it as a vector parameter in the RGB space.

(Second embodiment)
[7. Analytical apparatus according to the second embodiment]
In the first embodiment, the case where one annotation is specified for the pathological image and the analysis process is executed for the annotation has been described. In the second embodiment, an example will be described in which analysis processing is executed for each of a plurality of annotations designated for a plurality of pathological images, and information on the progress of each analysis processing is output in a visually recognizable state. Although not shown, it is assumed that the analysis device according to the second embodiment is connected to the pathological system 10 described with reference to FIG.

Here, the plurality of pathological images may be any combination of images. To enumerate an example of a combination of multiple images, "pathological images stained with different reagents", "bright-field images and fluorescent images", "general-stained images and special-stained images", "pathological images and pathological images" "Different medical images" are raised.

For example, a bright-field image is a microscope image observed by a known bright-field method in which a transmitted wave is selected and observed with an objective aperture. The fluorescent image is an image of cells in which the cells are subjected to fluorescent staining to increase the contrast. The general stained image is a pathological image obtained by staining the tissue collected from the lesion with hematoxylin and eosin (HE). The special stained image is a pathological image obtained by performing staining to visualize a specific tissue element for each purpose on the tissue collected from the lesion (special staining) and photographing the specially stained tissue. The medical image corresponds to an X-ray image (CT image), an endoscopic image, a microscope image, or the like.

FIG. 25 is a diagram showing an example of the analysis device according to the second embodiment. As shown in FIG. 25, the analysis device 200 includes a communication unit 210, an input unit 211, a display unit 212, a storage unit 220, and a control unit 230.

The communication unit 210 is realized by, for example, a NIC or the like. The communication unit 210 is connected to a network (not shown) by wire or wirelessly, and transmits / receives information to / from the pathology system 10 or the like via the network. The control unit 230, which will be described later, transmits / receives information to / from these devices via the communication unit 210.

The input unit 211 is an input device that inputs various information to the analysis device 200. The input unit 211 corresponds to a keyboard, mouse, touch panel, and the like.

The display unit 212 is a display device that displays information output from the control unit 230. The display unit 212 corresponds to a liquid crystal display, an organic EL display, a touch panel, and the like.

The storage unit 220 has a pathological image DB 220a, an analysis module table 220b, an analysis module configuration information 220c, a parameter table 220d, an intermediate data table 220e, and an outline parameter conversion table 220f. The storage unit 220 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.

The pathological image DB 220a is a database that stores a plurality of pathological images. The description of the pathological image DB 220a is the same as the description of the pathological image DB 120a described in the first embodiment.

The analysis module table 220b is a table that holds a plurality of analysis modules that execute analysis processing for pathological images. The description of the analysis module table 220b is the same as the description of the analysis module table 120b described in the first embodiment.

The analysis module configuration information 220c is information indicating the connection relationship of the analysis module. The description of the analysis module configuration information 220c is the same as the description of the analysis module configuration information 120c described in the first embodiment.

The parameter table 220d is a table that stores the parameters used when each analysis module executes the analysis process. The description of the parameter table 220d is the same as the description of the parameter table 120d described in the first embodiment.

The intermediate data table 220e is a table that stores the processing results of each analysis module. The description of the intermediate data table 220e is the same as the description of the intermediate data table 120e described in the first embodiment.

The summary parameter conversion table 220f is a table that is referred to when converting a parameter into a summary parameter. The description of the summary parameter conversion table 220f is the same as the description of the summary parameter conversion table 120f described in the first embodiment.

The control unit 230 has an acquisition unit 230a, an analysis target designation unit 230b, a reception unit 230c, an analysis unit 230d, and an output unit 230e. The control unit 230 is realized by, for example, executing a program (an example of an analysis program) stored in the analysis device 200 by a CPU or an MPU with a RAM or the like as a work area. Further, the control unit 230 may be executed by an integrated circuit such as an ASIC or FPGA.

The acquisition unit 230a is a processing unit that sends a pathological image acquisition request to the server 12 and acquires the pathological image from the server 12. The acquisition unit 230a registers the acquired pathological image in the pathological image DB 220a.

The analysis target designation unit 230b is a processing unit that designates annotations to be analyzed for a plurality of pathological images based on the user's operation.

FIG. 26 is a diagram for explaining the processing of the analysis target designation unit 230b according to the second embodiment. For example, the analysis target designation unit 230b acquires the information of the pathological images I1 and I2 from the pathological image DB 220a and displays them on the display unit 212.

The observation object A10 is depicted as a whole on the pathological image I1. The observation target A11 is entirely visualized on the pathological image I2. For example, the user operates the input unit 211 while referring to the display unit 212 to select the annotations RI21, RI22, and RI23. Each annotation for the pathological image may be designated in advance by the pathologist of the pathological system 10.

The analysis target designation unit 230b outputs the pathological image and annotation information to the analysis unit 230d.

The reception unit 230c is a processing unit that receives parameters used in the pathological image analysis processing. The reception unit 230c may receive changes in the parameters used by each analysis module in cooperation with the output unit 230e described later.

The analysis unit 230d is a processing unit that executes analysis processing by a plurality of analysis modules for a plurality of annotations designated for each pathological image. The process executed by the analysis unit 230d for one annotation is the same as the analysis process executed by the analysis unit 130d. For example, the analysis unit 230d receives the annotations RI21, RI22, and RI23 as inputs, and executes the analysis process described in the analysis unit 130d, respectively.

Although not shown, the analysis unit 230d has intermediate data of the analysis process (plural partial analysis process) executed for the annotation RI21 and intermediate data of the analysis process (plural partial analysis process) executed for the annotation RI22. , The intermediate data of the analysis process (plurality of partial analysis processes) executed for the annotation RI23 is distinguished and registered in the intermediate data table 220e.

The analysis unit 230d uses the parameters received by the reception unit 230c as parameters for the analysis module Mn (n = 1 to 8). When the analysis unit 230d does not accept the parameter, the analysis unit 230d uses the initial value of the parameter set in advance for the analysis module Mn. For example, the initial value of the parameter is set as the parameter of the parameter set "Set1" of the parameter table 220d.

When the analysis unit 230d receives the update of the parameter from the reception unit 230c, the analysis unit 230d executes each analysis module Mn using the updated parameter. When the parameters of the specific analysis module are updated, the analysis unit 230d may execute the partial analysis process of all the analysis modules Mn, or only the partial analysis process of the analysis module located downstream of the specific analysis module. May be executed.

Further, when the analysis unit 230d receives the outline parameter from the reception unit 230c, the analysis unit 230d executes each analysis module Mn using the outline parameter. It is assumed that each analysis module Mn defines a simple algorithm for partial analysis processing that is executed based on summary parameters, in addition to the algorithm for partial analysis processing that is executed based on parameters.

The output unit 230e is a processing unit that outputs information on the progress of the analysis process executed based on the parameters in a visually recognizable state. For example, the output unit 230e generates an analysis result screen 50 as shown in FIG. 27 and displays it on the display unit 112.

FIG. 27 is a diagram showing an example of the analysis result screen 50 according to the second embodiment. As shown in FIG. 27, icons IC1-1 to IC8-1, IC1-2 to IC8-2, and IC1-3 to IC8-3 are arranged on the analysis result screen 50. Icons IC1-1 to IC8-1, IC1-2 to IC8-2, and IC1-3 to IC8-3 are icons corresponding to the analysis modules M1 to M8, respectively. The output unit 230e identifies the connection relationship between the analysis modules M1 to M8 based on the analysis module configuration information 220c, and based on the connection relationship, the icons IC1 to IC8, IC1-2 to IC8-2, and IC1-3. ~ IC8-3 is arranged and connected.

The icon ICn-1 (n = 1 to 8) is associated with an area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the “annotation RI21”.

The icon ICn-2 (n = 1 to 8) is associated with an area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the “annotation RI22”.

The icon ICn-3 (n = 1 to 8) is associated with an area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the “annotation RI23”.

Here, change buttons for changing parameters are arranged on the icons ICn-1, ICn-2, and ICn-3 (n = 1 to 8), respectively. For example, by pressing the change button of the icon IC1-1, the user changes the parameters used in the analysis module M1 that executes the staining separation for the annotation RI21.

By pressing the change button of the icon IC1-2, the user changes the parameters used in the analysis module M1 that executes staining separation for the annotation RI22. By pressing the change button of the icon IC1-3, the user changes the parameters used in the analysis module M1 that executes the staining separation for the annotation RI23.

The process of pressing the change button to change the parameter is the same as that of the first embodiment. The user can specify independent parameters for the analysis module that executes the same partial analysis process. For example, the parameters specified individually are distinguished from each other by using the identification information of the icon or the like. The user may operate the input unit 111 to collectively specify the parameters. For example, the reception unit 230c may accept the parameters specified for the icons IC1-1 to IC8-1 as they are as the parameters of the icons IC1-2 to IC8-2 and IC1-3 to IC8-3.

When the reception unit 230c receives the parameter change by the above process, the reception unit 230c associates the changed parameter with the identification information of the icon in which the parameter has been changed and outputs the changed parameter to the analysis unit 230d. When the analysis unit 230d receives the update of the parameter from the reception unit 130c, the analysis unit 230d executes each analysis module Mn using the updated parameter.

For example, when the parameters of the icons IC1-1, IC1-2, and IC1-3 are updated, the analysis unit 230d loads the analysis module M1 and sets the parameters specified by the icon IC1-1. Intermediate data is calculated by executing the dyeing separation used, the dyeing separation using the parameters specified by the icon IC1-2, and the dyeing separation using the parameters specified by the icon IC1-3, respectively. The same applies to the other analysis modules Mn.

[8. Effect of Analytical Device According to Second Embodiment]
According to the analysis device 200 according to the second embodiment, the output unit 230e outputs information on the progress of each analysis result executed for a plurality of different pathological images in a visually recognizable state. As a result, the user can compare and confirm the influence of the analysis result by the parameter on a plurality of different pathological images. For example, even if the user can set the optimum parameters for one pathological image, can the analysis result be obtained appropriately when the analysis process is executed for other pathological images using such parameters? It is possible to compare and examine whether or not.

[9. Analysis combining multiple types of pathological images]
[9-1. Analysis example using HE-stained image and IHC-stained image]
The analysis device 200 according to the second embodiment described above can perform an analysis combining a plurality of types of pathological images. First, an analysis example using an HE-stained image and an IHC-stained image will be described. FIG. 28 is a diagram showing an example of an HE-stained image and an IHC-stained image. The analysis target designation unit 230b acquires the information of the HE-stained image I1a, the IHC-stained image I1b, and the IHC-stained image I1c from the pathological image DB 220a and displays them on the display unit 212.

The HE-stained image I1a corresponds to a general-stained image. The IHC-stained image I1b corresponds to a special-stained image (ER; nuclear-stained), and the IHC-stained image I1c corresponds to a special-stained image (HER2; membrane-stained). The observation target A10a is entirely extracted from the HE-stained image I1a. The observation target A10b is entirely extracted from the IHC-stained image I1b. The observation target A10c is entirely extracted from the IHC-stained image I1c.

The user operates the input unit 211 while referring to the display unit 212 to select the annotation RI21a. When the analysis target designation unit 230b accepts the selection of the annotation RI21a, it corresponds to the annotation RI21a after correcting the positions and rotations of the observation targets A10b and A10C in order to correct the deviation of the position and orientation from the observation target A10a. Annotations RI21b and RI21c are set at the positions, respectively. The user may operate the input unit 211 to specify the annotations RI21b and RI21c.

The analysis target designation unit 230b outputs the information of the HE-stained image I1a, the IHC-stained image I1b, the IHC-stained image I1c, and the annotations RI21a, RI21b, and RI21c to the analysis unit 230d.

The analysis unit 230d receives the annotation RI21a as an input, detects the region of the nucleus being stained, and calculates the shape feature amount (area, perimeter, major axis length, minor axis length, circularity, etc.). The analysis unit 230d calculates a "nuclear shape score" indicating the degree of nuclear atypia from the shape feature amount. For example, the analysis unit 230d executes HE staining separation, nuclear region detection, nuclear shape feature amount calculation, and nuclear shape score calculation as partial analysis processing.

The analysis unit 230d uses the annotation RI21b as an input to detect positive nuclei and negative nuclei. The analysis unit 230d calculates the nuclear staining intensity from the detected positive nuclei and the nuclear staining rate from the number of positive nuclei and the number of negative nuclei. The analysis unit 230d calculates the "nuclear staining score" from the nuclear staining intensity and the nuclear staining rate. For example, the analysis unit 230d executes IHC staining separation, negative nucleus detection, positive nucleus detection, nuclear staining rate calculation, nuclear staining intensity calculation, and nuclear staining score calculation as partial analysis processing.

The analysis unit 230d uses the annotation RI21c as an input to detect negative nuclei, and detects a positive film around the detected negative nuclei. The analysis unit 230d calculates the film staining intensity from the detected positive film and the film staining rate from the number of negative nuclei and the number of positive films. The analysis unit 230d calculates the "film staining score" from the film staining intensity and the film staining rate. For example, the analysis unit 230d executes IHC staining separation, negative nucleus detection, positive film detection, film staining rate calculation, film staining intensity calculation, and film staining score calculation as partial analysis processing.

Then, the analysis unit 230d calculates the total score from the nuclear shape score, the nuclear staining score, and the membrane staining score. For example, the analysis unit 230d executes total score calculation as a partial analysis process.

Although not shown, it is assumed that each analysis module Mn corresponding to each partial analysis process is registered in the analysis module table 220b. Further, it is assumed that the connection-related information of each analysis module is registered in the analysis module configuration information 220c.

The analysis unit 230d refers to the intermediate data of the analysis process (plural partial analysis processes) executed on the annotation RI21a, the intermediate data of the analysis process (plural partial analysis processes) executed on the annotation RI21b, and the annotation RI21c. The intermediate data of the executed analysis process (plurality of partial analysis processes) is distinguished and registered in the intermediate data table 220e.

The analysis unit 230d uses the parameter received by the reception unit 230c as a parameter for the analysis module Mn. When the analysis unit 230d does not accept the parameter, the analysis unit 230d uses the initial value of the parameter set in advance for the analysis module Mn.

The output unit 230e is a processing unit that outputs information on the progress of the analysis process executed based on the parameters in a visually recognizable state. For example, the output unit 230e generates an analysis result screen 60 as shown in FIG. 29 and displays it on the display unit 112.

FIG. 29 is a diagram showing an example of the analysis result screen 60 using the HE-stained image and the IHC-stained image. As shown in FIG. 29, icons IC61-1 to IC64-1, IC61-2 to IC66-2, IC61-3 to IC66-3, and IC67 are arranged on the analysis result screen 60.

Icons IC61-1 to IC64-1 are icons corresponding to analysis modules for HE staining separation, nuclear region detection, nuclear shape feature amount calculation, and nuclear shape score calculation. The icons IC61-2 to IC66-2 are icons corresponding to the analysis modules of IHC staining separation, negative nucleus detection, positive nucleus detection, nuclear staining rate calculation, nuclear staining intensity calculation, and nuclear staining score calculation. The icons IC61-3 to IC66-3 are icons corresponding to the analysis modules of IHC staining separation, negative nucleus detection, positive film detection, film staining rate calculation, film staining intensity calculation, and film staining score calculation. The icon IC67 is an icon corresponding to the analysis module for calculating the total score.

The icon ICn-1 (n = 61 to 64) is associated with an area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the “annotation RI21a”.

An area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the "annotation RI21b" is arranged in association with the icon ICn-2 (n = 61 to 66).

An area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the "annotation RI21c" is arranged in association with the icon ICn-3 (n = 61 to 66). An area for visually displaying the result of total score calculation is associated with the icon IC 67.

Note that the icons ICn-1, ICn-2, ICn-3, and IC67 are provided with change buttons for changing parameters, respectively. For example, the user can change the parameters by pressing the change button of the icon IC.

[9-2. Analysis example using HE-stained image and fluorescent IHC-stained image]
Subsequently, an analysis example using the HE-stained image and the fluorescent IHC-stained image will be described. FIG. 30 is a diagram showing an example of an HE-stained image and a fluorescence IHC-stained image. The analysis target designation unit 230b acquires the information of the HE-stained image I1a and the fluorescence IHC-stained image I1d from the pathological image DB 220a and displays them on the display unit 212.

The HE-stained image I1a corresponds to a general-stained image. The fluorescent IHC-stained image I1d is an image of cells in which the cells are subjected to fluorescent staining to enhance the contrast. The observation target A10a is entirely extracted from the HE-stained image I1a. The observation target A10d is entirely extracted from the fluorescent IHC-stained image I1d.

The user operates the input unit 211 while referring to the display unit 212 to select the annotation RI21a. When the analysis target designation unit 230b receives the selection of the annotation RI21a, the analysis target designation unit 230b corrects the position / rotation deviation of the observation target A10d from the observation target A10a. , Annotation RI21d is set. The user may operate the input unit 211 to specify the annotation RI21d.

The analysis target designation unit 230b outputs the information of the HE stained image I1a, the fluorescent IHC stained image I1d, and the annotations RI21a and RI21d to the analysis unit 230d.

The analysis unit 230d receives the annotation RI21a as an input, detects the region of the nucleus being stained, and calculates the shape feature amount (area, perimeter, major axis length, minor axis length, circularity, etc.). The analysis unit 230d performs a gating process (for example, a process of selecting only a cell population of a feature amount of interest in a sample and creating a histogram of only that cell) based on the shape feature amount, and a cell having a predetermined feature. Is extracted as tumor cells. For example, the analysis process 230d executes HE stain separation, nuclear region detection, nuclear shape feature calculation, and tumor cell extraction as partial analysis processes.

The analysis unit 230d receives the annotation RI21d as an input, performs fluorescence staining separation, and detects positive cells (T cells, B cells, etc.) for each staining. For example, the analysis processing unit 230d executes fluorescence staining separation, positive cell detection (fluorescence 1; T cells), and positive cell detection (fluorescence 2; B cells) as partial analysis processing.

The analysis unit 230d calculates the round-robin distance between the nucleus of the tumor cell detected from the HE-stained cells and the cell detected from the fluorescence image, and creates an intercellular distance distribution. For example, the analysis unit 230d executes the calculation of the distance distribution between the first cells and the calculation of the distance distribution between the second cells as the partial analysis process. The distance between the first cells indicates the distance between the nucleus of the tumor cell and the T cell. The second cell-to-cell distance indicates the distance between the nucleus of the tumor cell and the B cell.

FIG. 31 is a diagram showing an example of the intercellular distance distribution. In the graph shown in FIG. 31, the vertical axis corresponds to the frequency and the horizontal axis corresponds to the intercellular distance. The curve 31a shows the distribution of the first cell-cell distance calculated by the calculation of the first-cell distance distribution. The curve 31b shows the distribution of the second cell-to-cell distance calculated by the calculation of the second-cell distance distribution.

The analysis unit 230d distinguishes between the intermediate data of the analysis process (plural partial analysis processes) executed for the annotation RI21a and the intermediate data of the analysis process (plural partial analysis processes) executed for the annotation RI21d, and intermediates them. Register in the data table 220e.

The output unit 230e is a processing unit that outputs information on the progress of the analysis process executed based on the parameters in a visually recognizable state. For example, the output unit 230e generates an analysis result screen 70 as shown in FIG. 32 and displays it on the display unit 112.

FIG. 32 is a diagram showing an example of the analysis result screen 70 using the HE-stained image and the fluorescent IHC-stained image. As shown in FIG. 32, icons IC71-1 to IC74-1, IC71-2 to IC72-3, IC73, and IC74 are arranged on the analysis result screen 70.

Icons IC71-1 to IC74-1 are icons corresponding to the analysis modules of HE stain separation, nuclear region detection, nuclear shape feature calculation, and tumor cell extraction. Icons IC71-2 to IC73-2 are icons corresponding to analysis modules for fluorescence staining separation, positive cell detection (fluorescence 1; T cells), and positive cell detection (fluorescence 1; T cells). The icon IC73 is an icon corresponding to the analysis module for calculating the distance between the first cells. The icon IC74 is an icon corresponding to the analysis module for calculating the distance between the second cells.

The icon ICn-1 (n = 71 to 74) is associated with an area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the “annotation RI21a”.

The icon ICn-2 (n = 71 to 73) is associated with an area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the “annotation RI21d”.

The icon IC73 is set with an area for visually displaying the result (intermediate data) of the calculation of the distance between the first cells. The icon IC 74 is set with an area for visually displaying the result (intermediate data) of the calculation of the distance between the second cells.

Note that the icons ICn-1, ICn-2, IC73, and IC74 are provided with change buttons for changing parameters, respectively. For example, the user can change the parameters by pressing the change button of the icon IC.

[9-3. Analysis example using images other than HE-stained images and pathological images]
Subsequently, an analysis example using an image other than the HE-stained image and the pathological image will be described. Here, a CT image (X-ray image) will be used as an image other than the pathological image. FIG. 33 is a diagram showing an example of an HE-stained image and a CT image. The analysis target designation unit 230b acquires the information of the HE-stained image I1a from the pathological image DB 220a, acquires the information of the CT image G33 from a DB (not shown), and displays it on the display unit 212.

The HE-stained image I1a corresponds to a general-stained image. The CT image G33 is an image other than the pathological image. The observation target A10a is entirely extracted from the HE-stained image I1a.

The user operates the input unit 211 while referring to the display unit 212 to select the annotations RI21a and RI21e. The analysis target designation unit 230b may automatically select the annotation RI21e corresponding to the annotation RI21a.

The analysis target designation unit 230b outputs the information of the HE stained image I1a, the CT image G33, and the annotations RI21a and RI21e to the analysis unit 230d.

The analysis unit 230d uses the annotation RI21a as an input to detect the stained nuclear region. The analysis unit 230d classifies cell types using a model learned in advance by deep learning or the like. The analysis unit 230d classifies the cell type into one of undifferentiated, poorly differentiated, and well-differentiated. Alternatively, the analysis unit 230d classifies the cell type into one of stage 0, stage 1, stage 2, stage 3, and the like. For example, the analysis unit 230d executes HE staining separation, nuclear region detection, and cell type classification as a partial analysis process.

The analysis unit 230d receives the annotation RI21e as an input, detects the organ region, detects the tumor region from the inside of the organ region, and calculates the area of the tumor. For example, the analysis unit 230d executes organ region detection, tumor region detection, and tumor size as partial analysis processing.

The analysis unit 230d calculates the malignancy score of the tumor from the cell type and the tumor area. For example, the analysis unit 230d executes a tumor malignancy score calculation as a partial analysis process.

The analysis unit 230d distinguishes between the intermediate data of the analysis process (plural partial analysis processes) executed for the annotation RI21a and the intermediate data of the analysis process (plural partial analysis processes) executed for the annotation RI21e, and intermediates them. Register in the data table 220e.

The output unit 230e is a processing unit that outputs information on the progress of the analysis process executed based on the parameters in a visually recognizable state. For example, the output unit 230e generates an analysis result screen 80 as shown in FIG. 34 and displays it on the display unit 112.

FIG. 34 is a diagram showing an example of the analysis result screen 80 using the HE-stained image and the CT image. As shown in FIG. 34, icons IC81-1 to IC83-1, IC81-2 to IC83-2, and IC84 are arranged on the analysis result screen 80.

Icons IC81-1 to IC83-1 are icons corresponding to the analysis modules of HE staining separation, nuclear region detection, and cell type classification. Icons IC81-2 to IC83-2 are icons corresponding to analysis modules for visceral region detection, tumor region detection, and tumor size calculation. The icon IC84 is an icon corresponding to the analysis module for calculating the tumor malignancy score.

The icon ICn-1 (n = 81 to 83) is associated with an area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the “annotation RI21a”.

The icon ICn-2 (n = 81 to 83) is associated with an area for visually displaying the result (intermediate data) of the partial analysis process of the analysis module Mn for the “annotation RI21e”.

The icon IC84 is associated with an area for visually displaying the result of tumor malignancy score calculation.

Note that the icons ICn-1, ICn-2, and IC84 are provided with change buttons for changing parameters, respectively. For example, the user can change the parameters by pressing the change button of the icon IC.

[10. Hardware configuration]
The analysis apparatus according to each of the above-described embodiments is realized by, for example, a computer 1000 having a configuration as shown in FIG. 35. Hereinafter, the analysis device 100 according to the first embodiment will be described as an example. FIG. 35 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the analysis device. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each part of the computer 1000 is connected by a bus 1050.

The CPU 1100 operates based on the program stored in the ROM 1300 or the HDD 1400, and controls each part. For example, the CPU 1100 expands the program stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

The ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started, a program that depends on the hardware of the computer 1000, and the like.

The HDD 1400 is a computer-readable recording medium that non-temporarily records a program executed by the CPU 1100 and data used by the program. Specifically, the HDD 1400 is a recording medium for recording an information processing program according to the present disclosure, which is an example of program data 1450.

The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (for example, the Internet). For example, the CPU 1100 receives data from another device or transmits data generated by the CPU 1100 to another device via the communication interface 1500.

The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or mouse via the input / output interface 1600. Further, the CPU 1100 transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. Further, the input / output interface 1600 may function as a media interface for reading a program or the like recorded on a predetermined recording medium (media). The media is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Is.

For example, when the computer 1000 functions as the analysis device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes the analysis processing program loaded on the RAM 1200, thereby executing the acquisition unit 130a, the analysis target designation unit 130b, and the reception unit. Functions such as 130c, analysis unit 130d, and output unit 130e are realized. Further, the HDD 1400 stores an analysis processing program or the like according to the present disclosure. The CPU 1100 reads the program data 1450 from the HDD 1400 and executes the program, but as another example, these programs may be acquired from another device via the external network 1550.

[11. Conclusion]
The analysis device has a reception unit and an output unit. The reception unit receives the parameters used in the first analysis process of the image related to the pathology. The output unit can visually see the first result of the first analysis process executed based on the parameter and the second result of the second analysis process executed based on the first result. Output. The first result and the second result are image information. As a result, the user can confirm the influence of the analysis result by the parameter.

The reception unit further accepts the update of the parameter, and the output unit further receives the update of the parameter, and each time the parameter is updated, the first result and the second result are executed based on the updated parameter. Is output in a visually recognizable state. This allows the user to confirm the progress information of the analysis result caused by the parameter change.

The output unit outputs the first result, the second result, and the parameter in a visually recognizable state in association with each other. This allows the user to easily check the parameters used by the analysis module.

The output unit is in a visually recognizable state in which the first result executed based on the summary parameter in which a plurality of parameters are combined, the second result, and the summary parameter are associated with each other. Output with. As a result, the user can confirm the influence of the analysis result by the summary parameter. In addition, the user can roughly grasp a plurality of parameters with one summary parameter.

The analysis process includes a plurality of continuous analysis processes including the first analysis process and the second analysis process, and the output unit receives the plurality of analysis processes by the reception unit. The analysis process is performed based on the obtained parameters. The result of the analysis process is output in a visually recognizable state. As a result, the user can confirm the progress of the process output for each partial analysis process.

Further has an analysis unit that executes the plurality of analysis processes in order based on the order definition information that defines the order of the plurality of analysis processes. Based on the order definition information, the analysis unit identifies the next analysis process following the previous analysis process among the plurality of analysis processes, and inputs the result of the previous analysis process to the next. Executes the analysis process of. As a result, the partial analysis process of each analysis module can be executed in the order according to the order definition information (analysis module configuration information). In addition, the next analysis process can be executed using the result of the previous analysis process.

The output unit outputs the first result of the first analysis process executed on a plurality of different images and the second result of the second analysis process in a visually recognizable state. The plurality of images are pathological images stained with different reagents. The plurality of images include a bright field image and a fluorescence image. The plurality of images include a general-stained image and a special-stained image. The plurality of images include a pathological image and a medical image different from the pathological image. The medical image is an X-ray image, an endoscopic image or a microscope image. As a result, the user can compare and confirm the influence of the analysis result by the parameter on a plurality of different pathological images. For example, even if the user can set the optimum parameters for one pathological image, can the analysis result be obtained appropriately when the analysis process is executed for other pathological images using such parameters? It is possible to compare and examine whether or not.

The plurality of analysis processes include a process for detecting a cell nucleus, parameters of the process for detecting the cell nucleus include circularity, irregularity, and roundness, and the analysis unit includes the circularity and the irregularity. , At least one of the roundness is used to perform an analysis process for detecting cell nuclei from the image. Thereby, the cell nucleus can be detected by executing the analysis process using parameters such as circularity, irregularity, and roundness.

The plurality of analysis processes include a process of separating stains, a parameter of the process of separating stains includes a vector of a color space for performing the separation, and the analysis unit includes a vector of the color space. Is used to separate the region of the image into a stained region for detecting nuclei and a stained region for detecting membranes. As a result, the analysis process using the color space vector can be executed, and the dyeing separation can be executed.

The plurality of analysis processes include a process of detecting a plurality of cells and classifying them into a first cell group and a second cell group, and the parameters of the classifying process include a method of determining cells to be compared. The analysis unit calculates the distance between the first cell group and the second cell group. The analysis unit further executes a process of generating a histogram based on the relationship between the distance and the frequency between the first cell group and the second cell group. This makes it possible to easily confirm the relationship between the distance and frequency between the first cell group and the second cell group.

The plurality of analysis processes include a process of detecting a plurality of cell nuclei, and the analysis unit compares the distance between each cell nucleus and a predetermined region, and the cell nucleus whose distance is less than the threshold value corresponds to the predetermined region. Select as. This makes it possible to select the optimal cell nucleus.

In a diagnostic support system including a medical image acquisition device and software used for processing a medical image corresponding to an object imaged by the medical image acquisition device, the software (analysis program) relates to pathology. The first result of the first analysis process that accepts the parameters used in the first analysis process of the image and is executed based on the parameters, and the second analysis process that is executed based on the first result. Have the analyzer execute the process of outputting the second result in a visible state. As a result, the user can confirm the influence of the analysis result by the parameter.

Note that the effects described in this specification are merely examples and are not limited, and other effects may be obtained.

The present technology can also have the following configurations.
(1)
A reception section that accepts parameters used in the first analysis process of images related to pathology,
An output unit that visually outputs the first result of the first analysis process executed based on the parameters and the second result of the second analysis process executed based on the first result. An analyzer with and.
(2)
The analysis device according to (1) above, wherein the first result and the second result are image information.
(3)
The reception unit further accepts the update of the parameter,
The output unit is characterized in that each time the parameter is updated, the first result executed based on the updated parameter and the second result are output in a visually recognizable state. The analyzer according to (1) or (2) above.
(4)
The output unit is characterized in that the first result, the second result, and the parameter are associated with each other and output in a visually recognizable state (1), (2), or (3). The analyzer described.
(5)
The output unit is in a visually recognizable state in which the first result and the second result, which are executed based on the summary parameter in which a plurality of parameters are combined into one, are associated with the summary parameter. The analyzer according to (4) above, wherein the analyzer is characterized by outputting.
(6)
The analysis process includes a plurality of continuous analysis processes including the first analysis process and the second analysis process, and the output unit receives the plurality of analysis processes by the reception unit. The analysis apparatus according to any one of (1) to (5) above, wherein the result of the analysis process in which the analysis process is performed based on the obtained parameters is output in a visually recognizable state.
(7)
The analysis apparatus according to (6) above, further comprising an analysis unit that sequentially executes the plurality of analysis processes based on the order definition information that defines the order of the plurality of analysis processes.
(8)
Based on the order definition information, the analysis unit identifies the next analysis process following the previous analysis process among the plurality of analysis processes, and inputs the result of the previous analysis process to the next. The analysis apparatus according to (7) above, wherein the analysis process of the above is executed.
(9)
The output unit is characterized in that the first result of the first analysis process executed on a plurality of different images and the second result of the second analysis process are output in a visually recognizable state. The analyzer according to (1) above.
(10)
The analyzer according to (9) above, wherein the plurality of images are pathological images stained with different reagents.
(11)
The analysis apparatus according to (9) above, wherein the plurality of images include a bright-field image and a fluorescence image.
(12)
The analysis apparatus according to (9) above, wherein the plurality of images include a general-stained image and a special-stained image.
(13)
The analysis device according to (9) above, wherein the plurality of images include a pathological image and a medical image different from the pathological image.
(14)
The analyzer according to (13) above, wherein the medical image is an X-ray image, an endoscopic image, or a microscope image.
(15)
The plurality of analysis processes include a process for detecting a cell nucleus, parameters of the process for detecting the cell nucleus include circularity, irregularity, and roundness, and the analysis unit includes the circularity and the irregularity. The analysis apparatus according to (7), wherein an analysis process for detecting a cell nucleus from the image is executed using at least one of the roundness.
(16)
The plurality of analysis processes include a process of separating stains, a parameter of the process of separating stains includes a vector of a color space for performing the separation, and the analysis unit includes a vector of the color space. The analyzer according to (7) above, wherein the image region is separated into a stained region for detecting a nucleus and a stained region for detecting a membrane.
(17)
The plurality of analysis processes include a process of detecting a plurality of cells and classifying them into a first cell group and a second cell group, and the parameters of the classifying process include a method of determining cells to be compared. The analysis apparatus according to (7) above, wherein the analysis unit calculates the distance between the first cell group and the second cell group.
(18)
The analysis unit according to (17), wherein the analysis unit further executes a process of generating a histogram based on the relationship between the distance and the frequency between the first cell group and the second cell group. Analyzer.
(19)
The plurality of analysis processes include a process of detecting a plurality of cell nuclei, and the analysis unit compares the distance between each cell nucleus and a predetermined region, and the cell nucleus whose distance is less than the threshold value corresponds to the predetermined region. The analysis apparatus according to (7) above, wherein the analysis apparatus is selected as.
(20)
The analysis unit executes a process of detecting a region of the cell nucleus, a process of detecting a feature amount of the shape of the cell nucleus, a gating process based on the feature amount of the cell nucleus, and further executes a process of extracting tumor cells. The analyzer according to (7) above.
(21)
The computer
Accepts the parameters used in the first analysis process of images related to pathology,
A process of visually outputting the first result of the first analysis process executed based on the parameters and the second result of the second analysis process executed based on the first result. The analysis method to perform.
(22)
Computer,
A reception section that accepts parameters used in the first analysis process of images related to pathology,
An output unit that visually outputs the first result of the first analysis process executed based on the parameters and the second result of the second analysis process executed based on the first result. An analysis program to function as.
(23)
It is a diagnostic support system including a medical image acquisition device and software used for processing a medical image corresponding to an object imaged by the medical image acquisition device.
The software
Accepts the parameters used in the first analysis process of images related to pathology,
A process of visually outputting the first result of the first analysis process executed based on the parameters and the second result of the second analysis process executed based on the first result. A diagnostic support system that is executed by an analyzer.

100,200 Analytical device 110,210 Communication unit 111,211 Input unit 112,212 Display unit 120,220 Storage unit 120a, 220a Pathological image DB
120b, 220b Analysis module table 120c, 220c Analysis module configuration information 120d, 220d Parameter table 120e, 220e Intermediate data table 120f, 220f Outline Parameter conversion table 130, 230 Control unit 130a, 230a Acquisition unit 130b, 230b Analysis target specification unit 130c, 230c Reception unit 130d, 230d Analysis unit 130e, 230e Output unit

Claims

A reception section that accepts parameters used in the first analysis process of images related to pathology,
An output unit that visually outputs the first result of the first analysis process executed based on the parameters and the second result of the second analysis process executed based on the first result. An analyzer with and.
The analysis device according to claim 1, wherein the first result and the second result are image information.
The reception unit further accepts the update of the parameter,
The output unit is characterized in that each time the parameter is updated, the first result executed based on the updated parameter and the second result are output in a visible state. The analyzer according to claim 1.
The analysis device according to claim 1, wherein the output unit outputs the first result, the second result, and the parameter in a visually recognizable state in association with each other.
The output unit is in a visually recognizable state in which the first result and the second result, which are executed based on the summary parameter in which a plurality of parameters are combined into one, are associated with the summary parameter. The analysis device according to claim 4, wherein the analysis device is characterized by outputting.
The analysis process includes a plurality of continuous analysis processes including the first analysis process and the second analysis process, and the output unit receives the plurality of analysis processes by the reception unit. The analysis apparatus according to claim 1, wherein the result of the analysis processing in which the analysis processing is performed based on the obtained parameters is output in a visually recognizable state.
The analysis device according to claim 6, further comprising an analysis unit that sequentially executes the plurality of analysis processes based on the order definition information that defines the order of the plurality of analysis processes.
Based on the order definition information, the analysis unit identifies the next analysis process following the previous analysis process among the plurality of analysis processes, and inputs the result of the previous analysis process to the next. The analysis apparatus according to claim 7, wherein the analysis process of the above is executed.
The output unit is characterized in that the first result of the first analysis process executed on a plurality of different images and the second result of the second analysis process are output in a visually recognizable state. The analysis device according to claim 1.
The analysis device according to claim 9, wherein the plurality of images are pathological images stained with different reagents.
The analysis device according to claim 9, wherein the plurality of images include a bright-field image and a fluorescence image.
The analysis device according to claim 9, wherein the plurality of images include a general-stained image and a special-stained image.
The analysis device according to claim 9, wherein the plurality of images include a pathological image and a medical image different from the pathological image.
The analyzer according to claim 13, wherein the medical image is an X-ray image, an endoscopic image, or a microscope image.
The plurality of analysis processes include a process for detecting a cell nucleus, parameters of the process for detecting the cell nucleus include circularity, irregularity, and roundness, and the analysis unit includes the circularity and the irregularity. The analysis apparatus according to claim 7, wherein an analysis process for detecting a cell nucleus from the image is executed using at least one of the roundness.
The plurality of analysis processes include a process of separating stains, a parameter of the process of separating stains includes a vector of a color space for performing the separation, and the analysis unit includes a vector of the color space. The analyzer according to claim 7, wherein the region of the image is separated into a stained region for detecting a nucleus and a stained region for detecting a membrane.
The plurality of analysis processes include a process of detecting a plurality of cells and classifying them into a first cell group and a second cell group, and the parameters of the classifying process include a method of determining cells to be compared. The analysis device according to claim 7, wherein the analysis unit calculates the distance between the first cell group and the second cell group.
The analysis according to claim 17, wherein the analysis unit further executes a process of generating a histogram based on the relationship between the distance and the frequency between the first cell group and the second cell group. Device.
The plurality of analysis processes include a process of detecting a plurality of cell nuclei, and the analysis unit compares the distances between each cell nucleus and a predetermined region, and the cell nuclei having a distance less than the threshold value correspond to the predetermined region. The analysis apparatus according to claim 7, wherein the analysis apparatus is selected as.
The analysis unit executes a process of detecting a region of the cell nucleus, a process of detecting a feature amount of the shape of the cell nucleus, a gating process based on the feature amount of the cell nucleus, and further executes a process of extracting tumor cells. The analysis apparatus according to claim 7, wherein the analysis apparatus is used.
The computer
Accepts the parameters used in the first analysis process of images related to pathology,
A process of visually outputting the first result of the first analysis process executed based on the parameters and the second result of the second analysis process executed based on the first result. The analysis method to perform.
Computer,
A reception section that accepts parameters used in the first analysis process of images related to pathology,
An output unit that visually outputs the first result of the first analysis process executed based on the parameters and the second result of the second analysis process executed based on the first result. An analysis program to function as.
It is a diagnostic support system including a medical image acquisition device and software used for processing a medical image corresponding to an object imaged by the medical image acquisition device.
The software
Accepts the parameters used in the first analysis process of images related to pathology,
A process of visually outputting the first result of the first analysis process executed based on the parameters and the second result of the second analysis process executed based on the first result. A diagnostic support system that is executed by an analyzer.