JP2021007659A - Medical information processing device and program - Google Patents

Abstract

To assist in collecting a specimen suitable for diagnosis in performing biological examination to a subject, to prevent overlooking in findings, to reduce false positives and false negatives, and to reduce an examination time.SOLUTION: A control part of an information processing device extracts one or more areas, an area having pathologically the same characteristic within a lesion area included in a subject from a first image acquired by photographing the subject being one area. It also extracts, from a second image different from the first image acquired by photographing the subject, an area pathologically the same characteristic as the area extracted from the first image. Then, it associates the second image with information on the area extracted from a lesion area in the second image to output them.SELECTED DRAWING: Figure 4

Description

The present invention relates to a medical information processing device and a program.

In order to promote individualized treatment of cancer patients in the clinical setting, it is necessary to determine the treatment method such as surgical resection, chemotherapy, radiation therapy, etc. after diagnosing the type and advanced stage of the tumor. is there. In order to determine the treatment method, a needle is inserted under ultrasonic guidance to collect a tumor sample, and pathological analysis is performed.

In order to make the puncture work easier, an attachment for attaching a puncture needle to the ultrasonic probe and an ultrasonic probe dedicated to puncture have been put into practical use. By using these, it is possible to identify the path through which the puncture needle enters in the ultrasonic scan surface and display the approach prediction path as a line-shaped marker. Therefore, an operator such as a doctor can use these. Observe the tissue etc. existing on the marker with an ultrasonic tomographic image, and confirm that there are no blood vessels etc. on the marker and that the puncture needle is toward the lesion (collection target), and the patient's body The application position and angle of the ultrasonic probe with respect to the table can be searched.

In addition, a cross section corresponding to the ultrasonic scan surface according to the position and angle of the virtual puncture ultrasonic probe arbitrarily specified by the operator from the three-dimensional volume data collected by an image collecting device such as X-ray CT or MRI. There is also a system that realizes more accurate puncture by reconstructing an image with a puncture cross section and an information generator and displaying it on a monitor (see, for example, Patent Document 1).

As described above, there are many systems for guiding puncture biopsy, and the technique for collecting cells / tissues at a targeted site is becoming more sophisticated.

Here, one of the problems of pathological examination (biological examination) based on puncture / needle biopsy is intratumoral heterogeneity. One tumor is known to be formed from a heterogeneous cell population with diverse gene mutations, and Gerlinger et al. Detected multiple benign and poor prognosis gene expression patterns in the same tumor. (See, for example, Non-Patent Document 1). That is, in order to make an accurate diagnosis, it is necessary to collect cell / tissue samples at a location suitable for diagnosis in the tumor.

JP-A-2002-112998

Gerlinger M et al., “Intratumor Heterogeneity and Branched Evolution Revealed by Multiregion Sequencing”, N Engl J Med, 366: 883-92, 2012.

However, it is very difficult to accurately grasp from which part of the tumor having non-uniform properties it is appropriate to collect a sample suitable for diagnosis, and a series of operations is a doctor's past. It relies heavily on the knowledge and technology of.

In addition, biopsy is generally performed under the guidance of an ultrasonic diagnostic device, but while biopsy under the guidance of an ultrasonic diagnostic device is excellent in real time, especially in the case of manual scanning. Has the problem that it is difficult to instantly grasp the whole picture of the lesion.

An object of the present invention is to support the collection of a sample suitable for diagnosis when performing a biopsy on a subject. In addition, a region having the same pathological characteristics is extracted in the lesion region using a higher-definition and highly bird's-eye view image of the entire lesion as the first image, and based on the result, the first image is used from the second image. By extracting a region that has the same pathological characteristics as the region extracted in, it is easy to instantly grasp the whole picture of the lesion, prevent oversight of findings, reduce false positives and false negatives, and reduce examination time. The purpose is to plan.

In order to solve the above problem, the medical information processing apparatus according to claim 1 is
The first image obtained by photographing the subject is analyzed, and the region having the same pathological characteristics within the lesion region contained in the subject from the first image is regarded as one region or one or more. The first extraction means to extract multiple regions and
A second image different from the first image obtained by photographing the subject is analyzed, and the region has the same pathological characteristics as the region extracted from the second image by the first extraction means. A second extraction means to extract the area,
An output means that outputs the second image in association with information about a region extracted by the second extraction means, and
To be equipped.

The invention according to claim 2 is the invention according to claim 1.
The first extraction means is one or more of the lesion areas of the first image by a region extraction process associated with a pathological test result determined by histochemical staining, tumor marker analysis, amino acid analysis, or genetic analysis. Extract the area.

The invention according to claim 3 is the invention according to claim 2.
The first extraction means extracts the one or a plurality of regions from the lesion region of the first image by using a discriminator trained by associating an image feature amount with a pathological examination result.

The invention according to claim 4 is the invention according to any one of claims 1 to 3.
The second extraction means extracts a region from the second image by the same method as the first extraction means, and extracts from the pathological features of the region extracted from the first image and the second image. By comparing or collating the pathological characteristics of the extracted regions, a region having the same pathological characteristics as the region extracted by the first extraction means is extracted.

The invention according to claim 5 is the invention according to any one of claims 1 to 4.
When a plurality of regions are extracted by the first extraction means, a specific means for specifying a target region to be used for extracting the region by the second extraction means from the extracted plurality of regions is provided.
The second extraction means extracts from the second image a region extracted by the first extraction means that has the same pathological characteristics as the target region specified by the specific means.

The invention according to claim 6 is the invention according to claim 5.
The specific means is a target area based on a user's area designation operation, a predetermined priority for a pathological examination result, a priority based on the effectiveness of diagnosis, and / or a priority preset by the user. To identify.

The invention according to claim 7 is the invention according to any one of claims 1 to 6.
Has a display means
The output means outputs the information about the second image and the region extracted by the second extraction means to the display means.
The display means displays the second image and information about the region extracted by the second extraction means side by side or in an overlapping manner.

The invention according to claim 8 is the invention according to claim 7.
The output means further outputs information about the first image and / or the region extracted by the first extraction means to the display means.
The display means displays side by side information about the region extracted by the first extraction means and information about the region extracted by the second extraction means, or displays information about the region extracted by the first extraction means. The image superimposed on the first image and the information about the region extracted by the second extraction means are displayed side by side on the second image, or by the second extraction means superimposed on the second image. An image in which the information on the extracted region is further superimposed with the information on the region extracted by the first extraction means is displayed.

The invention according to claim 9 is the invention according to claim 7 or 8.
A calculation means for calculating the insertion route of the device for biopsy diagnosis based on the position of the region extracted by the second extraction means is provided.
The output means outputs the information of the insertion path calculated by the calculation means, and outputs the information.
The display means displays the output information of the insertion route.

The invention according to claim 10 is the invention according to any one of claims 1 to 9.
When the region is not extracted by the second extraction means, the notification means for notifying that the region is not extracted is provided.

The invention according to claim 11 is the invention according to any one of claims 1 to 10.
The first image is a mammography image, an MRI image, an ultrasonic image, or a tomosynthesis image.
The second image is an ultrasonic image.

The program of the invention according to claim 12
Computer,
The first image obtained by photographing the subject is analyzed, and the region having the same pathological characteristics within the lesion region contained in the subject from the first image is regarded as one region or one or more. First extraction means to extract multiple regions,
A second image different from the first image obtained by photographing the subject is analyzed, and the region has the same pathological characteristics as the region extracted from the second image by the first extraction means. Second extraction means to extract the region,
An output means that outputs the second image in association with information about the area extracted by the second extraction means.
To function as.

According to the present invention, it is possible to support the collection of a sample suitable for diagnosis when performing a biopsy on a subject. In addition, a region having the same pathological characteristics is extracted in the lesion region using a higher-definition and highly bird's-eye view image of the entire lesion as the first image, and based on the result, the first image is used from the second image. By extracting a region that has the same pathological characteristics as the region extracted in, it is easy to instantly grasp the whole picture of the lesion, prevent oversight of findings, reduce false positives and false negatives, and reduce examination time. Is possible.

It is a figure which shows the whole structure of the biopsy support system in embodiment of this invention. It is a block diagram which shows the functional structure of the information processing apparatus of FIG. It is a flowchart which shows the flow of the pre-learning process executed by the control part of FIG. It is a flowchart which shows the flow of the area information acquisition processing executed by the control part of FIG. It is a flowchart which shows the flow of the image analysis processing executed by the control unit of FIG. It is a figure which shows an example of the area information of a 1st image and a 1st image. It is a figure which shows the display example of the area information of the 2nd image and the 2nd image. It is a figure which shows the display example when the area information of a 1st image is displayed together with the area information of a 2nd image and a 2nd image. It is a flowchart which shows the flow of the puncture route display processing executed by the control part of FIG. It is a figure for demonstrating the calculation method of a puncture path.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[Configuration of biopsy support system 100]
First, the configuration will be described.
FIG. 1 shows the overall configuration of the biopsy support system 100 according to the present embodiment.
As shown in FIG. 1, in the biopsy support system 100, the first image capturing device 1, the second imaging device 2, and the information processing device (medical information processing device) 3 are connected to a LAN (Local Area Network) or the like. It is configured to be connected via a communication network N. Each device constituting the biopsy support system 100 conforms to HL7 (Health Level Seven) and DICOM (Digital Image and Communications in Medicine) standards, and communication between each device is performed according to HL7 and DICOM.

The first imaging device 1 includes, for example, an X-ray imaging device (DR (including those capable of tomosynthesis imaging), CR, CT, fluoroscopy device, angio device, etc.), an ultrasonic diagnostic device (US), MRI, PET. Or, it is a modality such as PET-CT, and a medical image (referred to as a first image) is generated by photographing a subject. The first image capturing apparatus 1 writes the incidental information (patient information, examination information, image ID, slice number, etc.) in the header of the image file of the medical image in accordance with the DICOM standard, thereby adding the incidental information to the first image. It is attached and transmitted to the information processing device 3.
In the present embodiment, the first image capturing device 1 is a breast X-ray imaging device, and the first image is a mammography image capable of observing the entire breast from a bird's-eye view, but the present invention is limited to this. It's not something.

The second imaging device 2 includes, for example, an X-ray imaging device (DR (including those capable of tomosynthesis imaging), CR, CT, fluoroscopy device, angio device, etc.), an ultrasonic diagnostic device (US), MRI, PET. , PET-CT and the like, and a medical image (called a second image) is generated by photographing a subject. The second image capturing apparatus 2 writes the incidental information (patient information, examination information, image ID, slice number, etc.) in the header of the image file of the medical image in accordance with the DICOM standard, thereby transmitting the incidental information to the second image. It is attached and transmitted to the information processing device 3.
In the present embodiment, the second image capturing device 2 is a handheld type ultrasonic diagnostic device, and the second image is an ultrasonic image of the breast, but the present invention is not limited to this.

The information processing device 3 analyzes the first image transmitted from the first image capturing device 1 and the second image transmitted from the second imaging device 2, and provides support information in the lesion region to support the biopsy. It is a computer device that outputs area information.
FIG. 2 shows the functional configuration of the information processing device 3. As shown in FIG. 2, the information processing device 3 includes a control unit 31, an operation unit 32, a display unit 33, a communication unit 34, a storage unit 35, and the like, and each unit is connected by a bus 36. ..

The control unit 31 is composed of a CPU, a ROM, a RAM, and the like, and comprehensively controls the processing operation of each unit of the information processing device 3. Specifically, the CPU reads various processing programs stored in the ROM, develops them in the RAM, and performs various processing in cooperation with the programs.
The control unit 31 functions as a first extraction means, a second extraction means, an output means, a specific means, and a calculation means by executing the area information acquisition process described later. In addition, it functions as a notification means in cooperation with the display unit 33.

The operation unit 32 includes a keyboard equipped with cursor keys, character / number input keys, various function keys, and a pointing device such as a mouse, and controls operation signals input by key operations on the keyboard and mouse operations. Output to unit 31. When the operation unit 32 is composed of a touch panel laminated on the display unit 33, the operation signal corresponding to the position of the touch operation by the user's finger or the like is output to the control unit 31.

The display unit 33 is configured to include a monitor such as an LCD (Liquid Crystal Display), and displays various screens according to an instruction of a display signal input from the control unit 31. The number of monitors may be one or a plurality.

The communication unit 34 is configured by a network interface or the like, and transmits / receives data to / from an external device connected via the communication network N.

The storage unit 56 is composed of a hard disk, a non-volatile semiconductor memory, or the like, and stores various data.

[Flow of biopsy using biopsy support system 100]
Next, the flow of the biopsy using the biopsy support system 100 will be described.
In this embodiment, a case where a biopsy of breast cancer is performed will be described as an example.

First, an X-ray image of a subject (the breast of a subject) is taken by the first image capturing device 1, a mammography image as the first image is generated, and the mammography image is transmitted to the information processing device 3.
An examiner (user) such as a doctor obtains a bird's-eye view of the lesion area from a mammography image, aims at the lesion area, and takes an image with the second image capturing device 2 which is an ultrasonic diagnostic device to perform a second image. The ultrasonic image is generated and transmitted to the information processing apparatus 3.
In the information processing apparatus 3, the area information acquisition process (see FIG. 4) is executed based on the input first image and the second image, and the lesion area detected in the second image is further pathological. It is divided according to the features, and the area information (area position information, area feature information) related to each divided area is output.
An examiner such as a doctor identifies an area to be punctured in the lesion area based on the second image and area information output from the information processing device 3, and performs a biopsy.

[Pre-learning of classifier]
Here, in the area information acquisition process, a pre-learned discriminator for estimating the pathological feature in the lesion area from the image feature amount in the lesion area of the first image and the second image is used. Therefore, first, the pre-learning process for generating the pre-learned classifier will be described.

FIG. 3 is a flowchart showing the flow of the pre-learning process. The pre-learning process of FIG. 3 is executed in collaboration with the program stored in the control unit 31 and the storage unit 35. By executing the pre-learning process using the input image as the first image, a discriminator for estimating the pathological feature in the lesion area from the image feature amount of the lesion area in the first image is generated. By executing the pre-learning process using the input image as the second image, it is possible to generate a discriminator for estimating the pathological feature in the lesion area from the image feature amount of the lesion area in the second image.
In addition, this pre-learning process may be performed by an external computer device or the like, and the obtained classifier may be stored in the storage unit 35.

First, the control unit 31 accepts inputs of a plurality of (large amount) input images (step S1).

Next, the control unit 31 extracts the ROI (region of interest) for each input image (step S2).
In this embodiment, the ROI is the lesion area where the lesion is present. The ROI may be extracted using a CAD (Computer-Aided Diagnosis / Detection) system, or specified by the user using the operation unit 32 from the input image displayed on the display unit 33. You may.

Next, the control unit 31 extracts the image feature amount in each ROI extracted from each input image (step S3).
In step S3, for example, the amount of image features such as density, shape, texture, and multi-resolution features based on wavelet transform is extracted for each pixel. One type of image feature amount may be extracted, or a plurality of types may be extracted.

Next, the control unit 31 uses the result of the pathological examination performed on the sample taken out from the position corresponding to each ROI of each input image of the subject who is the subject of each input image as a correct label in each ROI. It is stored in the storage unit 35 in association with the image feature amount extracted in (step S4).
For example, each input image is displayed on the display unit 33, the correct answer label for each ROI is input according to the user operation, and the storage unit 35 associates the image feature amount extracted from the ROI with the input correct answer label. Remember.
Further, for example, from the pathological image of the sample taken out from the position corresponding to the ROI, a feature map for each pixel (a map showing the value of the pathological test result) is created as a correct answer label, and each pixel is extracted in the ROI. It may be stored in the storage unit 35 in association with the image feature amount to be generated.

Here, for example, any of the following can be mentioned as the result of the pathological examination which becomes the correct answer label.
-Various molecular marker values (histochemical staining, tumor marker analysis, amino acid analysis, gene analysis (for example, microarray analysis, polymerase chain reaction (PCR), gene (DNA / RNA) sequence analysis, etc.) pathway activation, noncoding RNA, multiple RNAs, single nucleotide polymorphism, copy number polymorphism, epigenetic polymorphism, etc.) The value of the pathological test result determined by)
-Pathological diagnosis name determined by the above-mentioned molecular marker value or morphological diagnosis-Information indicating whether it is benign or malignant determined by the above-mentioned molecular marker value or morphological diagnosis-The above-mentioned molecular marker value or morphology Malignancy of lesions determined by diagnosis ・ Stage classification determined by the above-mentioned molecular marker values and morphological diagnosis

In the pathological examination, if the extracted sample has pathologically non-uniform characteristics, different test results can be obtained depending on the position in the sample. Therefore, different correct labels may be associated with one ROI depending on the position.

Then, the control unit 31 associates the image feature amount with the correct answer label and learns it by machine learning to generate a classifier (step S5), and ends the pre-learning process.
As the machine learning algorithm, for example, random forest, decision tree, support vector machine, logistic regression, k-nearest neighbor method, topic model, various deep learning network models (Alex Net, Google Net, etc.) can be used. In the case of deep learning, since the image feature amount in step S3 is automatically extracted, it is not performed independently.

By the above pre-learning process, it is possible to generate a discriminator for estimating the pathological feature from the image feature amount of each lesion region of the first image and the second image.

[Area information acquisition process]
Hereinafter, the area information acquisition process executed by the information processing apparatus 3 will be described.
FIG. 4 is a flowchart showing the flow of the area information acquisition process. The area information acquisition process shown in FIG. 4 is executed in cooperation with the program stored in the control unit 31 and the storage unit 35.

First, the control unit 31 accepts the input of the first image (step S11).
Next, the control unit 31 executes the image analysis process with the first image as the target image (step S12).

FIG. 5 is a flowchart showing the flow of the image analysis process. The image analysis process is executed in collaboration with the program stored in the control unit 31 and the storage unit 35.

First, the ROI is extracted from the target image (step S101).
The ROI is the lesion area where the lesion is present. The ROI may be extracted by using CAD, or the user may specify the ROI from the target image displayed on the display unit 33 by using the operation unit 32.

When the ROI, that is, the lesion area is not extracted by the ROI extraction by CAD, the control unit 31 displays, for example, on the display unit 33 that the lesion area has not been extracted, and ends the process for the user. Have them choose whether to manually specify the ROI or move on to the next image analysis. When it is selected to end the process, the control unit 31 ends the image analysis process and also ends the area information acquisition process. When it is selected to move to the analysis of the next image, the control unit 31 waits for the input of the next target image and extracts the ROI for the next target image. Further, even when the user specifies the ROI using the operation unit 32, for example, an end button for instructing the end of processing, a transition button for instructing the transition to the analysis of the next image, and the like are provided to provide a lesion area. It is preferable to be able to end the process or move on to the analysis of the next image if is not present.

Next, the control unit 31 extracts the image feature amount in the ROI (step S102).
In step S102, for example, an image feature amount such as density, shape, texture, and multi-resolution feature based on wavelet transform is extracted for each pixel. One type of image feature amount may be extracted, or a plurality of types may be extracted.

Next, the control unit 31 performs feature quantity analysis on the ROI using a pre-learned classifier, and acquires information indicating a pathological feature for each pixel in the ROI (step S103).
The classifier used in step S103 is the classifier generated by the pre-learning process described above. The information indicating the pathological characteristics includes the above-mentioned various molecular marker values, pathological diagnosis names, information indicating whether the disease is benign or malignant, the degree of malignancy, stage classification, and the like.

Then, the control unit 31 outputs information indicating the pathological features of each pixel in the ROI as an analysis result (step S104), and proceeds to step S13 of FIG.

In step S13 of FIG. 4, the control unit 31 extracts one or a plurality of regions from within the ROI, with regions having the same pathological characteristics as one region, based on the analysis result of the image analysis process. (Step S13). If the entire ROI is a single region with the same pathological characteristics, the entire ROI is extracted as one region, and if the ROI contains multiple regions with multiple pathological characteristics, multiple regions are extracted. Extract the area of.
Here, "having the same pathological characteristics" is not limited to, for example, when various molecular marker values are used as information indicating pathological characteristics, the values are not completely the same. , The range in which the values can be regarded as pathologically the same (for example, the range in which the diagnosis name, stage classification, etc. are the same) is included.

Next, the control unit 31 outputs the area information of the area extracted from the ROI (step S14).
For example, the region information of each region extracted from the first image and the ROI of the first image (for example, the position information of each extracted region, the region feature information indicating the pathological characteristics of each region (for example, various types) Molecular marker value, pathological diagnosis name, malignancy, stage classification, etc.)) are output to the display unit 33 and displayed. Further, the first image and the area information may be output to a printer or the like and printed on paper.

FIG. 6 shows an example of the first image and area information output in step S14. As shown in FIG. 6, in step S14, for example, the annotation A1 indicating the ROI (lesion region) extracted in the first image is displayed, and the region information of the region extracted from the ROI is displayed in the vicinity thereof. Is output.

Here, when a plurality of regions are extracted from the ROI in step S13, the control unit 31 may specify a target region used for region extraction in the second image from the extracted plurality of regions. Thereby, the target area used for the area extraction in the second image can be limited to the important ones.
For example, the target area can be specified based on one or more of the following.
-Preset priorities (for example, molecular marker values) for pathological features such as the above-mentioned various molecular marker values, pathological diagnosis names, lesion malignancy, stage classification (stage), and benign or malignant The larger the area, the higher the priority, etc.)
-Priority based on the effectiveness of the diagnosis (priority is given based on whether the obtained pathological features contribute to treatment, for example, whether there is a therapeutic drug, whether chemotherapy or radiation therapy is effective, etc. )
-Priority other than the above, which is arbitrarily set by the user-User area designation by the operation unit 32

Next, the control unit 31 waits for the input of the second image from the second image capturing device 2. When the second image is input (step S15; YES), the control unit 31 executes the image analysis process with the second image as the target image (step S16).
Since the image analysis process is the same as that described with reference to FIG. 5, the description is incorporated.

Next, the control unit 31 extracts from the second image a region having the same pathological characteristics as each region extracted from the ROI of the first image, based on the analysis result of the image analysis process (step S17). ..
In step S17, based on the analysis result of the image analysis process, one or a plurality of regions are extracted in the ROI of the second image with the regions having the same pathological characteristics as one region, and the extracted regions are extracted. The region feature information is compared or collated with the region feature information of the region extracted in the ROI of the first image, and a region having the same pathological characteristics as each region extracted from the first image is extracted.

In step S17 as well as in step S14, the control unit 31 may specify a target area to be used for subsequent processing (for example, display or the like).
Further, in step S17, when a region having the same pathological characteristics as each region extracted from the ROI of the first image is not extracted from the ROI of the second image, the control unit 31 informs that fact. It is displayed on the display unit 33.

Next, the control unit 31 outputs the region information (region position information, region feature information) of each region extracted from the ROI of the second image and the second image in association with each other (step S18).
Here, the second image and the area information are associated and output to the display unit 33.

Next, the control unit 31 causes the display unit 33 to display the area information of the second image and the area extracted from the ROI of the second image in association with each other (step S19).

7 (a) to 7 (c) are diagrams showing an example of the second image and area information displayed in step S19.
In step S19, for example, as shown in FIG. 7A, the second image 331 and the area information 332 are displayed side by side. In the second image 331, for example, the annotation A2 indicating the extracted ROI (lesion region) is displayed. As the area information 332, for example, the area position information 332a and the area position information 332a, which are extracted in the ROI indicated by the annotation A2 in the second image and indicate the positions of each area having the same pathological characteristics, are displayed. Region feature information 332b and the like indicating the pathological features of each region are displayed.
Alternatively, as shown in FIG. 7B, the area information 332 (for example, the area position information 332a) may be superimposed and displayed on the second image 331. Alternatively, the area information 332 may be displayed separately.
Alternatively, as shown in FIG. 7C, only the area position information 332a of the designated area among the areas extracted from the ROI may be superimposed and displayed on the second image 331.
Alternatively, although not shown, only the area information 332 may be independently displayed on the display unit 33.

In this way, within the lesion area of the second image, the position of the area having the same pathological characteristics as each area obtained by dividing the lesion area of the first image into each pathological feature and Since the pathological features are displayed, the user can insert the needle anywhere in the lesion area of the second image to collect a diagnostically suitable sample for the lesion area detected in the first image. It is possible to recognize whether it can be done without the knowledge and experience of the user. As a result, it becomes possible to collect a sample suitable for diagnosis, and false positives and false negatives can be reduced.

Alternatively, the area information in the lesion area of the first image may also be displayed.
For example, as shown in FIG. 8A, the area information 332 of the second image and the area information 333 of the first image may be displayed side by side with the second image 331. In FIG. 8A, only the area feature information is shown as the area information 333, but the area position information may be separately displayed side by side.
Alternatively, as shown in FIG. 8B, the first image 334 with the area position information 333a superimposed, the second image 331 with the area position information 332a superimposed, and the second image area information 332 are displayed side by side. You may do it. Further, the area information 333 of the first image 334 may be displayed together.
As a result, the regions having pathologically important features in the lesion region extracted in the first image are visible in the second image, or each region extracted from the lesion region in the first image is in the second image. It is possible to confirm where to correspond.

Next, the control unit 31 determines whether or not the operation unit 32 has instructed the display of the insertion route (referred to as the puncture route) of the device for biopsy diagnosis (referred to as the puncture device) (step S20).
When it is determined that the display of the puncture route is not instructed (step S20; NO), the control unit 31 proceeds to step S22.

When it is determined that the display of the puncture route is instructed (step S20; YES), the control unit 31 executes the puncture route display process (step S21).
FIG. 9 is a flowchart showing the puncture route display process executed in step S21. The puncture route display process is executed in cooperation with the program stored in the control unit 31 and the storage unit 35.

First, the control unit 31 receives the designation of the puncture target area (the target area for performing the biopsy diagnosis (collecting the tissue)) by the operation unit 32 (step S201).
For example, the user looks at the area information 332 displayed on the display unit 33 and identifies the puncture target area from the areas extracted from the ROI. As described above, in the ROI, the priority set in advance for the pathological examination result, the priority based on the effectiveness of the diagnosis, the priority other than the above and arbitrarily set in advance by the user, etc. The control unit 31 may automatically specify the puncture target area from the area extracted from.

Next, the control unit 31 calculates the puncture route (step S202).
For example, in the case of puncturing the breast, as shown in FIG. 10A, the puncture route to the puncture target area R is calculated so as to be the shortest, parallel to the chest wall, and in the long axis direction of the puncture target area R. To do. For example, as shown in FIG. 10B, the distance between the center O1 of the puncture target region R and its body surface (probe center) O2 in the second image is y, and the distance between the body surface O2 and the insertion position O3 of the puncture device. Let x be, and since x and y are known, the puncture angle θ and the puncture path z can be calculated from x and y.

Next, the control unit 31 displays the calculated puncture route information on the display unit 33 (step S203). For example, the information on the puncture route is output to the display unit 33 and displayed on the second image. As a result, the user can recognize the puncture route to the puncture target area R, and can puncture the target puncture target area with high accuracy regardless of knowledge or experience.
When the display of the puncture route information is completed, the control unit 31 proceeds to step S22.

In step S22, the control unit 31 determines whether or not the second image has been received from the second image capturing device 2 by the communication unit 34 (step S22).
When it is determined that the second image has been received (step S22; YES), the control unit 31 returns to step S16 and repeatedly executes the processes of steps S16 to S22.
If it is determined in step S22 that the second image has not been received (step S22; NO), the control unit 31 ends the area information acquisition process of the second image.

As described above, according to the information processing apparatus 3, the control unit 31 analyzes the first image acquired by photographing the subject, and within the lesion region included in the subject from the first image. One or more regions are extracted, with regions having the same pathological characteristics as one region. In addition, the control unit 31 analyzes a second image different from the first image acquired by photographing the subject, and has a region having the same pathological characteristics as the region extracted from the first image. Is extracted from the second image. Then, the second image and the information about the region extracted from the lesion region of the second image are associated and output.

Therefore, the user can easily recognize where in the second image the region having a pathologically important feature is in the lesion region detected in the first image, and thus in the lesion region of the second image. It is possible to recognize where the needle should be inserted to collect a sample suitable for diagnosis from the lesion area detected in the first image, regardless of the user's knowledge or experience. Become. As a result, it becomes possible to collect a sample suitable for diagnosis, and false positives and false negatives can be reduced.
In addition, a region having the same pathological features is extracted from the lesion region in the first image, with an image having higher definition, a high bird's-eye view of the entire lesion, and an image that makes it easy to instantly capture the entire lesion as the first image. Based on the results, by extracting from the second image a region that has the same pathological characteristics as the region extracted in the first image, it becomes easier to instantly grasp the entire image of the lesion and prevent oversight of findings. , False positives and false negatives can be reduced, and test time can be reduced.

The description in the above embodiment is an example of a suitable biopsy support system according to the present invention, and is not limited thereto.

For example, in the above embodiment, the case where the first image is a mammography image and the second image is an ultrasonic image has been described as an example, but as described above, the first image and the second image are limited to these. It's not a thing. The first image is preferably an image that allows a bird's-eye view of the entire inspection site (including the entire inspection site). For example, a mammography image, an MRI image, and an ultrasound image (particularly, an automatic scanning type ultrasound). Ultrasound images obtained by a diagnostic device), tomosynthesis images, and the like are preferable. As the second image, an image used together with the biopsy, for example, an ultrasonic image (particularly, an ultrasonic image obtained by a handheld type ultrasonic image) is preferable.

Further, in the above embodiment, the first image and the second image have been described as images having different modality, but may be images having different shooting times. For example, if the above-mentioned area information acquisition process is executed with the past image taken in the past examination as the first image and the current image taken in the current examination as the second image, the lesion area detected in the past image is included. It is possible to extract a region having the same pathological characteristics as the region having a predetermined characteristic from the current image, and output information on the position of the extracted region and the pathological characteristics.

Further, in the above embodiment, the subject is described as a breast, but the present invention is not limited to this, and other parts may be used.

Further, in the above description, an example in which a hard disk, a non-volatile memory of a semiconductor, or the like is used as a computer-readable medium of the program according to the present invention has been disclosed, but the present invention is not limited to this example. As another computer-readable medium, a portable recording medium such as a CD-ROM can be applied. A carrier wave is also applied as a medium for providing data of a program according to the present invention via a communication line.

In addition, the detailed configuration and detailed operation of each device constituting the biopsy support system 100 can be appropriately changed without departing from the spirit of the present invention.

100 Biopsy support system 1 First imaging device 2 Second imaging device 3 Information processing device 31 Control unit 32 Operation unit 33 Display unit 34 Communication unit 35 Storage unit 36 Bus

Claims (12)

The first image obtained by photographing the subject is analyzed, and the region having the same pathological characteristics within the lesion region contained in the subject from the first image is regarded as one region or one or more. The first extraction means to extract multiple regions and
A second image different from the first image obtained by photographing the subject is analyzed, and the region has the same pathological characteristics as the region extracted from the second image by the first extraction means. A second extraction means to extract the area,
An output means for outputting the second image in association with information about a region extracted by the second extraction means.
Medical information processing device equipped with. The first extraction means is one or more of the lesion areas of the first image by a region extraction process associated with a pathological test result determined by histochemical staining, tumor marker analysis, amino acid analysis, or genetic analysis. The medical information processing apparatus according to claim 1, wherein an area is extracted. The first extraction means according to claim 2, wherein the one or a plurality of regions are extracted from the lesion region of the first image by using a discriminator trained by associating an image feature amount with a pathological test result. Medical information processing device. The second extraction means extracts a region from the second image by the same method as the first extraction means, and extracts from the pathological features of the region extracted from the first image and the second image. Any one of claims 1 to 3 for extracting a region having the same pathological characteristics as the region extracted by the first extraction means by comparing or collating the pathological characteristics of the regions. The medical information processing apparatus described in. When a plurality of regions are extracted by the first extraction means, a specific means for specifying a target region to be used for extracting the region by the second extraction means from the extracted plurality of regions is provided.
The second extraction means claims 1 to 1 to extract a region extracted from the second image by the first extraction means that has the same pathological characteristics as the target region specified by the specific means. The medical information processing apparatus according to any one of 4. The specific means is a target area based on a user's area designation operation, a predetermined priority for a pathological examination result, a priority based on the effectiveness of diagnosis, and / or a priority preset by the user. The medical information processing apparatus according to claim 5. Has a display means
The output means outputs the information about the second image and the region extracted by the second extraction means to the display means.
The medical information processing apparatus according to any one of claims 1 to 6, wherein the display means displays the second image and information about the region extracted by the second extraction means side by side or in an overlapping manner. The output means further outputs information about the first image and / or the region extracted by the first extraction means to the display means.
The display means displays side by side information about the region extracted by the first extraction means and information about the region extracted by the second extraction means, or displays information about the region extracted by the first extraction means. The image superimposed on the first image and the information about the region extracted by the second extraction means are displayed side by side on the second image, or by the second extraction means superimposed on the second image. The medical information processing apparatus according to claim 7, wherein an image in which the information on the extracted region is further superimposed with the information on the region extracted by the first extraction means is displayed. A calculation means for calculating the insertion route of the device for biopsy diagnosis based on the position of the region extracted by the second extraction means is provided.
The output means outputs the information of the insertion path calculated by the calculation means, and outputs the information.
The medical information processing device according to claim 7 or 8, wherein the display means displays the output information of the insertion path. The medical information processing apparatus according to any one of claims 1 to 9, further comprising a notifying means for notifying that the region has not been extracted when the region is not extracted by the second extraction means. The first image is a mammography image, an MRI image, an ultrasonic image, or a tomosynthesis image.
The medical information processing apparatus according to any one of claims 1 to 10, wherein the second image is an ultrasonic image. Computer,
The first image obtained by photographing the subject is analyzed, and the region having the same pathological characteristics within the lesion region contained in the subject from the first image is regarded as one region or one or more. First extraction means to extract multiple regions,
A second image different from the first image obtained by photographing the subject is analyzed, and the region has the same pathological characteristics as the region extracted from the second image by the first extraction means. Second extraction means to extract the region,
An output means that outputs the second image in association with information about the area extracted by the second extraction means.
A program to function as.

Images