JP2021171117A - Similar case retrieval program, similar case retrieval method, and similar case retrieval system - Google Patents

Abstract

To allow user's determination to be reflected to a retrieval result of a similar image.SOLUTION: A retrieval part retrieves an image similar to a first image obtained by photographing patient's lungs from a prescribed storage area. A display control part aligns respective images obtained by retrieval in descending order of similarity to the first image, together with a mark for showing whether a prescribed flag is attached to each image or not, and allows a display device to display the result.SELECTED DRAWING: Figure 4

Description

The present invention relates to a similar case search program, a similar case search method, and a similar case search system.

Conventionally, there is known a case search system that presents a CT image of the lungs of a patient with diffuse lung disease to a doctor to support a medical examination.

innnavinet, "Fujifilm expands the target diseases of the similar case search system" SYNAPSE Case Match "that supports image diagnosis using artificial intelligence ~ Case search function of" diffuse lung disease "that is difficult to diagnose Equipped with "~ (https://www.innervision.co.jp/products/release/20170414) FUJIFILM, "Successful joint development of diagnosing support technology for interstitial pneumonia using FUJIFILM and Kyoto University AI technology" (https://www.fujifilm.co.jp/corporate/news/articleffnr_1417.html)

However, in the conventional technique, there is a problem that it may be difficult to reflect the judgment of the user in the search result of the similar image. For example, in the conventional technique, a machine learning model searches for images similar to each other. On the other hand, the criteria for determining similar images for obtaining an optimal diagnosis may differ depending on the doctor performing the examination. The users of the case search system are mainly doctors.

In one aspect, it is intended to reflect the user's judgment in the search results of similar images.

In one embodiment, the similar case search program causes a computer to perform a process of searching a predetermined storage area for an image similar to a first image of a patient's lungs. The similar case search program displays each of the images obtained by the search on the display device in descending order of similarity with the first image, together with a mark indicating whether or not each image is given a predetermined flag. Let the computer perform the process to make it.

In one aspect, the doctor's judgment can be reflected in the search results of similar images.

FIG. 1 is a diagram showing a configuration example of a similar case search system. FIG. 2 is a diagram showing an example of the data structure of the patient DB. FIG. 3 is a diagram showing an example of the data structure of the case DB. FIG. 4 is a diagram showing an example of a screen for displaying search results. FIG. 5 is a flowchart showing the flow of the search process. FIG. 6 is a flowchart showing the flow of the update process. FIG. 7 is a diagram illustrating a hardware configuration example.

Hereinafter, examples of the similar case search program, the similar case search method, and the similar case search system according to the present invention will be described in detail with reference to the drawings. The present invention is not limited to this embodiment. In addition, each embodiment can be appropriately combined within a consistent range.

The similar case search system is a system that supports a doctor's examination by searching for patient images stored in a DB and displaying the images obtained by the search. The image here is a medical image such as a CT image. For example, the similar case search system 1 can acquire an image similar to the image of the patient under examination from the DB and present information on the disease associated with the acquired image to the doctor. This allows the doctor to make a medical examination with reference to the presented image and information about the disease. The user of the similar case search system is, for example, a doctor.

FIG. 1 is a diagram showing a configuration example of a similar case search system. As shown in FIG. 1, the similar case search system 1 has a similar case search device 10 and a display device 20. Further, the similar case search device 10 has an interface unit 11, a storage unit 12, and a control unit 13.

The interface unit 11 is an interface for inputting / outputting data and performing data communication with other devices. Further, for example, the interface unit 11 outputs image data to the display device 20. At this time, the display device 20 displays an image based on the image data received via the interface unit 11. The display device 20 is, for example, a display and a touch panel.

The storage unit 12 is an example of a storage device that stores a program executed by the control unit 13 and data necessary for executing the program. The storage unit 12 stores the patient DB 121, the case DB 122, and the image DB 123.

Information for each patient is stored in the patient DB 121. FIG. 2 is a diagram showing an example of the data structure of the patient DB. As shown in FIG. 2, the patient DB 121 stores the patient ID, name, gender, and date of birth for each patient. In the example of FIG. 2, it is shown that the name of the patient whose patient ID is "0000000074" is "Patient Taro", the gender is male, and the date of birth is "1950/1/1". ..

The test result is stored in the case DB 122. FIG. 3 is a diagram showing an example of the data structure of the case DB. As shown in FIG. 3, the case DB 122 stores a flag and an image path together with a patient ID for identifying a patient and a UID for identifying an image. The flag is added or deleted based on the user's operation. In the embodiment, the state in which the value of the flag is "1" is defined as the state in which the flag is given. On the contrary, in the embodiment, the state where the value of the flag is "0" is the state where the flag is not given. Further, the case DB 122 stores one or more search conditions for searching each image. The search condition is, for example, a feature amount extracted from an image.

In the example of FIG. 3, the flag of the image in which the patient ID is "0000000074" and the UID of the patient is "1.1.1.1" is "1", the search condition f1 is "100", and the search condition f2 is "". It is shown to be "300".

The image obtained in the inspection is stored in the image DB 123. The image stored in the image DB 123 is uniquely identified by the UID. Further, the path for acquiring the image in the image DB 123 may be stored in the case DB 122.

In the control unit 13, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), or the like executes a program stored in an internal storage device using the RAM as a work area. Is realized by. Further, the control unit 13 may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 13 includes a display control unit 131, an extraction unit 132, a search unit 133, and an update unit 134.

The display control unit 131 causes the display device 20 to display an image generated based on the result of processing by each processing unit of the control unit 13. For example, the display control unit 131 outputs image data to the display device 20 via the interface unit 11.

The extraction unit 132 extracts the feature amount from the image. The extraction unit 132 may extract the feature amount of the entire image or may extract the feature amount for each patch.

Here, the patch is a plurality of regions obtained by dividing the CT image. For example, a patch is a 16 × 16 pixel image obtained by dividing a CT image. In addition, a shadow type can be obtained based on the degree of similarity between the characteristics of each patch and the characteristics of each predetermined shadow. For example, the similar case search system 1 can detect the shadow type corresponding to each patch by using a machine learning model that calculates the similarity between the feature amount extracted from the patch and the feature amount of each shadow type. can.

The search unit 133 searches the image DB 123 for an image similar to the first image obtained by photographing the lungs of the patient. The image DB 123 is an example of a predetermined storage area. For example, the search unit 133 can compare the similarity between the feature amount of the designated image and the feature amount of the image stored in the image DB 123, and acquire a predetermined number of images in descending order of similarity. In addition, the search unit 133 may acquire an image whose similarity is equal to or higher than a predetermined value and which matches other specified conditions. Here, other conditions include a patient ID, a type of disease, an examination date, and the like. In addition, the other condition may be whether or not a flag is added.

The display control unit 131 arranges each of the images obtained by the search by the search unit 133 in descending order of similarity with the first image, together with a mark indicating whether or not each image is given a predetermined flag. Display on the display device 20.

FIG. 4 is a diagram showing an example of a screen for displaying search results. As shown in FIG. 4, the image to be inspected is displayed in the area 201 of the search result display screen. The image to be inspected displayed in the area 201 corresponds to the first image. On the image, the patch is represented by a square. Also, the color of the square border representing the patch may be different for each type of shadow detected.

Further, in the lower part of the area 201, information about the patient acquired from the patient DB 121 is displayed. Further, at the lower part of the area 201, a group of check boxes for selecting the type of patch to be displayed is displayed.

On the other hand, the image obtained by the search is displayed in the area 201a, the area 201b, the area 201c, and the area 201d. Further, it is assumed that the degree of similarity between the displayed image and the image to be inspected is large in the order of region 201a, region 201b, region 201c, and region 201d.

The mark 211, the mark 211a, the mark 211b, the mark 211c, and the mark 211d can visually indicate whether or not the corresponding image is flagged in the image DB 123. For example, it is assumed that each mark has two states, lighting and extinguishing. When the flag is given, the state of the mark is lit. On the other hand, if the flag is not given, the mark state is turned off. For example, lighting is a state in which the mark is filled with a color different from the background. Further, for example, turning off is a state in which the mark is filled with the same color as the background.

The display mode of the mark corresponding to the flag is not limited to the above. For example, the mark may be filled with a first color if the image is flagged, or with a second color different from the first color if the image is not flagged. .. Further, for example, the mark may be visibly displayed when the image is flagged, and may not be displayed when the image is not flagged.

In the example of FIG. 4, the states of the mark 211, the mark 211b, the mark 211c, and the mark 211 are off. In this case, the flag of the record of the case DB 122 corresponding to the image displayed in the area 201, the area 201b, the area 201c, and the area 201d is "0". On the other hand, the state of the mark 211a is lit. In this case, the flag of the record of the case DB 122 corresponding to the image displayed in the area 201a is "1".

When a predetermined operation is performed on the mark corresponding to the image displayed on the display device 20, the updating unit 134 updates the information indicating whether or not the flag is added to the image corresponding to the mark. do. For example, the update unit 134 updates the flag of the case DB 122 to "0" or "1".

For example, when the mark 211 of FIG. 4 is pressed, the update unit 134 updates the flag of the record of the corresponding image DB from "0" to "1". On the other hand, when the mark 211a in FIG. 4 is pressed, the update unit 134 updates the flag of the record of the corresponding image DB from "1" to "0".

FIG. 5 is a flowchart showing the flow of the search process. First, the search unit 133 searches for an image similar to the image to be diagnosed (step S101). Next, the display control unit 131 displays the images obtained by the search in descending order of similarity (step S102). Then, the display control unit 131 lights and displays the mark of the image whose flag is “1” (step S103).

FIG. 6 is a flowchart showing the flow of the update process. First, the display control unit 131 accepts an operation of pressing the mark (step S201).

Here, when the state of the mark immediately before the operation is extinguished (step S202, extinguished), the update unit 134 updates the flag corresponding to the operated mark to "1" (step S203). Then, the display control unit 131 lights and displays the operated mark (step S204).

On the other hand, when the state of the mark immediately before the operation is lit (step S202, lit), the update unit 134 updates the flag corresponding to the operated mark to "0" (step S205). The display control unit 131 turns off the operated mark (step S206).

As described above, the search unit 133 searches for an image similar to the first image of the patient's lungs from a predetermined storage area. The display control unit 131 arranges each of the images obtained by the search on the display device 20 in descending order of similarity with the first image, together with a mark indicating whether or not each image is given a predetermined flag. Display it. In this way, the similar case search device 10 can display the search results in descending order of similarity and display whether or not the flags are given, regardless of whether or not the flags are given. Since the flag can be arbitrarily given by the user, according to the present embodiment, the user's judgment can be reflected in the search result of the similar image.

In this embodiment, only two types of whether or not the flag is attached are shown, but the flag may have three or more types of states. For example, it is possible to give it as multi-step information according to the user's feeling. When such a flag has multi-step information, the computer can also display each step of the case to which the flag information is given in an identifiable manner. For example, when the flag has information of multiple stages, the display control unit 131 makes each stage identifiable and displays it on the display device 20.

When a predetermined operation is performed on the mark corresponding to the image displayed on the display device 20, the updating unit 134 updates the information indicating whether or not the flag is added to the image corresponding to the mark. do. In this way, the similar case search device 10 can update the DB according to the user's operation on the screen. As a result, according to the present embodiment, the user's judgment can be reflected in the search result of the similar image.

Information including processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. Further, the specific examples, distributions, numerical values, etc. described in the examples are merely examples and can be arbitrarily changed.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific forms of distribution and integration of each device are not limited to those shown in the figure. That is, all or a part thereof can be functionally or physically distributed / integrated in an arbitrary unit according to various loads, usage conditions, and the like. Further, each processing function performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

FIG. 7 is a diagram illustrating a hardware configuration example. As shown in FIG. 11, the similar case search device 10 includes a communication interface 10a, an HDD (Hard Disk Drive) 10b, a memory 10c, and a processor 10d. Further, the parts shown in FIG. 7 are connected to each other by a bus or the like.

The communication interface 10a is a network interface card or the like, and communicates with another server. The HDD 10b stores a program or DB that operates the function shown in FIG.

The processor 10d is a hardware that operates a process that executes each function described in FIG. 1 or the like by reading a program that executes the same processing as each processing unit shown in FIG. 1 from the HDD 10b or the like and expanding the program into the memory 10c. It is a wear circuit. That is, this process executes the same function as each processing unit of the similar case search device 10. Specifically, the processor 10d reads a program having the same functions as the display control unit 131, the extraction unit 132, the search unit 133, and the update unit 134 from the HDD 10b or the like. Then, the processor 10d executes a process of executing the same processing as the display control unit 131, the extraction unit 132, the search unit 133, the update unit 134, and the like.

In this way, the similar case search device 10 operates as an information processing device that executes the learning method by reading and executing the program. Further, the similar case search device 10 can realize the same function as that of the above-described embodiment by reading the program from the recording medium by the medium reading device and executing the read program. The program referred to in the other examples is not limited to being executed by the similar case search device 10. For example, the present invention can be similarly applied when another computer or server executes a program, or when they execute a program in cooperation with each other.

This program can be distributed via a network such as the Internet. In addition, this program is recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), or DVD (Digital Versatile Disc), and is recorded from the recording medium by the computer. It can be executed by being read.

1 Similar case search system 10 Similar case search device 11 Interface unit 12 Storage unit 13 Control unit 20 Display device 121 Patient DB
122 Case DB
123 image DB
131 Display control unit 132 Extraction unit 133 Search unit 134 Update unit 201, 201a, 201b, 201c, 201d Area 211, 211a, 211b, 211c, 211d Mark

Claims (5)

An image similar to the first image of the patient's lungs was searched from a predetermined storage area.
The computer is subjected to a process of displaying each of the images obtained by the search on the display device in descending order of similarity with the first image, together with a mark indicating whether or not each image is given a predetermined flag. A similar case search program characterized by being executed. When a predetermined operation is performed on the mark corresponding to the image displayed on the display device, the process of updating the information indicating whether or not the flag is given to the image corresponding to the mark is further added. The similar case search program according to claim 1, wherein the program is executed. The similar case search program according to claim 1, wherein when the flag has information in multiple stages, a process of making each stage identifiable and displaying the information is further executed. An image similar to the first image of the patient's lungs was searched from a predetermined storage area.
The computer performs a process of displaying each of the images obtained by the search on the display device in descending order of similarity with the first image, together with a mark indicating whether or not each image is given a predetermined flag. A similar case search method characterized by performing. A similar case search system having a similar case search device and a display device.
The similar case search device is
A search unit that searches a predetermined storage area for an image similar to the first image of the patient's lungs, and
A display control unit that displays each of the images obtained by the search on the display device by arranging them in descending order of similarity with the first image together with a mark indicating whether or not each image is given a predetermined flag. When,
Have,
The display device is
A similar case search system characterized in that a screen is displayed according to the control of the display control unit.

Images