JP2021174073A - Similar case search program, similar case search method, and similar case search system - Google Patents

Abstract

To clarify the relation between pieces of information in a display.SOLUTION: A display control unit controls to display images of a patient's lung and medical record information concerning the images in a switchable manner in an area on the screen of a display device. The display control unit is configured so as to, when an image and medical record information are displayed in a switchable manner in each of the two areas side-by-side and when an operation to switch between the image and medical record information is made on one of the two areas, switch the image and medical record information in both of the two areas.SELECTED DRAWING: Figure 5

Description

The present invention relates to a similar case search program, a similar case search method, and a similar case search system.

Conventionally, there is known a case search system that presents a CT image of the lungs of a patient with diffuse lung disease to a doctor to support a medical examination.

innnavinet, "Fujifilm expands the target diseases of the similar case search system" SYNAPSE Case Match "that supports image diagnosis using artificial intelligence ~ Case search function of" diffuse lung disease "that is difficult to diagnose Equipped with "~ (https://www.innervision.co.jp/products/release/20170414) FUJIFILM, "Successful joint development of diagnosing support technology for interstitial pneumonia using FUJIFILM and Kyoto University AI technology" (https://www.fujifilm.co.jp/corporate/news/articleffnr_1417.html)

However, in the conventional technique, there is a problem that the relationship between the displayed information may not be clear. For example, when the images of two patients are displayed side by side on a screen in which the image and the medical record information can be switched, and one of the images is switched to the medical record information, the displayed image and the medical record information are related to each other. It may not be clear whether or not it is.

One aspect aims to clarify the relationship between the displayed information.

In one embodiment, the similar case search program causes a computer to perform a process of switchingably displaying an image of a patient's lungs and medical record information related to the image in an area on the screen of the display device. The similar case search program executes a process of switching between both images and medical record information in the two areas on the computer when an operation for switching between the image and the medical record information is performed on one of the two arranged areas. Let me.

On one side, the relationship between the displayed information can be clarified.

FIG. 1 is a diagram showing a configuration example of a similar case search system. FIG. 2 is a diagram showing an example of the data structure of the patient DB. FIG. 3 is a diagram showing an example of the data structure of the case DB. FIG. 4 is a diagram showing a configuration example of the display control unit. FIG. 5 is a diagram showing an example of a screen for comparing one-to-many images. FIG. 6 is a diagram showing an example of a screen for comparing one-to-many medical record information. FIG. 7 is a diagram showing an example of a screen for comparing one-to-one images. FIG. 8 is a diagram showing an example of a screen for comparing one-to-one medical record information. FIG. 9 is a flowchart showing the flow of the display control process. FIG. 10 is a diagram illustrating a hardware configuration example.

Hereinafter, examples of the similar case search program, the similar case search method, and the similar case search system according to the present invention will be described in detail with reference to the drawings. The present invention is not limited to this embodiment. In addition, each embodiment can be appropriately combined within a consistent range.

The similar case search system is a system that supports a doctor's examination by searching for patient images stored in a DB and displaying the images obtained by the search. The image here is a medical image such as a CT image. For example, the similar case search system 1 can acquire an image similar to the image of the patient under examination from the DB and present information on the disease associated with the acquired image to the doctor. This allows the doctor to make a medical examination with reference to the presented image and information about the disease. The user of the similar case search system is, for example, a doctor.

FIG. 1 is a diagram showing a configuration example of a similar case search system. As shown in FIG. 1, the similar case search system 1 has a similar case search device 10 and a display device 20. Further, the similar case search device 10 has an interface unit 11, a storage unit 12, and a control unit 13.

The interface unit 11 is an interface for inputting / outputting data and performing data communication with other devices. Further, for example, the interface unit 11 outputs image data to the display device 20. At this time, the display device 20 displays an image based on the image data received via the interface unit 11. The display device 20 is, for example, a display and a touch panel.

The storage unit 12 is an example of a storage device that stores a program executed by the control unit 13 and data necessary for executing the program. The storage unit 12 stores the patient DB 121, the case DB 122, and the image DB 123.

Information for each patient is stored in the patient DB 121. FIG. 2 is a diagram showing an example of the data structure of the patient DB. As shown in FIG. 2, the patient DB 121 stores the patient ID, name, gender, and date of birth for each patient. In the example of FIG. 2, it is shown that the name of the patient whose patient ID is "0000000074" is "Patient Taro", the gender is male, and the date of birth is "1950/1/1". ..

The test result is stored in the case DB 122. FIG. 3 is a diagram showing an example of the data structure of the case DB. As shown in FIG. 3, the case DB 122 stores image information and medical record information together with a patient ID for identifying a patient and a UID for identifying an image.

The image information f1, the image information f2, and the like are, for example, feature quantities extracted from the image. The image information is used for calculating the similarity of images. In addition, the medical record information includes detailed test values such as clinical diagnosis, image diagnosis, pathological diagnosis result, image test date, and white blood cell count.

In the example of FIG. 3, the image information f1 of the image in which the patient ID is "0000000074" and the UID of the patient is "1.1.1.1" is "100", the image information f2 is "300", and the pathological diagnosis is performed. As a result, it was diagnosed as "○○ disease", the imaging examination date was "2019/10/15", and the leukocyte count was "10000".

The image obtained in the inspection is stored in the image DB 123. The image stored in the image DB 123 is uniquely identified by the UID. Further, the path for acquiring the image of the image DB 123 may be stored in the case DB 122.

In the control unit 13, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), or the like executes a program stored in an internal storage device using the RAM as a work area. Is realized by. Further, the control unit 13 may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 13 includes a display control unit 131, an extraction unit 132, and a search unit 133.

The display control unit 131 causes the display device 20 to display an image generated based on the result of processing by each processing unit of the control unit 13. For example, the display control unit 131 outputs image data to the display device 20 via the interface unit 11.

The extraction unit 132 extracts the feature amount from the image. The extraction unit 132 may extract the feature amount of the entire image or may extract the feature amount for each patch.

Here, the patch is a plurality of regions obtained by dividing the CT image. For example, a patch is a 16 × 16 pixel image obtained by dividing a CT image. In addition, a shadow type can be obtained based on the degree of similarity between the characteristics of each patch and the characteristics of each predetermined shadow. For example, the similar case search system 1 can detect the shadow type corresponding to each patch by using a machine learning model that calculates the similarity between the feature amount extracted from the patch and the feature amount of each shadow type. can.

The search unit 133 searches the image DB 123 for an image similar to the first image obtained by photographing the lungs of the patient. The image DB 123 is an example of a predetermined storage area. For example, the search unit 133 can compare the similarity between the feature amount of the designated image and the feature amount of the image stored in the image DB 123, and acquire a predetermined number of images in descending order of similarity. In addition, the search unit 133 may acquire an image whose similarity is equal to or higher than a predetermined value and which matches other specified conditions. Here, other conditions include patient ID and various medical record information. In addition, the other condition may be whether or not a flag is added.

The display control unit 131 causes the display device 20 to display the image obtained by the search by the search unit 133. In addition, the display control unit 131 displays the medical record information corresponding to the image. The display control unit 131 can acquire medical record information from the case DB 122.

Here, the display control unit 131 makes the area on the screen of the display device 20 switchably display an image of the patient's lungs and medical record information related to the image. Further, the display control unit 131 can display either a screen for comparing one-to-many images or medical record information or a screen for comparing one-to-one images or medical record information according to the user's designation.

FIG. 4 is a diagram showing a configuration example of the display control unit. As shown in FIG. 4, the display control unit 131 includes a diagnosis target display content switching unit 131a and a comparison target display content switching unit 131b. The diagnosis target display content switching unit 131a switches the display content of the area for displaying the diagnosis target. On the other hand, the comparison target display content switching unit 131b switches the display content of the area for displaying the comparison target. The diagnosis target display content switching unit 131a and the comparison target display content switching unit 131b are examples of the switching unit.

When one of the diagnosis target display content switching unit 131a and the comparison target display content switching unit 131b receives a request to switch the display content, the other also requests to switch the display content. For example, it is assumed that the diagnosis target display content switching unit 131a receives a request to display the medical record information of the patient A while displaying the image information of the patient A. At this time, the diagnosis target display content switching unit 131a requests the comparison target display content switching unit 131b to switch the display content from the image information of the patient B to the chart information of the patient B. Hereinafter, the operation of the display control unit 131 will be described in detail using a screen example.

FIG. 5 is a diagram showing an example of a screen for comparing one-to-many images. As shown in FIG. 5, the image to be diagnosed is displayed in the area 201 of the screen. Further, as a screen to be compared, four images are displayed in the area group 202 on the right side of the area 201. Further, in the lower part of the area 201, an inspection image tab 211 and a medical record detail tab 212 are provided.

When the inspection image tab 211 is selected, the display control unit 131 displays a screen for comparing images. On the other hand, when the medical record details tab 212 is selected, the display control unit 131 displays a screen for comparing medical record information. In the example of FIG. 5, since the inspection image tab 211 is selected, a screen for comparing images is displayed.

FIG. 6 is a diagram showing an example of a screen for comparing one-to-many medical record information. In the state shown in FIG. 5, when the chart detail tab 212 is selected, the display control unit 131 displays the screen of FIG. As shown in FIG. 6, in the area 201 of the screen for comparing one-to-many medical record information, the medical record information corresponding to the image to be diagnosed is displayed instead of the image to be diagnosed. On the other hand, the four images of the area group 202 are displayed as they are even when the chart detail tab 212 is selected.

In this way, when the image and the medical record information are displayed in each of the three or more areas, the display control unit 131 performs an operation of switching between the image and the medical record information for one of the areas. If so, the image of only the area where the operation is performed and the medical record information are switched.

Here, when any of the four images displayed in the area group 202 is selected, the display control unit 131 displays a screen for comparing the image to be diagnosed with the selected image on a one-to-one basis. Let me.

FIG. 7 is a diagram showing an example of a screen for comparing one-to-one images. As shown in FIG. 7, an image to be diagnosed is displayed in the area 201a of the screen. Further, the image to be compared is displayed in the area 201b. An inspection image tab 211a and a chart detail tab 212a are provided below the area 201a. Further, in the lower part of the area 201b, an inspection image tab 211b and a medical record detail tab 212b are provided. Further, in the area group 202c, an image that is not selected from the images displayed in the area group 202 is displayed. When comparing images in detail in this way, the areas 201a and 201b have the same size and layout within the areas so that the comparison can be easily performed by the user.

When the inspection image tab 211a or the inspection image tab 211b is selected, the display control unit 131 displays a screen for comparing images. On the other hand, when the medical record detail tab 212a or the medical record detail tab 212b is selected, the display control unit 131 displays a screen for comparing medical record information.

When the image and the medical record information are switchably displayed in each of the two arranged areas, the display control unit 131 performs an operation of switching between the image and the medical record information for one of the two areas. If so, switch between both images and chart information in the two areas.

Therefore, in the example of FIG. 7, the inspection image tab 211a and the inspection image tab 211b are selected, and a screen for comparing images is displayed in both the area 201a and the area 201b. At this time, for example, when the medical record detail tab 212a is selected, the display control unit 131 causes the medical record information to be displayed in both the area 201a and the area 201b.

FIG. 8 is a diagram showing an example of a screen for comparing one-to-one medical record information. In the state shown in FIG. 7, when either the chart detail tab 212a or the chart detail tab 212b is selected, the display control unit 131 displays the screen of FIG. In the example of FIG. 8, the medical record information is displayed in both the area 201a and the area 201b.

FIG. 9 is a flowchart showing the flow of the display control process. As shown in FIG. 9, first, the search unit 133 searches for an image similar to the image to be diagnosed (step S101). Next, the display control unit 131 displays the images obtained by the search in descending order of similarity (step S102).

Here, the display control unit 131 accepts the selection of the image to be compared (step S103). The display control unit 131 displays the image to be diagnosed and the image to be compared, or the medical record information corresponding to each image in an arranged area (step S104).

The display control unit 131 waits until the tab in any of the areas is changed (step S105, No). When the tab of any area is changed (step S105, Yes), the display control unit 131 switches the tabs of both areas in conjunction with each other (step S106). After that, the display control unit 131 returns to step S104 to display an image, and then waits for the tab operation.

As described above, the display control unit 131 makes the area on the screen of the display device 20 switchably display an image of the patient's lungs and medical record information related to the image.
When the image and the medical record information are switchably displayed in each of the two arranged areas, the display control unit 131 performs an operation of switching between the image and the medical record information for one of the two areas. If so, switch between both images and chart information in the two areas. For example, when the user displays an image or medical record information side by side in two areas on the screen, consider a case where an image of a patient in one area is displayed and a medical record information of another patient is displayed in the other area. .. In this case, it is possible that the user mistakenly determines that the image and the medical record information belong to the same patient. In this embodiment, the relationship between the displayed information can be clarified by interlocking the switching of the information displayed in the two side-by-side areas. As a result, it is possible to prevent the occurrence of the above-mentioned erroneous judgment.

In this embodiment, when the user makes a detailed comparison of images, the comparison of two areas arranged on the screen is described as an example. In reality, the user may confirm three or more areas displayed side by side while comparing the information displayed in each area. For example, when the size of each displayed area and the layout in the area are the same, the user may check the information displayed in each area while comparing them. Further, for example, when the size of each displayed area and the layout in the area are the same as the size of the area in which the diagnosis target is displayed and the layout in the area, the user analyzes the case while comparing them. there is a possibility. Therefore, the display control unit 131 can also interlock the switching of the information displayed in the three or more areas arranged on the screen in the scene where the user compares the three or more areas arranged on the screen in detail. ..

When the display control unit 131 has three or more areas for displaying images or medical record information about the patient and the size of each area and the layout in the area are the same, the display control unit 131 is used for one of the areas. When the operation of switching between the image and the medical record information is performed, the process of switching the image and the medical record information in the area other than the area where the operation is performed may be performed according to the operation.

The display control unit 131 manages, for example, the size of each area displayed on the screen and the layout information in the area. When the display control unit 131 receives an instruction to change the information displayed in one area A displayed on the screen, one or more of the other areas on the screen have the same size and layout as the area A. Specify the area B of. The display control unit 131 changes the information displayed in the area A in accordance with the change instruction for the area A, and also changes the information displayed in the one or more areas B in accordance with the change instruction in the area A. conduct.

With such a configuration, even when three or more areas are displayed, when the display of one area is changed, the other areas can be changed in conjunction with each other.

On the other hand, even if three or more areas for displaying the image or the medical record information about the patient are arranged side by side, in the case of a display pattern such as one-to-many, the user can compare each area for details. It may not be compared to. In this way, when the operation is for an area different from the size of the area for displaying the diagnosis target and the layout in the area, the display control unit 131 displays an image and medical record information in each of the three or more areas. When is displayed as switchable and an operation for switching between the image and the medical record information is performed for one of the areas, the image and the medical record information of only the area where the operation is performed are switched. .. As described above, in the case of one-to-many comparison, it is considered that the above-mentioned erroneous judgment is unlikely to occur even if the switching of the information to be displayed is not linked. According to the embodiment, in the case of one-to-many comparison, the screen display can be performed according to the display mode desired by the user.

The screen switching method is not limited to tab switching. For example, the display control unit 131 may accept screen switching by scrolling. For example, in the example of FIG. 7, when the area 201a is scrolled downward, the display control unit 131 causes the area 201a to display the medical record information. At this time, the display control unit 131 interlocks with the area 201a to display the medical record information in the area 201b as well.

Further, for example, the display control unit 131 may accept screen switching by calling a child screen. For example, in the example of FIG. 7, when the area 201a is clicked, the display control unit 131 calls the child screen and displays the medical record information on the child screen. At this time, the display control unit 131 interlocks with the area 201a to display the medical record information related to the image displayed in the area 201b on another child screen.

Information including processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. Further, the specific examples, distributions, numerical values, etc. described in the examples are merely examples and can be arbitrarily changed.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific forms of distribution and integration of each device are not limited to those shown in the figure. That is, all or a part thereof can be functionally or physically distributed / integrated in an arbitrary unit according to various loads, usage conditions, and the like. Further, each processing function performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

FIG. 10 is a diagram illustrating a hardware configuration example. As shown in FIG. 10, the similar case search device 10 includes a communication interface 10a, an HDD (Hard Disk Drive) 10b, a memory 10c, and a processor 10d. Further, the parts shown in FIG. 10 are connected to each other by a bus or the like.

The communication interface 10a is a network interface card or the like, and communicates with another server. The HDD 10b stores a program or DB that operates the function shown in FIG.

The processor 10d is a hardware that operates a process that executes each function described in FIG. 1 or the like by reading a program that executes the same processing as each processing unit shown in FIG. 1 from the HDD 10b or the like and expanding the program into the memory 10c. It is a wear circuit. That is, this process executes the same function as each processing unit of the similar case search device 10. Specifically, the processor 10d reads a program having the same functions as the display control unit 131, the extraction unit 132, and the search unit 133 from the HDD 10b or the like. Then, the processor 10d executes a process of executing the same processing as the display control unit 131, the extraction unit 132, the search unit 133, and the like.

In this way, the similar case search device 10 operates as an information processing device that executes the learning method by reading and executing the program. Further, the similar case search device 10 can realize the same function as that of the above-described embodiment by reading the program from the recording medium by the medium reading device and executing the read program. The program referred to in the other examples is not limited to being executed by the similar case search device 10. For example, the present invention can be similarly applied when another computer or server executes a program, or when they execute a program in cooperation with each other.

This program can be distributed via a network such as the Internet. In addition, this program is recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), or DVD (Digital Versatile Disc), and is recorded from the recording medium by the computer. It can be executed by being read.

1 Similar case search system 10 Similar case search device 11 Interface unit 12 Storage unit 13 Control unit 20 Display device 121 Patient DB
122 Case DB
123 image DB
131 Display control unit 131a Diagnosis target display content switching unit 131b Comparison target display content switching unit 132 Extraction unit 133 Search unit 201, 201a, 201b Area 202, 202c Area group 211, 211a, 211b Inspection image tab 212, 212a, 212b Chart details tab

Claims (5)

An image of the patient's lungs and medical record information related to the image can be switched and displayed in the area on the screen of the display device.
When the image and the medical record information are switchably displayed in each of the two arranged areas, an operation of switching between the image and the medical record information is performed on one of the two areas. In the case of a similar case search program, the computer is made to execute a process of switching between the image and the medical record information in both of the two areas. The switching process is performed in one of the areas when the size of each area and the layout in the area are the same in a state where three or more areas for displaying the image or the medical record information about the patient are lined up. On the other hand, when the operation of switching between the image and the medical record information is performed, the image and the medical record information of the area other than the area where the operation is performed are also switched according to the operation. The similar case search program according to claim 1. In the switching process, in a state where three or more areas for displaying the image or the medical record information about the patient are lined up, the size of the area for displaying the diagnosis target and the layout in the area are different from the areas. The similar case search program according to claim 1, wherein when the operation is performed, the image of only the area where the operation is performed and the medical record information are switched. An image of the patient's lungs and medical record information related to the image can be switched and displayed in the area on the screen of the display device.
When the image and the medical record information are switchably displayed in two side-by-side areas, and an operation of switching between the image and the medical record information is performed on one of the two areas. , A similar case search method, characterized in that a computer executes a process of switching between the image and the medical record information in both of the two areas. A similar case search system having a similar case search device and a display device.
The similar case search device is
A display control unit that switchesably displays an image of a patient's lungs and medical record information related to the image in an area on the screen of the display device.
When the image and the medical record information are switchably displayed in two side-by-side areas, and an operation of switching between the image and the medical record information is performed on one of the two areas. , A switching unit for switching between the image and the medical record information in both of the two areas,
Have,
The display device is
A similar case search system characterized in that a screen is displayed according to the control of the display control unit and the switching unit.

Images