JP7441155B2 - Information processing device, information processing method and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

There are viewer systems that display patients' electronic medical records and medical images. For example, in Patent Document 1, by displaying mapping data in which medical events that have occurred for multiple patients can be identified for each patient and mapped onto a common time axis, medical information on patients who should be given priority treatment can be displayed. A medical information display device that can shorten the time required to access the information has been disclosed.

JP2020-38642A

However, in the invention according to Patent Document 1, in order to access the details of a patient's medical information, it is necessary to transition from the screen displaying mapping data to another screen, and it is not possible to freely access it with a simple operation. do not have.

One aspect of the present invention is to provide an information processing device or the like that can suitably support medical personnel.

An information processing device according to one aspect includes an acquisition unit that acquires a plurality of medical images taken of a patient's biological lumen at a plurality of time points, time information indicating the time point at which each medical image is taken, and a time axis. , a display unit that displays a screen including the respective medical images arranged corresponding to positions on the time axis corresponding to the time of imaging, and a reception unit that receives operation input from a user, the display unit receives a specific operation input from the user to enlarge the screen , enlarges and displays a partial area of the screen , and when the partial area is enlarged to a first magnification, displays the partial area. display related information of the medical image included in the partial area, and when the partial area is further enlarged to a second magnification higher than the first magnification, perform a predetermined operation related to the medical image included in the partial area. A plurality of operation objects for receiving inputs are displayed, and the reception unit receives operation inputs to the operation objects .

In one aspect, it is possible to suitably support medical personnel.

FIG. 1 is an explanatory diagram showing a configuration example of an image diagnosis system. FIG. 2 is a block diagram showing a configuration example of a server. FIG. 1 is a block diagram showing a configuration example of an image diagnostic apparatus. It is an explanatory diagram showing an example of record layout of image DB and case DB. It is an explanatory diagram showing an example of a time series screen. FIG. 3 is an explanatory diagram regarding enlarged display of a time-series screen. FIG. 2 is an explanatory diagram regarding an operation panel for medical image analysis. It is an explanatory diagram regarding comment input operation. FIG. 6 is an explanatory diagram regarding outline display of treatment information. 2 is a flowchart showing a processing procedure executed by the first image diagnostic apparatus 2. FIG. 3 is a flowchart showing a processing procedure of an image analysis subroutine. FIG. 7 is an explanatory diagram showing an example of a display screen of the image diagnostic apparatus according to Embodiment 2; FIG. 7 is an explanatory diagram showing an example of a display screen of the image diagnostic apparatus according to Embodiment 2; 7 is a flowchart showing a processing procedure executed by the image diagnostic apparatus according to Embodiment 2. FIG.

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below based on drawings showing embodiments thereof.
(Embodiment 1)
FIG. 1 is an explanatory diagram showing an example of the configuration of an image diagnosis system. In this embodiment, medical images taken of blood vessels (body lumen) of a patient to be treated, treatment information indicating the details of the treatment for the patient, etc. are displayed in chronological order. Next, an image diagnostic system that can access detailed information such as medical images will be explained. The image diagnosis system includes a server 1, a first image diagnosis device (information processing device) 2, a second image diagnosis device 3, an external display device 4, and an electronic medical record server 5. Each device is communicatively connected via a network N.

In this embodiment, a blood vessel is taken as an example of a living body lumen, but the living body lumen to be photographed is not limited to blood vessels, and may be other living body lumens such as bile ducts, pancreatic ducts, bronchi, intestines, etc. good.

The server 1 is a server computer capable of various information processing and transmission/reception of information. The server 1 according to the present embodiment functions as a database server that stores medical images and the like, and stores the medical images and the like captured by the first image diagnostic apparatus 2 and the second image diagnostic apparatus 3 in a database.

The first image diagnostic device 2 is an imaging device that captures a medical image of a patient's biological lumen, and is, for example, an IVUS (Intravascular Ultrasound) device that performs an ultrasound examination using a catheter 201. In this embodiment, a case where catheter treatment such as PCI (Percutaneous Coronary Intervention) is performed will be described as an example. Catheter 201 is a medical instrument inserted into a patient's blood vessel, and transmits ultrasound signals and receives reflected waves from an ultrasound probe attached to its tip. The first image diagnostic apparatus 2 generates and displays an ultrasonic tomographic image (cross-sectional image) based on the reflected waves received by the catheter 201.

In this embodiment, the description will be made assuming that the first image diagnostic device 2 that captures blood vessel tomograms is an IVUS device, but an optical imaging device using OCT (Optical Coherence Tomography), etc. It may be.

The second image diagnostic device 3 is an imaging device that captures a different type of medical image from the first image diagnostic device 2, and is, for example, an angiography device that performs an angiography test. The second image diagnostic apparatus 3 includes an X-ray source 301 and an X-ray sensor 302, and the X-ray sensor 302 receives X-rays emitted from the X-ray source 301, thereby generating an X-ray fluoroscopic image (hereinafter referred to as "fluoroscopic image"). (referred to as "image"). In this embodiment, the first image diagnostic apparatus 2 acquires a fluoroscopic image from the second image diagnostic apparatus 3 via the network N, and displays the fluoroscopic image together with the ultrasound tomographic image.

In the present embodiment, the medical image captured by the second image diagnostic apparatus 3 will be described as a fluoroscopic image (angiography image), but X-ray CT (Computed Tomography), MRI (Magnetic Resonance Imaging), PET (Positron Emission Tomography), ultrasonic echo, and the like may be used.

The external display device 4 is an external display connected to the first image diagnostic device 2, and is, for example, a ceiling-suspended monitor installed in a catheter room where catheter treatment is performed, or a monitor installed in the front room of the catheter room. . As in Embodiment 2, which will be described later, the first image diagnostic apparatus 2 can also output and display medical images on the external display device 4.

The electronic medical record server 5 is a database server that stores electronic medical record data of patients at medical facilities (hospitals, etc.) that perform treatment. For example, the first image diagnostic apparatus 2 acquires patient information regarding a patient undergoing catheter treatment from the electronic medical record server 5 and displays it.

In this embodiment, the explanation will be made assuming that each medical facility has an electronic medical record server 5 that manages electronic medical record data, but it is also possible to configure the server 1 and the electronic medical record server 5 as an integrated server computer. Of course.

In this embodiment, the first image diagnostic apparatus 2 displays various information related to a patient undergoing catheter treatment (under treatment) in chronological order, and also displays individual information according to an operation to enlarge the display screen. Make it accessible. Specifically, as will be described later, the first image diagnostic apparatus 2 displays medical images taken of the patient's body lumen at each time point, treatment information indicating the details of the treatment performed at each time point, etc. along the time axis. Display the series screen (see Figure 5). The first image diagnostic apparatus 2 then enlarges a partial area of the screen by scrolling or the like, displays related information of the medical image included in the enlarged area, and performs a predetermined operation for analyzing the medical image. Accept input. Thereby, the user (medical worker) who operates the first image diagnostic apparatus 2 can have an overview of patient information on a time-series screen, and can intuitively and easily access individual information.

In addition, although this embodiment will be described assuming a scene during treatment as an example, this system can be applied to a scene before or after a treatment to display a time-series display regarding a patient scheduled to undergo catheter treatment, or A chronological display of patients for whom the treatment has been performed may be performed.

FIG. 2 is a block diagram showing an example of the configuration of the server 1. As shown in FIG. The server 1 includes a control section 11 , a main storage section 12 , a communication section 13 , and an auxiliary storage section 14 .
The control unit 11 is an arithmetic processing device such as one or more CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), etc., and reads out the program P1 stored in the auxiliary storage unit 14. By executing the command, various information processing, control processing, etc. are performed. The main memory unit 12 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc., and temporarily stores data necessary for the control unit 11 to perform arithmetic processing. Remember. The communication unit 13 is a communication module for performing processing related to communication, and sends and receives information to and from the outside.

The auxiliary storage unit 14 is a nonvolatile storage area such as a large capacity memory or a hard disk, and stores the program P1 and other data necessary for the control unit 11 to execute processing. Further, the auxiliary storage unit 14 stores an image DB 141 and a case DB 142. The image DB 141 is a database that stores medical images captured by the first image diagnostic apparatus 2 and the second image diagnostic apparatus 3. The case DB 142 is a database that stores case information regarding cases of patients who have been treated in the past.

Note that the auxiliary storage unit 14 may be an external storage device connected to the server 1. Further, the server 1 may be a multicomputer consisting of a plurality of computers, or may be a virtual machine virtually constructed by software.

Further, in this embodiment, the server 1 is not limited to the above configuration, and may include, for example, an input section that receives operation input, a display section that displays images, and the like. The server 1 also includes a reading unit that reads a portable storage medium 1a such as a CD (Compact Disk)-ROM or a DVD (Digital Versatile Disc)-ROM, and is configured to read and execute the program P1 from the portable storage medium 1a. You can also do it. Alternatively, the server 1 may read the program P1 from the semiconductor memory 1b.

FIG. 3 is a block diagram showing a configuration example of the first image diagnostic apparatus 2. As shown in FIG. The first image diagnostic apparatus 2 includes a control section 21 , a main storage section 22 , a communication section 23 , a display section 24 , an input section 25 , an input/output I/F 26 , an image processing section 27 , and an auxiliary storage section 28 .
The control unit 21 is an arithmetic processing device such as one or more CPUs, MPUs, GPUs, etc., and performs various information processing, control processing, etc. by reading and executing the program P2 stored in the auxiliary storage unit 28. . The main storage unit 22 is a temporary storage area such as a RAM, and temporarily stores data necessary for the control unit 21 to perform arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing, and sends and receives information to and from the outside.

The display unit 24 is a display screen such as a liquid crystal display, and displays images. The input unit 25 is an operation interface such as a keyboard and a mouse, and receives operation input from the user. The input/output I/F 26 is an interface for connecting the external display device 4 to the first image diagnostic device 2, and outputs image data to the external display device 4. The image processing unit 27 is an image processing module that processes signals transmitted and received via the catheter 201 and generates a blood vessel tomogram (medical image).

The auxiliary storage unit 28 is a nonvolatile storage area such as a hard disk or a large capacity memory, and stores the program P2 and other data necessary for the control unit 21 to execute processing. Further, the auxiliary storage unit 28 stores the identification model 20. The identification model 20 is a machine learning model generated by learning predetermined training data. This is a trained model that identifies an image region (hereinafter referred to as an "object region") corresponding to a specific object. The identification model 20 is assumed to be used as a program module that constitutes a part of artificial intelligence software.

Note that the first image diagnostic apparatus 2 may include a reading section that reads a portable storage medium 2a such as a CD-ROM, and may read and execute the program P2 from the portable storage medium 2a. Alternatively, the first image diagnostic apparatus 2 may read the program P2 from the semiconductor memory 2b.

FIG. 4 is an explanatory diagram showing an example of the record layout of the image DB 141 and the case DB 142. The image DB 141 includes an image ID string, an image string, and an additional information string. The image ID column stores image IDs for identifying each medical image. The image string and the supplementary information column each store a medical image captured by the first image diagnostic apparatus 2 or the second image diagnostic apparatus 3 and supplementary information of the medical image in association with the image ID.

The supplementary information is metadata attached to medical images captured by the first image diagnostic device 2 and the second image diagnostic device 3, and is, for example, metadata attached to medical images based on the DICOM (Digital Imaging and Communication in Medicine) standard. This is the tag data to be used. The incidental information string stores, for example, patient attribute information, examination information, series information, and the like. The attribute information is information related to the patient who is the subject, and includes data such as a patient ID, patient name, date of birth, and gender. The examination information is information related to the image examination, and includes, for example, the examination date, examination time, technician name, examination conditions, and the like. The series information is information regarding the series of images, and includes, for example, the modality (type) of the images, the imaged region, the series number, and the like.

The case DB 142 includes a case ID column, a patient information column, and a medical image column. The case ID column stores case IDs for identifying each case. The patient information column and the medical image column each store patient information regarding a case patient and a medical image with the case patient as a subject, in association with a case ID. The patient information column stores, for example, basic patient information, treatment information, examination information other than image diagnosis (to be distinguished from the above-mentioned examination information, it will be referred to as "second examination information" for convenience), and the like. The basic information includes, for example, the patient's age, gender, diagnosis name, medical history, and the like. The treatment information is information indicating the contents of the patient's treatment, and includes information such as the lesion site, the nature of the lesion, the puncture site of the catheter 201, and the treatment device used (for example, stent, balloon, etc.). The second test information includes, for example, blood test results.

5 to 9 are explanatory diagrams showing examples of display screens of the first image diagnostic apparatus 2. An overview of this embodiment will be explained based on FIGS. 5 to 9.

As described above, the first image diagnosis apparatus 2 displays a blood vessel tomogram imaged using the catheter 201, and also displays a fluoroscopic image imaged by the second image diagnosis apparatus 3. The first image diagnostic device 2 and the second image diagnostic device 3 generate image files in a format compliant with the DICOM standard, for example, and perform communication between the devices. The functions described below are realized, for example, as new functions of an existing DICOM viewer system. Note that, of course, it may be realized as an application independent of the DICOM viewer system.

FIG. 5 is an explanatory diagram showing an example of a time-series screen. A switching button 50 is displayed at the lower left of the screen. In addition to the time-series display mode shown in FIG. 5, the switch button 50 is used for a blood vessel measurement (imaging) mode using the catheter 201, and a list view mode (see the bottom left of FIG. This button is used to switch between the following: When the time series display mode is selected using the switching button 50, the first image diagnostic apparatus 2 displays the time series screen shown in FIG.

The time series screen is a screen that displays information on a patient undergoing treatment in time series, and includes a time axis 51, a plurality of medical images 52 arranged corresponding to positions on the time axis 51 corresponding to the time of imaging, 52, 52... and a patient information column 53. The first image diagnostic apparatus 2 displays thumbnails of medical images (for example, moving images consisting of a plurality of frames) captured at each time point along a time axis 51, and also displays a patient information column 53 at the upper right of the screen.

For example, as shown in FIG. 5, the first image diagnostic apparatus 2 displays a time axis 51 located at the center of the screen in the vertical direction and extending in the horizontal direction. Note that the time axis 51 is displayed as a strip-shaped display area having a certain width in the vertical direction. Further, although the time (point in time) is not displayed on the time axis 51 in the screen example of FIG. 5, it goes without saying that the time may be displayed as a plot or the like.

The first image diagnostic apparatus 2 generates different types of medical images 52 (vascular tomograms and fluoroscopic images) depending on the type of image in a direction intersecting the time axis 51 (for example, a vertical direction perpendicular to the time axis 51). Display in position. Specifically, the first image diagnostic apparatus 2 displays a tomographic image in a lower region across the time axis 51 at the center of the screen, and a fluoroscopic image in an upper region. Note that although there are two types of medical images 52 in this embodiment, there may be three or more types.

Further, in the present embodiment, different types of medical images 52 are displayed above and below the time axis 51, but the present embodiment is not limited to this. For example, the first image diagnostic apparatus 2 may display the time axis 51 at the lower end of the screen, and arrange the tomographic image in the lower half of the screen and the fluoroscopic image in the upper half. In this way, the first image diagnostic apparatus 2 only needs to be able to arrange each image at a different position (height) depending on the type of image.

Ancillary information including patient information, examination information, series information, etc. is attached to the image file of each medical image 52. In this embodiment, the first image diagnostic apparatus 2 acquires (extracts) time information indicating the imaging time, such as examination date and examination time, from the associated information, specifies the imaging time, and arranges each medical image 52. . Note that a series number is added to each medical image 52.

Further, the first image diagnostic apparatus 2 displays treatment information indicating the details of the treatment performed at each point in time during the treatment in association with the position on the time axis 51. Specifically, the first image diagnostic apparatus 2 displays icons 54, 54, 54, . . . indicating the details of treatment for the patient at each position within the band-shaped time axis 51. The icon 54 is a display object representing the details of the treatment performed at each point in time, and is, for example, an icon representing a treatment device used for the treatment. Taking PCI as an example, FIG. 5 shows a state in which a catheter 201 inserted into a patient's blood vessel, a stent placed in the patient's blood vessel, and the like are displayed as icons 54.

The icon 54 is set, for example, by a user's screen operation. The first image diagnostic apparatus 2 receives an operation input (for example, a drag and drop operation) for arranging an icon 54 representing a treatment device at an arbitrary position within the time axis 51. The first image diagnostic apparatus 2 displays an icon 54 at the position placed by the user, and performs treatment using the device represented by the icon 54 (for example, dilation of a blood vessel before stent placement, placement of a stent, dilation of a blood vessel after stent placement, etc.). to show that it has been carried out.

In the above description, the icon 54 is set by the user's screen operation, but the input operation for arranging the icon 54 is not limited to this. For example, the first image diagnostic apparatus 2 receives input for arranging the icon 54 on the time axis 51 by reading a two-dimensional code such as a QR code (registered trademark) or a barcode attached to a treatment device when the device is used. It's okay. For example, when a medical worker uses a treatment device such as a catheter 201 or a stent, he or she reads a two-dimensional code attached to the device using a reading means (not shown). The first image diagnostic apparatus 2 selects an icon 54 based on the device ID read from the two-dimensional code, reading time, etc., and arranges the icon 54 at a corresponding position (time) within the time axis 51. Thereby, treatment information can be recorded without having to perform troublesome screen operations.

The patient information field 53 is a display field that displays patient information regarding the patient undergoing treatment. For example, when the first image diagnostic apparatus 2 receives an operation input to the patient information column 53, it accesses the electronic medical record server 5 to acquire patient information, and displays information such as the patient's name and age. The patient information column 53 is displayed fixedly at the upper right of the screen even when the time series screen is enlarged by the enlarging operation described below, so that the user can check the patient information at any time.

FIG. 6 is an explanatory diagram regarding enlarged display of a time series screen. In the present embodiment, when receiving a specific operation input from the user, the first image diagnostic apparatus 2 enlarges and displays a partial area of the time-series screen illustrated in FIG. 5 . FIG. 6 shows a partially enlarged area of the time series screen.

For example, the first image diagnostic apparatus 2 enlarges (and reduces) a time-series screen by accepting a scroll operation of rotating a wheel of an operating mouse (not shown). Note that this operation is just an example, and the screen may be enlarged by, for example, a click operation. The first image diagnostic apparatus 2 changes the magnification of the time-series screen centering on the position of the mouse cursor on the screen and enlarges a partial area.

Although the present embodiment will be described assuming that screen operations are performed using a mouse (and keyboard), for example, the display of the first image diagnostic apparatus 2 may be used as a touch panel to accept touch operations such as pinching out or zooming out the screen. Good too. Alternatively, the first image diagnostic apparatus 2 may include a camera that captures an image of the user, and may enlarge the screen by recognizing a specific action (for example, a pinch-out or zoom-out gesture) from the image captured by the camera. In this way, the manner of screen operation is not particularly limited.

For example, the user places the cursor on any medical image 52 among the plurality of medical images 52, 52, 52, . . . displayed on the time-series screen to enlarge the screen. Here, when the time-series screen is enlarged to a predetermined first magnification, the first image diagnostic apparatus 2 displays images related to the medical image 52 (the medical image 52 on which the cursor is placed) included in the enlarged partial area. Related information is displayed near the medical image 52.

FIG. 6 illustrates a case where a blood vessel tomogram (cross-sectional image) captured by the first image diagnostic apparatus 2 is enlarged. In this case, the first image diagnostic apparatus 2 displays the longitudinal tomographic image 61 as related information. The longitudinal tomographic image is an image representing a longitudinal section of a blood vessel, and is an image obtained by reconstructing cross-sectional images of multiple frames captured by the pullback operation of the catheter 201. The first image diagnostic apparatus 2 displays the longitudinal tomographic image 61 next to the cross-sectional tomographic image when enlarged to the first magnification.

Although FIG. 6 shows a longitudinal tomographic image 61 as an example of related information, the related information is not limited to the longitudinal tomographic image 61. For example, the first image diagnostic apparatus 2 may display supplementary information (the above-mentioned patient attribute information, examination information, series information, etc.) attached to the medical image 52 as related information. For example, the first image diagnostic apparatus 2 uses a comment input function (see FIG. 8), which will be described later, to input information at the same position (point in time) as the medical image 52 on the time axis 51 or at a position within a predetermined range from the position of the medical image 52. Comments input at (time within a predetermined time) may be displayed. In this way, the related information is not limited to the longitudinal tomographic image 61.

Note that the first image diagnostic apparatus 2 may determine the priority order for displaying each of the above-mentioned related information, and display the related information sequentially according to the priority order. For example, the first image diagnostic apparatus 2 first displays the longitudinal tomographic image 61 when it is enlarged to the first magnification, and displays additional information and then comments when it is further enlarged. Thereby, it is possible to prioritize and present information that the user should pay attention to.

FIG. 7 is an explanatory diagram regarding the operation panel for analyzing the medical image 52. FIG. 7 shows a case where the time series screen is further enlarged from the state shown in FIG. When the first image diagnostic apparatus 2 is enlarged to a second magnification higher than the first magnification, the first image diagnostic apparatus 2 displays a time-series screen in the state shown in FIG. 7 .

Note that although the second magnification will be described as being higher than the first magnification in this embodiment, the second magnification may be lower than the first magnification or may be the same magnification.

For example, the first image diagnostic apparatus 2 continues to display the longitudinal tomographic image 61 from FIG. 6, and also newly displays the operation panel 71 for accepting predetermined operation input related to the medical image 52 (cross-sectional image). do. The operation panel 71 is an operation menu for providing various image analysis functions described below, and is a display field that displays a plurality of operation buttons (operation objects) for executing each function.

For example, the operation panel 71 includes a playback button 711 (playback object), a rotation button 712, a search button 713 (search object), a length measurement button 714 (length measurement object), a bookmark button 715 (bookmark object), a record button 716, and a capture button. 717 included.

The playback button 711 is a button for playing back the image file (video) of the medical image 52 (cross-sectional image) consisting of multiple frames. The first image diagnostic apparatus 2 changes the frame of the medical image 52 to be displayed in response to an operation input to the playback button 711. The rotation button 712 is a button for rotating the medical image 52 being displayed. The first image diagnostic apparatus 2 rotates the medical image 52 by an arbitrary angle around the image center point in response to an operation input to the rotation button 712.

The length measurement button 714 is a button for measuring the length of the medical image 52. The first image diagnostic apparatus 2 selects a specific object within the medical image 52, such as a lumen of a blood vessel, an external elastic membrane (EEM), a lesion site (plaque), in response to an operation input to the length measurement button 714. etc.) to measure the length.

For example, when the first image diagnostic apparatus 2 receives an operation input to the length measurement button 714, it automatically performs length measurement using the manual length measurement button 7141 for manually performing length measurement and a machine learning model. A length measurement button 7142 is displayed. The first image diagnostic apparatus 2 performs length measurement manually or automatically according to operation input to each button.

When receiving an operation input to the manual length measurement button 7141, the first image diagnostic apparatus 2 performs length measurement by the user's own screen operation. For example, the first image diagnostic apparatus 2 accepts an operation input for setting (for example, drawing) a circular line 72 corresponding to the boundary of the lumen, EEM, etc., and a straight line 73 corresponding to the diameter of the circular line 72. The first image diagnostic apparatus 2 measures the circumference and diameter of the lumen, EEM, etc. by measuring the length of the straight line 73 set by the user.

When receiving an operation input to the AI length measurement button 7142, the first image diagnostic apparatus 2 performs length measurement using the identification model 20. The identification model 20 is a machine learning model that has been trained on predetermined training data, and has been trained to identify object regions such as lumen regions and EEM regions using a cross-sectional image (medical image) of a blood vessel as input. It is.

For example, the identification model 20 is a neural network constructed by deep learning, and is a semantic segmentation model that is a type of CNN (Convolution Neural Network). A semantic segmentation model has a convolution layer (encoder) that convolves the data of the input image, and a deconvolution layer (decoder) that maps the convolved data to its original size. Identifies whether it is applicable or not in pixel units.

In this embodiment, a semantic segmentation model is used as the identification model 20 used for length measurement, but a model based on a neural network other than the semantic segmentation model or a learning algorithm such as GAN (Generative Adversarial Network) may also be used. good.

For example, the server 1 generates the identification model 20 by learning training data to which data indicating the correct object region is added to the cross-layer image for training, and transfers the data of the identification model 20 to the first image diagnostic apparatus 2. It is installed on. When receiving an operation input to the AI length measurement button 7142, the first image diagnostic apparatus 2 inputs the medical image 52 (cross-layer image) of the currently displayed frame to the identification model 20, and identifies the object area. Thereby, the first image diagnostic apparatus 2 identifies the boundary of the object area corresponding to the circular line 72. The first image diagnostic apparatus 2 measures the circumference, diameter, etc. from the identified boundary.

The search button 713 is a button for searching for similar cases similar to the patient's symptoms. The first image diagnostic apparatus 2 searches the case DB 142 for case information of similar cases similar to the patient's symptoms in response to an operation input to the search button 713.

Although the method of searching for similar cases is not particularly limited, for example, the first image diagnostic apparatus 2 extracts the feature amount of the medical image 52 being displayed (for example, a cross-sectional image of a blood vessel), and searches for similar cases based on the extracted feature amount. Search for. For example, the first image diagnostic apparatus 2 uses the identification model 20 to extract the feature amount of the medical image 52. Specifically, the first image diagnostic apparatus 2 uses the feature amount extracted by the convolution layer (encoder) that convolves the input image among the network layers constituting the identification model 20 as the feature amount of the medical image 52. The first image diagnostic apparatus 2 inputs the medical image 52 being displayed into the identification model 20 to extract the feature amount, compares it with the feature amount of the medical image of each case stored in the case DB 142, and determines the image features. Search for one or more cases that are similar in quantity (for example, cases in which the degree of similarity of feature vectors is greater than or equal to a threshold). The first image diagnostic apparatus 2 displays case information (patient information, treatment information, medical images, etc. of the case patient) of one or more similar cases that have been retrieved.

Note that the above search method is an example, and the present embodiment is not limited thereto. For example, the first image diagnostic apparatus 2 determines the degree of similarity (for example, cosine similarity) between the patient information (age, gender, diagnosis name, etc.) of the patient undergoing treatment and the patient information of each case patient stored in the case DB 142. may be calculated to search for similar cases. At this time, case information of past similar cases of the same patient may be searched preferentially. Alternatively, the first image diagnostic apparatus 2 may combine these search methods. In this way, the method of searching for similar cases does not particularly matter.

The bookmark button 715 is a button for bookmarking (registering) the frame of the medical image 52 being displayed. The first image diagnostic apparatus 2 registers frame information (series number, frame number, etc.) of the medical image 52 being displayed in response to an operation input to the bookmark button 715.

The first image diagnostic apparatus 2 displays the position within the blood vessel corresponding to the bookmarked frame in association with the longitudinal tomographic image 61. Specifically, as shown in FIG. 7, the first image diagnostic apparatus 2 displays the bookmarked frame as a bookmark image 74 in the vicinity of the longitudinal tomographic image 61. The first image diagnostic apparatus 2 displays the bookmark image 74 on the right side of the longitudinal tomographic image 61, and also displays the bookmark image 74 (cross-sectional image) so that the position in the blood vessel corresponding to the bookmarked frame can be identified. ) is displayed on the longitudinal tomographic image 61.

Note that the display function according to the bookmark is not limited to the above. For example, when playing each frame of the medical image 52 in response to an operation input to the playback button 711, the first image diagnostic apparatus 2 changes the display mode of the frame that has already been bookmarked (for example, by coloring the frame part of the frame). ) may be done. Further, for example, when displaying each medical image 52 on the time-series screen before enlargement (FIG. 5), the first image diagnostic apparatus 2 may display bookmarked frames as thumbnails.

The record button 716 is a button for recording multiple frames of the medical image 52. The first image diagnostic apparatus 2 records several frames of the medical image 52, including the frame of the medical image 52 currently being displayed, in response to an operation input to the record button 716. For example, the first image diagnostic apparatus 2 records several frames with the currently displayed frame at the center position. The capture button 717 is a button for capturing a frame of the medical image 52 being displayed. The first image diagnostic apparatus 2 captures (saves) a frame of the medical image 52 being displayed in response to an operation input to the capture button 717. Thereby, it is possible to save an image file (moving image) or a captured image of the medical image 52 of only the point of interest.

FIG. 8 is an explanatory diagram regarding the comment input operation. FIG. 8 illustrates inputting a comment 81 at an arbitrary position on the time axis 51 on a time series screen similar to FIG. 5 and the like. Although it has been explained in FIG. 5 that the icon 54 representing the treatment content (treatment device) can be inputted at any position on the time axis 51 as treatment information, the first image diagnostic apparatus 2 can also input the treatment information such as the procedure content and the doctor. It may also be possible to input any comments 81, such as impressions, as treatment information.

For example, the first image diagnostic apparatus 2 can select an arbitrary position on the time axis 51 (
Accepts a specified input specifying the point in time). The first image diagnostic apparatus 2 then receives a text input for an arbitrary comment 81 using a text input means such as a keyboard. The first image diagnostic apparatus 2 displays the input comment 81 in association with a specified position on the time axis 51, that is, a point in time during treatment.

In the above description, it is assumed that the input of the comment 81 is accepted using a text input means such as a keyboard, but the present embodiment is not limited to this. For example, the first image diagnostic apparatus 2 may receive the input of the comment 81 using a voice recognition means. For example, a user (such as a doctor performing a procedure) inputs voice using voice input means such as a headset microphone during treatment. The first image diagnostic device 2 receives the voice and converts it into text. The first image diagnostic apparatus 2 displays the converted text corresponding to the position on the time axis 51 corresponding to the voice input time point. In this way, the means for inputting comments 81 is not limited to text input.

FIG. 9 is an explanatory diagram regarding the outline display of treatment information. When a predetermined operation input (for example, a click operation on one of the switching buttons 50) is received, the first image diagnostic apparatus 2 displays an outline display field 91 on the left side of the screen. The outline display column 91 is a display column that displays a list of patient treatment steps in chronological order. The first image diagnostic apparatus 2 displays a list of treatment information corresponding to the icon 54 and the comment 81 in text in the outline display column 91 in association with the time (point in time).

As described above, the first image diagnostic apparatus 2 displays the medical image 52 and treatment information (icons 54, comments 81) of the patient undergoing treatment in chronological order. Furthermore, the first image diagnostic apparatus 2 receives a specific operation input for enlarging the time-series screen, and when the time-series screen is enlarged to a predetermined magnification, displays information related to the medical image 52, an operation panel 71 for image operation, etc. and provide access to more detailed information. Thereby, detailed information can be easily accessed while overlooking the entire treatment, and the user (medical staff) can be suitably supported.

In this embodiment, the first image diagnostic apparatus 2 generates and saves a predetermined save file so that the time series screens illustrated in FIGS. 5 to 9 can be viewed again. The saved file is data in which time information corresponding to a position on the time axis 51 is associated with the medical image 52 and treatment information (icons 54, comments 81) displayed on a time-series screen. The first image diagnostic apparatus 2 generates a save file in which time information is added as metadata to each medical image 52 and treatment information. In addition, the first image diagnostic apparatus 2 stores bookmarked frame information (series number, frame number, etc.), image files (video data) of several recorded frames of medical images 52, captured images, etc. in a storage file. . The first image diagnostic apparatus 2 or another computer can redisplay the time series screen based on the saved file.

Note that although the case where the time series screen is enlarged has been described above, the display function of the time series screen is not limited to the above.

For example, FIG. 5 illustrates a time series screen displaying information during a certain surgery. On the other hand, if an operation input for moving the time series screen in the left-right direction (direction parallel to the time axis 51) along the time axis 51 (for example, a scroll operation with a mouse, a swipe operation by touching the screen, etc.) is received, the first The image diagnostic apparatus 2 may move the screen itself in the left-right direction and display information that is temporally before or after the surgery (for example, medical images 52 captured during the previous surgery) on the same time axis 51. .

Further, although FIG. 5 illustrates a time series screen displaying two types of medical images 52, when three or more types of medical images 52 can be acquired, the first image diagnostic apparatus 2 By accepting an operation input to move the time-series screen in the intersecting direction), the screen itself may be moved in the vertical direction and a different type of medical image 52 from the two types of medical images 52 currently being displayed may be displayed. .

In this manner, the first image diagnostic apparatus 2 takes the time axis 51 in one direction, and performs medical imaging in a direction intersecting the time axis 51 (for example, in a vertical direction orthogonal to the time axis 51) according to the type (modality) of the image. A time series screen may be created in which images 52 are arranged, and detailed information may be accessible while changing the display range of the time series screen according to operations such as enlargement, reduction, movement, etc. Thereby, desired information can be freely accessed, such as a map display such as Google Maps (registered trademark), and medical personnel can be suitably supported.

FIG. 10 is a flowchart showing the processing procedure executed by the first image diagnostic apparatus 2. Based on FIG. 10, the contents of the process executed by the first image diagnostic apparatus 2 will be explained. For convenience of explanation, the first image diagnostic apparatus 2 will be described as having already acquired medical images.
The control unit 21 of the first image diagnostic apparatus 2 determines whether or not to display a time series screen based on the operation input to the switching button 50 (step S11). If it is determined that the time series screen is not to be displayed (S11: NO), the control unit 21 transitions to the screen of the mode (measurement mode, list view, etc.) selected by the switching button 50 (Step S12). The control unit 21 moves the process to step S21.

When it is determined that the time series screen is to be displayed (S11: YES), the control unit 21 displays the time axis 51 and the medical images 52 arranged corresponding to the positions on the time axis 51 corresponding to the imaging time of each image. A time series screen including the above is displayed (step S13). Specifically, the control unit 21 displays a time axis 51 in the center of the screen, and displays thumbnails of different types of medical images 52 (cross-layer images and fluoroscopic images of blood vessels) above and below the time axis 51. .

The control unit 21 receives input of treatment information representing the content of treatment for the patient in association with a position on the time axis 51 (step S14). For example, the control unit 21 receives an operation input for arranging an icon 54 (display object) representing a treatment device used for treatment at an arbitrary position within the time axis 51, which is a band-shaped display area. Further, the control unit 21 receives input of a comment 81 at an arbitrary position on the time axis 51.

The control unit 21 determines whether an enlargement operation (specific operation input) for enlarging a partial area of the time-series screen has been received (step S15). If it is determined that the enlargement operation has not been accepted (S15: NO), the control unit 21 shifts the process to step S21.

If it is determined that the enlargement operation has been accepted (S15: YES), the control unit 21 determines whether the time-series screen has been enlarged to a predetermined first magnification (Step S16). If it is determined that the image has not been enlarged to the first magnification (S16: NO), the control unit 21 waits for processing. If it is determined that the medical image 52 has been enlarged to the first magnification (S16: YES), the control unit 21 displays information related to the medical image 52 included in the enlarged area (Step S17). For example, when the enlarged medical image 52 is a cross-sectional image of a blood vessel, the control unit 21 displays a longitudinal tomographic image 61 that is a reconstructed cross-sectional image of a plurality of frames.

The control unit 21 determines whether the time-series screen has been enlarged to the second magnification (step S18). If it is determined that the image has not been enlarged to the second magnification (S18: NO), the control unit 21 waits for processing. If it is determined that the image has been enlarged to the second magnification (S18: YES), the control unit 21 displays an operation panel 71 for image analysis that has a plurality of operation buttons (operation objects) (Step S19).

The control unit 21 executes a subroutine for performing image analysis in response to operations on the operation panel 71 (step S20). Specifically, as described above, the control unit 21 changes (plays) and rotates the frame of the medical image 52 to be displayed, searches for similar cases, measures length, bookmarks, records, captures, and the like.

The control unit 21 determines whether or not to generate a save file (step S21). If it is determined that a save file is not to be generated (S21: NO), the control unit 21 returns the process to step S11. If it is determined that a save file is to be generated (S21: YES), the control unit 21 sets the medical image 52 and treatment information (icon 54, comment 81) on the time axis 51 at which the image was captured and the treatment information was input, respectively. A save file associated with time information indicating a time point corresponding to the position is generated and saved (step S22). The control unit 21 ends the series of processing.

FIG. 11 is a flowchart showing the processing procedure of the image analysis subroutine. Based on FIG. 11, the processing contents of the subroutine of step S20 will be explained.
The control unit 21 of the first image diagnostic apparatus 2 determines whether or not an operation input to the playback button 711 (playback object) has been received (step S31). If it is determined that the operation input to the playback button 711 has been accepted (S31: YES), the control unit 21 changes the frame of the medical image 52 being displayed according to the operation content (Step S32).

After executing the process in step S32, or in the case of NO in step S31, the control unit 21 determines whether or not an operation input to the rotation button 712 has been received (step S33). If it is determined that the operation input to the rotation button 712 has been received (S33: YES), the control unit 21 rotates the medical image 52 by the angle operated by the user (Step S34).

After executing the process in step S34, or in the case of NO in step S33, the control unit 21 determines whether or not an operation input to the search button 713 (search object) has been received (step S35). If it is determined that the operation input to the search button 713 has been accepted (S35: YES), the control unit 21 searches the case DB 142 for case information similar to the patient's symptoms, and displays the search results (Step S36).

After executing the process in step S36, or in the case of NO in step S35, the control unit 21 determines whether or not an operation input to the length measurement button 714 has been received (step S37). If it is determined that the operation input to the length measurement button 714 (length measurement object) has been received (S37: YES), the control unit 21 determines whether or not the operation input to the manual length measurement button 7141 has been received (step S38). ). If it is determined that the operation input to the manual length measurement button 7141 has been received (S38: YES), the control unit 21 measures the length of the object in the image according to the operation input from the user (step S39).

If it is determined that the operation input to the manual length measurement button 7141 is not accepted (S38: NO), that is, if the operation input to the AI length measurement button 7142 is accepted, the control unit 21 identifies the medical image 52 being displayed. The model 20 is input to identify image regions corresponding to specific objects (step S40). The control unit 21 measures the length of the identified object (step S41).

After executing the process in step S39 or S41, or in the case of NO in step S37, the control unit 21 determines whether or not an operation input to the bookmark button 715 has been received (step S42). When receiving an operation input to the bookmark button 715, the control unit 21 bookmarks (registers) the frame of the medical image 52 that is being displayed (step S43).

After executing the process in step S43, or in the case of NO in step S42, the control unit 21 determines whether or not an operation input to the recording button 716 has been received (step S44). If it is determined that the operation input to the recording button 716 has been received (S44: YES), the control unit 21 records several frames of the medical image 52, including the frame of the medical image 52 currently being displayed (step S45).

After executing the process in step S45, or in the case of NO in step S44, the control unit 21 determines whether or not an operation input to the capture button 717 has been received (step S46). If it is determined that the operation input to the capture button 717 has been accepted (S46: YES), the control unit 21 captures (saves) the frame of the medical image 52 being displayed (step S47). After executing the process in step S47, or in the case of NO in step S46, the control unit 21 returns the subroutine.

As described above, according to the first embodiment, patient information can be viewed from a bird's-eye view using a time-series screen, and a partial area of the screen can be enlarged with a simple operation such as a scroll operation, and the medical images 52 can be related to each other. Information can be viewed or various operations related to the medical image 52 can be performed. Thereby, patient information can be easily grasped, and the user (medical staff) can be suitably supported.

Further, according to the first embodiment, the state of the blood vessel can be suitably understood by making the longitudinal tomographic image 61 appear when the cross-sectional image of the blood vessel (living body lumen) is enlarged.

Further, according to the first embodiment, it is possible to measure the length of a specific object (for example, the lumen of a blood vessel, EEM, etc.) shown in the medical image 52. In particular, in this embodiment, length measurement can be suitably performed by using the identification model 20.

Further, according to the first embodiment, it is possible to easily access case information of similar cases having similar symptoms to the patient.

Further, according to the first embodiment, it is possible to reproduce the medical image 52 (for example, a cross-sectional image of a blood vessel) consisting of a plurality of frames, and bookmark (register) any frame. Particularly in this embodiment, by displaying the blood vessel position corresponding to the bookmarked frame on the longitudinal tomographic image 61, the position of the point of interest can be easily grasped.

Further, according to the first embodiment, treatment information (an icon 54 representing a treatment device used for treatment, an arbitrary comment 81, etc.) can be recorded in association with a position on the time axis 51, and for example, multiple pieces of treatment information can be recorded. It is possible to suitably support the implementation of treatment by sharing information among users (medical professionals).

Further, according to the first embodiment, by generating a save file in which the medical image 52 and treatment information are associated with time information, the time-series screen can be redisplayed (reproduced).

(Embodiment 2)
In the first embodiment, a mode in which a display screen is shared between the first image diagnostic apparatus 2 and the external display device 4 will be described. Note that the same reference numerals are given to the same content as in the first embodiment, and the explanation thereof will be omitted.

12 and 13 are explanatory diagrams showing examples of display screens of the first image diagnostic apparatus 2 according to the second embodiment. An overview of this embodiment will be explained based on FIGS. 12 and 13.

FIG. 12 is an explanatory diagram showing an example of a time-series screen according to the second embodiment. The time series screen according to this embodiment includes a share button 120. The share button 120 is an operation button for switching whether or not to share the screen displayed on the first image diagnostic apparatus 2 with the external display device 4 such as a ceiling-mounted monitor, and to switch between sharing or non-sharing of the display screen.

Usually, a plurality of monitors are installed in a catheter room where catheter treatment is performed, and various medical images 52 are displayed on each monitor. Medical staff perform treatment while checking each monitor. Among medical workers, technicians work while looking at the screen of the first image diagnostic apparatus 2.

However, a medical worker such as a doctor and a technician who operates the first image diagnostic apparatus 2 may want to view different images. For example, when analyzing a blood vessel tomogram, a technician may refer to a fluoroscopic image captured by the second image diagnostic device 3. However, since the fluoroscopic image displayed on the external display device 4 is the fluoroscopic image that the doctor wants to see at that time, it may be different from the fluoroscopic image that the technician wants to refer to.

Therefore, in this embodiment, the sharing button 120 is used to switch between sharing and non-sharing of the display screen. When the first image diagnostic apparatus 2 receives a predetermined operation input to the share button 120 (in the example of FIG. 12, an operation to the "1=2" button), the first image diagnostic apparatus 2 outputs the screen being displayed to the external display device 4. and display it. On the other hand, if another operation input to the share button 120 (operation to the buttons "1 and 2") is received, the first image diagnostic apparatus 2 does not output screen data and turns off screen sharing. In this case, for example, the external display device 4 displays the medical image 52 (tomographic image and/or fluoroscopic image) currently being captured by the first image diagnostic device 2 and/or the second image diagnostic device 3. As a result, the reference screen can be suitably distinguished between a doctor who performs treatment while checking the medical image 52 and a technician who operates the first image diagnostic device 2 to record treatment information, manipulate images, etc. be able to.

Furthermore, in this embodiment, the first image diagnostic apparatus 2 can display multiple types of medical images 52 (for example, blood vessel tomograms and fluoroscopic images) at the same time. Specifically, the first image diagnostic apparatus 2 accepts a selection input for selecting a plurality of types of medical images 52 on the time-series screen. In FIG. 12, the state in which the medical image 52 is selected is illustrated by a thick line frame.

FIG. 13 is an explanatory diagram showing an example of an analysis screen of the medical image 52. When multiple types of medical images 52 are selected, the second image diagnostic apparatus 3 transitions to the screen shown in FIG. 13. The analysis screen (second screen) is a screen for simultaneously analyzing multiple types of selected medical images 52.

The first image diagnostic apparatus 2 displays information related to the medical image 52 and the operation panel 71 on the analysis screen, similar to the time-series screen during enlarged display illustrated in FIG. In this case, the first image diagnostic apparatus 2 displays a plurality of operation panels 71 corresponding to the plurality of types of medical images 52. The user (technician) refers to related information from the screen and uses the operation panel 71 to play back the medical image 52, measure the length, search for similar cases, etc.

A share button 120 is also displayed on the analysis screen, similar to the time series screen. When sharing the display screen by inputting an operation to the share button 120, the first image diagnostic apparatus 2 outputs data on the analysis screen being displayed to the external display device 4 for display. Note that in this case, the first image diagnostic apparatus 2 shares the analysis screen with the exception of the operation panel 71. Thereby, the analysis results on the medical image 52 can be suitably shared with other medical personnel (doctors, etc.).

FIG. 14 is a flowchart showing the processing procedure executed by the first image diagnostic apparatus 2 according to the second embodiment. If NO in step S15, the first image diagnostic apparatus 2 executes the following process.
The control unit 21 of the first image diagnostic apparatus 2 determines whether a selection input for selecting a plurality of types of medical images 52 displayed on a time-series screen has been received (step S201). If it is determined that the selection input has been accepted (S201: YES), the control unit 21 transitions to an analysis screen that displays the selected plurality of types of medical images 52 (Step S202). Specifically, the control unit 21 displays a plurality of types of medical images 52 and separately displays an operation panel 71 for analyzing each type of medical image 52. The control unit 21 executes a subroutine for analyzing the medical image 52 in accordance with the operation input to each operation panel 71 (step S203). Note that the execution contents of the subroutine in step S203 are the same as in the flowchart of FIG. 11. The control unit 21 moves the process to step S21.

If it is determined that the selection input has not been accepted (S201: NO), the control unit 21 determines whether an operation input to the share button 120 (shared object) for sharing the display screen with the external display device 4 has been accepted. It is determined whether or not (step S204). If it is determined that the operation input to the share button 120 is not accepted (S204: NO), the control unit 21 shifts the process to step S21. If it is determined that the operation input to the share button 120 has been accepted (S204: YES), the control unit 21 outputs the screen being displayed on the display unit 24 to the external display device 4 for display (step S205), and executes the process. The process moves to step S21.

As described above, according to the second embodiment, the reference screen is differentiated between the user (physician, etc.) who performs treatment and the user (technician) who operates the first image diagnostic apparatus 2, and necessary information (for example, longitudinal Related information such as layer images 61, length measurement results of internal cavities, similar cases, etc.) can be shared. Furthermore, in the present embodiment, multiple types of medical images 52 (for example, blood vessel tomograms and fluoroscopic images) can be simultaneously analyzed and the results shared, thereby improving user convenience.

The embodiments disclosed herein are illustrative in all respects and should be considered not to be restrictive. The scope of the present invention is indicated by the claims rather than the above-mentioned meaning, and is intended to include meanings equivalent to the claims and all changes within the scope.

1 Server 11 Control unit 12 Main storage unit 13 Communication unit 14 Auxiliary storage unit P1 Program 2 First image diagnostic device 21 Control unit 22 Main storage unit 23 Communication unit 24 Display unit 25 Input unit 26 Input/output I/F
27 Image processing section 28 Auxiliary storage section P2 Program 20 Identification model 3 Second image diagnostic device 4 External display device 5 Electronic medical record server 50 Switch button 51 Time axis 52 Medical image 53 Patient information column 54 Icon 61 Longitudinal tomographic image 71 Operation panel 711 Play button 712 Rotate button 713 Search button 714 Measurement button 715 Bookmark button 716 Record button 717 Capture button 81 Comment 120 Share button

Claims (17)

an acquisition unit that acquires a plurality of medical images taken of a patient's biological lumen at a plurality of time points, and time information indicating the time point at which each medical image was taken;
a display unit that displays a screen including a time axis and each of the medical images arranged corresponding to a position on the time axis corresponding to the imaging time;
Equipped with a reception section that receives operation input from the user,
The display section is
When a specific operation input for enlarging the screen is received from a user , enlarging and displaying a partial area of the screen;
when the partial area is enlarged to a first magnification, displaying information related to the medical image included in the partial area;
When the partial area is further enlarged to a second magnification higher than the first magnification, a plurality of operation objects are displayed for respectively accepting predetermined operation inputs related to the medical image included in the partial area. ,
The reception unit receives an operation input to the operation object.
Information processing device. The medical image includes a cross-layer image of a blood vessel consisting of multiple frames,
The related information includes longitudinal tomographic images obtained by reconstructing the cross-sectional images of multiple frames.
The information processing device according to claim 1 . The related information includes a priority order of each related information,
The display section sequentially displays related information according to the priority order according to the magnification of the screen.
The information processing device according to claim 1 or 2. The plurality of operation objects include a length measurement object for measuring the length of an object in the medical image,
The information processing apparatus according to any one of claims 1 to 3, further comprising a length measurement unit that measures the length when an operation input to the length measurement object is accepted. The length measuring section is
When the medical image is input, inputting the medical image included in the partial area to a model that has been trained to identify an image area corresponding to a specific object to identify the image area,
The information processing device according to claim 4 , wherein the length of the object is measured based on the identified image area. The plurality of operation objects include a search object for searching for similar cases similar to the patient's symptoms,
a search unit that searches for case information of the similar case from a storage unit that stores case information of each of a plurality of cases based on the medical image or patient information regarding the patient when an operation input to the search object is received; ,
The information processing device according to any one of claims 1 to 5 , wherein the display unit displays the retrieved case information. The medical image includes an image consisting of multiple frames,
The plurality of operation objects include a playback object for changing the frame of the medical image to be displayed, and a bookmark object for registering the frame of the medical image,
The information processing device according to any one of claims 1 to 6, wherein the reception unit receives an operation input for registering a frame of the medical image being displayed. The medical image includes a cross-layer image of a blood vessel consisting of multiple frames,
The information according to claim 7 , wherein when a frame of the cross-sectional image is registered, a position corresponding to the registered frame is displayed in association with a longitudinal tomogram obtained by reconstructing the plurality of frames of the cross-sectional image. Processing equipment. The acquisition unit acquires a plurality of types of the medical images,
The display unit displays the screen on which each type of the medical image is arranged at different positions depending on the type of image in a direction intersecting the time axis . Information processing device. The reception unit receives a selection input for selecting the plurality of types of medical images,
When the plurality of types of medical images are selected, the display unit displays a second screen on which the plurality of selected medical images and the plurality of operation objects respectively corresponding to the plurality of types of medical images are arranged. The information processing device according to claim 9 . comprising a second reception unit that receives input of treatment information representing treatment details for the patient in association with the position on the time axis;
The information processing device according to any one of claims 1 to 10 , wherein when receiving the input of the treatment information, the display unit displays the treatment information on the screen in association with the position on the time axis. . The information processing apparatus according to claim 11 , wherein the second reception unit receives an input for arranging a display object representing a treatment device used for treatment on the time axis. The information processing apparatus according to claim 11 or 12 , wherein the second reception unit receives comment input from the user who treats the patient. comprising a generation unit that generates a save file in which the medical image and treatment information displayed on the screen are respectively associated with time information representing the time point corresponding to the imaging time point and the position on the time axis;
The information processing apparatus according to any one of claims 11 to 13 , wherein the display unit redisplays the screen by referring to the saved file. The display unit displays a shared object on the screen for sharing a display screen with an external display device,
The reception unit accepts an operation input to the shared object,
The information processing apparatus according to any one of claims 1 to 14 , further comprising an output unit that outputs a screen being displayed by the display unit to the external display device in response to an operation input to the shared object. Obtaining a plurality of medical images of a patient's biological lumen at a plurality of time points, and time information indicating the time point at which each medical image was taken;
Displaying on a display unit a screen including a time axis and each of the medical images arranged corresponding to a position on the time axis corresponding to the imaging time point;
Accepts operation input from the user,
When a specific operation input for enlarging the screen is received from a user, enlarging and displaying a partial area of the screen;
when the partial area is enlarged to a first magnification, displaying information related to the medical image included in the partial area;
When the partial area is further enlarged to a second magnification higher than the first magnification, a plurality of operation objects are displayed for respectively accepting predetermined operation inputs related to the medical image included in the partial area. ,
Accept operation input to the operation object
An information processing method in which processing is performed by a computer. Obtaining a plurality of medical images of a patient's biological lumen at a plurality of time points, and time information indicating the time point at which each medical image was taken;
Displaying on a display unit a screen including a time axis and each of the medical images arranged corresponding to a position on the time axis corresponding to the imaging time point;
Accepts operation input from the user,
When a specific operation input for enlarging the screen is received from a user, enlarging and displaying a partial area of the screen;
when the partial area is enlarged to a first magnification, displaying information related to the medical image included in the partial area;
When the partial area is further enlarged to a second magnification higher than the first magnification, a plurality of operation objects are displayed for respectively accepting predetermined operation inputs related to the medical image included in the partial area. ,
Accept operation input to the operation object
A program that causes a computer to perform a process.

Images