JP2021071736A - Medical information processing apparatus, x-ray diagnostic system, and production method for learned model - Google Patents

Abstract

To support a medical worker to make preparations for a procedure for the next patient as planned.SOLUTION: A medical information processing apparatus according to an embodiment comprises a data acquisition unit and a time calculation unit. The data acquisition unit sequentially acquires pieces of data related to the progress of a procedure during the execution of the procedure. The time calculation unit calculates the time required for the procedure based on the plurality of pieces of data related to the progress of the procedure acquired by the data acquisition unit at different timings during the execution of the procedure.SELECTED DRAWING: Figure 1

Description

Embodiments of the present invention relate to a medical information processing apparatus, an X-ray diagnostic apparatus, and a method for producing a trained model.

Conventionally, there are known techniques such as interventional treatment in which a procedure is performed while imaging a subject with an X-ray diagnostic apparatus. In such a procedure, the difficulty level varies greatly depending on the lesion site and shape, so that it may be difficult for a doctor or the like to predict the time required for the procedure. For this reason, it may be difficult to schedule the procedure, and it may be difficult for the medical staff to systematically prepare the procedure for the next patient.

U.S. Patent Application Publication No. 2019/0117087

An object to be solved by the present invention is to help healthcare professionals systematically prepare for the procedure for the next patient.

The medical information processing apparatus according to the embodiment includes a data acquisition unit and a time calculation unit. The data acquisition unit sequentially acquires data related to the progress of the procedure during the execution of the procedure. The time calculation unit calculates the time required for the procedure based on the data related to the progress of the plurality of procedures acquired by the data acquisition unit at different timings during the execution of the procedure.

FIG. 1 is a block diagram showing an example of the configuration of the medical information processing system according to the embodiment. FIG. 2 is a diagram showing an example of input data and output data of the trained model according to the embodiment. FIG. 3 is a diagram showing an example of image data and non-image data according to the embodiment. FIG. 4 is a diagram showing an example of input data and output data that change with the progress of the procedure according to the embodiment. FIG. 5 is a diagram showing an example of displaying the time required for the procedure according to the embodiment and the determination result of the success or failure of the procedure. FIG. 6 is a diagram showing an example of learning image data according to the embodiment. FIG. 7 is a diagram showing an example of non-image data for learning according to the embodiment. FIG. 8 is a diagram showing an example of a trained model generation method according to the embodiment. FIG. 9 is a flowchart showing an example of a flow of processing for calculating the time required for the procedure according to the embodiment and determining the success or failure of the procedure.

Hereinafter, embodiments of a medical information processing apparatus, an X-ray diagnostic apparatus, and a method for producing a trained model will be described in detail with reference to the drawings.

(Embodiment)
FIG. 1 is a block diagram showing an example of the configuration of the medical information processing system S according to the present embodiment. As shown in FIG. 1, the medical information processing system S includes an X-ray diagnostic apparatus 10 and a terminal apparatus 30.

The X-ray diagnostic apparatus 10 includes an X-ray high-voltage device 11, an X-ray tube 12, an X-ray throttle 13, a top plate 14, a C-arm 15, an X-ray detector 16, and a C-arm rotation / movement mechanism. It has a top plate moving mechanism 18, a C-arm / top plate mechanism control circuit 19, an aperture control circuit 20, a processing circuit 21, an input interface 22, displays 23a and 23b, and a storage circuit 24. The X-ray diagnostic apparatus 10 is, for example, an X-ray angiography apparatus. Further, in the present embodiment, the X-ray diagnostic apparatus 10 is an example of a medical information processing apparatus.

Among the configurations of the X-ray diagnostic device 10, the X-ray high voltage device 11, the X-ray tube 12, the X-ray throttle 13, the top plate 14, the C arm 15, the X-ray detector 16, and the C arm rotation. The moving mechanism 17, the top plate moving mechanism 18, the C-arm / top plate mechanism control circuit 19, the aperture control circuit 20, and the display 23b are installed in the inspection room R1.

Further, the processing circuit 21, the input interface 22, the display 23a, and the storage circuit 24 are installed in, for example, the operation room R2.

Laboratory R1 is a room in which a procedure is performed on a subject (patient) P. For example, in the present embodiment, as an example of the procedure, the laboratory R1 is also referred to as a catheter room because an operator such as a doctor performs catheter treatment on the subject P1. In the laboratory R1, a surgeon such as a doctor and a plurality of other medical staff perform the work. The laboratory R1 is an example of a space in which the procedure for the subject P1 in the present embodiment is performed. Medical staff includes not only doctors but also nurses or technicians. In the present embodiment, the operator and other medical staff are collectively referred to as a medical worker.

Further, the operation room R2 is, for example, a separate room adjacent to the examination room R1. In the operation room R2, an operator such as an engineer, a supervisor, or the like operates the input interface 22 or refers to the display 23a. The supervisor is, for example, a senior doctor. It should be noted that the configuration in which all the configurations of the X-ray diagnostic apparatus 10 are installed in the examination room R1 may be adopted.

Further, as shown in FIG. 1, the terminal device 30 is provided outside the examination room R1 and the operation room R2. For example, the terminal device 30 may be installed in a waiting room of an operator or a medical staff, or may be carried by each medical worker. The terminal device 30 is, for example, a PC (Personal Computer), a tablet terminal, or the like, but is not limited thereto. The number of terminal devices 30 included in the medical information processing system S is not particularly limited, and may be one or a plurality.

Further, although the terminal device 30 is described in FIG. 1 so as to be connected to the X-ray diagnostic device 10 by a wired network, the terminal device 30 may be connected to the X-ray diagnostic device 10 by a wireless network.

Further, the X-ray high voltage device 11 is a high voltage power source that generates a high voltage under the control of the processing circuit 21 and supplies the generated high voltage to the X-ray tube 12.

The X-ray tube 12 generates X-rays by using the high voltage supplied from the X-ray high voltage device 11. The X-ray diaphragm 13 narrows down the X-rays generated by the X-ray tube 12 so as to selectively irradiate the region of interest (ROI) of the subject P1 under the control of the diaphragm control circuit 20.

The top plate 14 is a bed on which the subject P1 is placed, and is arranged on a bed device (not shown). Further, the sleeper device (not shown) may be included in the X-ray diagnostic device 10 or may be outside the X-ray diagnostic device 10. The sleeper device may be included in the medical information processing system S even if it is not included in the X-ray diagnostic device 10. The subject P1 is not included in the X-ray diagnostic apparatus 10.

The X-ray detector 16 detects the X-rays that have passed through the subject P1 and transmits the detection result to the processing circuit 21.

The C-arm 15 holds an X-ray tube 12, an X-ray diaphragm 13, and an X-ray detector 16. The C-arm rotation / movement mechanism 17 is a mechanism for rotating and moving the C-arm 15 by driving a motor or the like provided on the support. The top plate moving mechanism 18 is a mechanism for moving the top plate 14. For example, the top plate moving mechanism 18 moves the top plate 14 by using the power generated by the actuator.

The C-arm / top plate mechanism control circuit 19 controls the C-arm rotation / movement mechanism 17 and the top plate movement mechanism 18 under the control of the processing circuit 21 to rotate and move the C-arm 15 and the top plate 14. Adjust the movement. The diaphragm control circuit 20 controls the irradiation range of X-rays irradiated to the subject P1 by adjusting the opening degree of the diaphragm blades of the X-ray diaphragm 13 under the control of the processing circuit 21.

The input interface 22 is realized by a trackball, a switch button, a mouse, a keyboard, a touch pad, etc., a foot switch for irradiating X-rays, and the like. The input interface 22 is connected to the processing circuit 21, converts the input operation received from the operator into an electric signal, and outputs the input operation to the processing circuit 21.

The displays 23a and 23b display a GUI (Graphical User Interface) for receiving an operator's instruction and various images generated by the processing circuit 21. In the present embodiment, the displays 23a and 23b display the time required for the procedure output from the learned model described later.

More specifically, the display 23a is installed, for example, suspended from the ceiling of the examination room R1. The display 23a is not limited to one screen, and may have a plurality of screens. The display 23a is also referred to as an inspection room display.

Further, the display 23b is installed in the operation room R2. The display 23b is also referred to as an operation room display. The number of displays 23b is not particularly limited, and may be one or a plurality of displays. Hereinafter, when the displays 23a and 23b are not particularly distinguished, they are simply referred to as the display 23.

Further, in the X-ray diagnostic apparatus 10, each processing function is stored in the storage circuit 24 in the form of a program that can be executed by a computer. The C-arm / top plate mechanism control circuit 19, the aperture control circuit 20, and the processing circuit 21 are processors that realize functions corresponding to each program by reading a program from the storage circuit 24 and executing the program. In other words, each circuit in the state where each program is read has a function corresponding to the read program.

The storage circuit 24 stores programs corresponding to various functions read and executed by each circuit shown in FIG. Further, the storage circuit 24 stores logs of various processes executed by the processing circuit 21. The storage circuit 24 is realized by, for example, a RAM (Random Access Memory), a semiconductor memory element such as a flash memory, a hard disk, an optical disk, or the like. The storage circuit 24 is an example of a storage unit.

The processing circuit 21 has a data acquisition function 211, a learned model operation function 212, an output function 213, a reception function 214, an imaging function 215, a learning data generation function 216, and a learning function 217. The data acquisition function 211 is an example of a data acquisition unit. The trained model operation function 212 is an example of the trained model operation unit. Further, the trained model operation function 212 has a time calculation function and a determination function. The time calculation function is an example of the time calculation unit. The determination function is an example of a determination unit. The output function 213 is an example of an output unit. The reception function 214 is an example of the reception unit. The image pickup function 215 is an example of an image pickup unit. The learning data generation function 216 is an example of a learning data generation unit. The learning function 217 is an example of the learning unit.

Further, the process executed by the data acquisition function 211 is referred to as a data acquisition step. The process executed by the trained model operation function 212 is referred to as a trained model operation step, a time calculation step, or a determination step. The process executed by the output function 213 is called an output step. The process executed by the reception function 214 is called a reception step. The process executed by the image pickup function 215 is called an image pickup step. The process executed by the training data generation function 216 is called a training data generation step. The process executed by the learning function 217 is called a learning step.

In addition, in FIG. 1, in a single processing circuit 21, a data acquisition function 211, a trained model operation function 212, an output function 213, a reception function 214, an imaging function 215, a training data generation function 216, and a learning function 217 Although each of the above processing functions has been described as being realized, a processing circuit 21 may be formed by combining a plurality of independent processors, and each processing function may be realized by executing each program by each processor.

The word "processor" used in the above description means, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an integrated circuit for a specific application (Application Specific Integrated Circuit: ASIC), or a programmable logic device (for example, a programmable logic device). It means a circuit such as a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA). Instead of storing the program in the storage circuit 24, the program may be directly incorporated in the circuit of the processor. In this case, the processor realizes the function by reading and executing the program embedded in the circuit.

The data acquisition function 211 sequentially acquires data related to the progress of the procedure during the execution of the procedure.

The data related to the progress of the procedure includes image data and non-image data.

The image data in the present embodiment is a medical image obtained by imaging the subject P1 to which the procedure is performed. More specifically, the image data in the present embodiment is an X-ray image captured by the X-ray diagnostic apparatus 10.

In addition, the non-image data in the present embodiment includes procedure information regarding the procedure, subject information regarding the subject P1, operator information regarding the operator performing the procedure, measurement data measured during the procedure, or equipment used for the procedure. Includes at least one of the device information for. The non-image data in this embodiment is an example of non-image information.

The procedure information is, for example, the type of procedure. The type of procedure is specifically the type of protocol or program executed by the X-ray diagnostic apparatus 10 when performing the procedure. For example, a protocol is a processing procedure defined for each treatment target site. The program is a process executed by each protocol, and as an example, is a program that executes a scan step such as DSA (Digital Subtraction Angiography).

Further, the procedure information may be information indicating the difficulty level of the procedure. Further, the procedure information may be information indicating the type of approach. The procedure approach is, for example, an antegrade approach or a retrograde approach.

The subject information regarding the subject P1 is, for example, information regarding the severity of the medical condition of the subject P1, the age of the subject P1, the sex of the subject P1, and the like. Further, the subject information may be information indicating the state of the affected portion of the subject P1. The information representing the state of the affected part of the subject P1 is, for example, the formation status of the collateral flow (collateral flow).

The operator information regarding the operator performing the procedure is, for example, information indicating the skill level of the operator. For example, the surgeon information may be in a category according to the skill level such as "young", "experienced", and "instructor". In addition, the surgeon information may represent the skill level of the surgeon numerically. The operator information is not limited to this, and may include various information other than the skill level.

Further, the measurement data measured during the procedure is a measurement result regarding the state of the treatment target portion of the subject P1 during the procedure. For example, when the procedure is percutaneous coronary intervention (PCI) for CTO (Chronic Total Occlusion), the measurement data is the measurement result at the occlusion point of the coronary artery. The measurement data is, for example, the measurement result of the Coronary Blood Flow Reserve (FFR).

The device information regarding the device used in the procedure is, for example, information about the X-ray diagnostic apparatus 10 that images the subject P1 during the procedure, or the device used in the procedure.

For example, the device information includes the position information of the movable mechanism included in the X-ray diagnostic apparatus 10. The movable mechanism is, for example, a C arm 15 or a top plate 14. For example, the data acquisition function 211 acquires the position information of the C arm 15 or the top plate 14 from the C arm / top plate mechanism control circuit 19. For example, the position of the C arm 15 in the procedure is called a working angle.

Further, the device information may be the operating state of the application used for the procedure. For example, when an operator places a stent in a blood vessel during a procedure, an application that generates a stent-enhanced image of the X-ray diagnostic apparatus 10 may be executed. The device information may be information indicating whether or not an application executed by the X-ray diagnostic apparatus 10 is executed during such a procedure.

Further, for example, the device information may be information about the device used in the procedure. Information about the device used in the procedure is, for example, the number of catheters used in the procedure, or the type of catheter. The device is not limited to the catheter, and may be a balloon, a stent, or the like. For example, the device information may be information about the type of stent.

Further, the device information may be information indicating the operating state of the device during the procedure. For example, the device information may be the expansion time of the balloon during the procedure.

The non-image data is not limited to the above example, and may further include other information. The data acquisition function 211 sends the acquired image data and non-image data to the trained model operation function 212.

The trained model operation function 212 has a time calculation function for calculating the time required for the procedure and a determination function for determining the success or failure of the procedure.

For example, the trained model operation function 212 calculates the time required for the procedure based on the plurality of image data and the plurality of non-image data acquired by the data acquisition function 211 at different timings during the execution of the procedure.

In the present embodiment, the time required for the procedure is the remaining time until the end of the procedure. The end of the procedure may be a success of the procedure or a cancellation due to a failure of the procedure. The trained model operation function 212 may calculate the time from the start to the end of the procedure as the time required for the procedure.

Further, the trained model operation function 212 determines the success or failure of the procedure based on the plurality of image data and the plurality of non-image data acquired by the data acquisition function 211 at different timings during the execution of the procedure.

The success or failure of the procedure shall be either "successful" or "unsuccessful". Further, in the present embodiment, the success of the procedure is to achieve the purpose of the procedure. The purpose of the procedure is, for example, restoration of blood flow in the coronary arteries, but it depends on the procedure. In addition, the unsuccessful procedure is, for example, when the surgeon abandons the continuation of the procedure and the procedure is stopped.

The success or failure of the procedure determined by the trained model operation function 212 is an estimation result of the result of the ongoing procedure. In other words, the fact that the trained model operation function 212 determines “success” means that “the procedure currently being continued is likely to succeed”. Further, the fact that the trained model operation function 212 determines "unsuccessful" means that "the procedure currently being continued is likely to result in unsuccessfulness".

More specifically, the trained model operation function 212 reads the trained model from the storage circuit 24 and inputs the image data and the non-image data acquired by the data acquisition function 211 into the trained model. The required time and the result of determining the success or failure of the procedure are output to the trained model.

The time required for the procedure and the result of determining the success or failure of the procedure are the output data of the trained model of the present embodiment.

FIG. 2 is a diagram showing an example of input data and output data 80 of the trained model 90 according to the present embodiment. As shown in FIG. 2, when the image data 71 and the non-image data 72 are input, the trained model 90 outputs the time required for the procedure corresponding to the input data and the judgment result of the success or failure of the procedure. Output as data 80.

The trained model 90 is, for example, a trained model generated by deep learning (deep learning) such as a neural network. The trained model 90 is composed of a neural network and trained parameter data. As a deep learning method, for example, a recurrent neural network (RNN), an LSTM (Long short-term memory), or the like can be applied. Further, the trained model 90 may be generated by other deep learning or machine learning. In the present embodiment, it is assumed that the trained model 90 is generated by the learning function 217 described later and stored in the storage circuit 24.

Next, the details of the image data 71 and the non-image data 72 input to the trained model 90 in the present embodiment will be described.

FIG. 3 is a diagram showing an example of image data 71 and non-image data 72 according to the present embodiment. The plurality of image data 71a to 71g and the plurality of non-image data 72a to 72l shown in FIG. 3 are the image data 71 and the non-image data 72 acquired at different timings during the execution of the procedure by the data acquisition function 211.

Specifically, the plurality of image data 71a to 71g shown in FIG. 3 are a plurality of X-ray images captured from the start to the end of the procedure when the procedure is CTO. In FIG. 3, the timing at which each image data 71 is acquired is displayed in association with the elapsed time from the start of the procedure.

From the plurality of captured X-ray images, information such as the degree of progress of the procedure, the speed of the procedure, and the possibility of success can be obtained. For example, the image data 71a is an X-ray image obtained by contrast-enhancing the blood vessel of the treatment target portion by the X-ray diagnostic apparatus 10 before introducing the catheter into the blood vessel of the subject P1. Further, the image data 71b to 71d are X-ray images in which the state in which the catheter introduced into the blood vessel of the subject P1 by the operator advances in the blood vessel is sequentially imaged. In the example shown in FIG. 3, it takes 60 minutes to reach the state of the image data 71d, but the speed of the procedure varies depending on the skill level of the operator, the severity of the subject P1 and the like.

Further, the image data 71e is an X-ray image in which the balloon expands in the blood vessel of the subject P1. Further, the image data 71f is an X-ray image in which the stent placed in the blood vessel of the subject P1 is imaged. If the stent is visualized on the X-ray image as in the image data 71f, it becomes clear that the progress of the procedure has progressed to the placement of the stent.

Further, the image data 71g is an X-ray image obtained by contrast-enhancing the blood vessel of the treatment target portion of the subject P1 again at the end of the catheter treatment. In the image data 71a, the blood vessel that was occluded in the middle is conducting in the image data 71g, and it can be seen that the procedure is successful.

Hereinafter, when the individual image data 71a to 71g are not distinguished, it is simply referred to as image data 71. When the individual non-image data 72a to 72l are not distinguished, it is simply referred to as non-image data 72.

All the X-ray images captured during the procedure may be input to the trained model 90 as image data 71, or the latest X-ray image may be input to the trained model 90 as image data 71 at regular intervals. It may be entered. The fixed time is not particularly limited, and is, for example, 5 minutes or 10 minutes. Alternatively, each time the X-ray diagnostic apparatus 10 captures an X-ray image at an X-ray dose higher than that through fluoroscopy, the trained model operation function 212 may input the X-ray image into the trained model 90.

Further, the plurality of non-image data 72a to 72l shown in FIG. 3 are non-image data 72 sequentially acquired by the data acquisition function 211 as the procedure progresses.

The trained model 90 is subjected to a procedure based on, for example, image data 71 and various non-image data 72 such as the skill level of the operator represented in the operator information and the severity of the medical condition of the subject P1. Estimate the time required and the result of determining the success or failure of the procedure.

Further, the acquisition source of the non-image data 72 by the data acquisition function 211 is not particularly limited, but for example, the data acquisition function 211 may acquire the subject information from an external electronic medical record system or the like. Further, the data acquisition function 211 may acquire the protocol used for the procedure, the program, the operating status of the application used, and the like from the log stored in the storage circuit 24. Further, the data acquisition function 211 may acquire the type of catheter, the type of approach, etc. input by the operator from the input interface 22.

Further, FIG. 4 is a diagram showing an example of input data and output data that change with the progress of the procedure according to the present embodiment. As shown in FIG. 4, the number of image data 71 and non-image data 72 input to the trained model 90 increases with the passage of time.

The upper part of FIG. 4 is an example of image data 71, non-image data 72, and output data 80 10 minutes after the start of the procedure. At this point, the trained model 90 outputs that the procedure is successful and that the time required for the procedure, that is, the time to complete the procedure is 90 minutes.

The middle part of FIG. 4 is an example of image data 71, non-image data 72, and output data 80 60 minutes after the start of the procedure. At this point, the trained model 90 outputs that the procedure is successful and that the time required for the procedure is 40 minutes.

Further, the lower part of FIG. 4 is an example of image data 71, non-image data 72, and output data 80 100 minutes after the start of the procedure. At this point, the trained model 90 outputs that the procedure was successful and that the time required for the procedure was 0 minutes, that is, the procedure was completed.

The trained model operation function 212 sends the output data 80 output from the trained model 90, that is, the time required for the procedure and the determination result of the success or failure of the procedure to the output function 213.

Returning to FIG. 1, the output function 213 outputs the time required for the procedure calculated by the trained model operation function 212. Further, the output function 213 outputs the success or failure of the procedure determined by the learned model operation function 212.

More specifically, the output function 213 displays the contents of the output data 80 on the displays 23a and 23b. FIG. 5 is a diagram showing an example of displaying the time required for the procedure according to the present embodiment and the determination result of the success or failure of the procedure. For example, as shown in FIG. 5, the output function 213 displays the time required for the procedure as "30 minutes left" on the display 23a installed in the examination room R1.

For example, the output function 213 displays the time required for the procedure and the result of determining the success or failure of the procedure as "success, 30 minutes left" on the display 23b installed in the operation room R2. The supervisor P3 and the like in the operation room R2 refer to the time required for the procedure displayed on the display 23b and the judgment result of the success or failure of the procedure. The supervisor P3 or the like may, for example, give an instruction to the operator P2 of the examination room R1 to change the approach of the procedure or change the device, or change to another operator, if necessary. You may instruct. The output function 213 may display a message suggesting a change in the approach or the operator on the display 23b when the success / failure determination result of the procedure is “unsuccessful”.

Further, in the example shown in FIG. 5, the output function 213 does not display the success / failure determination result of the procedure on the display 23a. This is because it may be better not to disclose the result of determining the success or failure of the procedure in the laboratory R1 where the operator P2 or the subject P1 is located.

The display content shown in FIG. 5 is an example, and the output function 213 displays the success / failure judgment result of the procedure on the display 23a when the success / failure judgment result of the procedure is “success”, and determines the success / failure of the procedure. If the determination result is "unsuccessful", the display content may be changed according to the determination result of the success or failure of the procedure so that the determination result of the success or failure of the procedure is not displayed on the display 23a. Further, the output function 213 may display the same contents on the displays 23a and 23b.

Further, the output function 213 causes the contents of the output data 80 to be displayed on the display 301 of the terminal device 30 provided outside the inspection room R1 and the operation room R2. For example, the output function 213 displays a message including the time required for the procedure, "The procedure will be completed in 30 minutes. Please start the preparation." As shown in FIG. In the example shown in FIG. 5, the output function 213 displays a message prompting the medical staff in charge of the procedure of the next patient to start the preparation, "Please start the preparation." It may be changed according to the length of time required for the procedure. For example, the output function 213 displays a message instructing standby when the time required for the procedure is longer than 30 minutes, and displays a message prompting the start of preparation when the time required for the procedure is 30 minutes or less. It may be something to do.

Further, the output function 213 controls a speaker (not shown) installed in the examination room R1, the operation room R2, the waiting room of the operator or the medical staff, etc., and waits for the time required for the procedure, the result of determining the success or failure of the procedure, and the standby. A message instructing the above, a message prompting the start of preparation, or the like may be output by voice.

Returning to FIG. 1, the reception function 214 receives the operation input by the operator via the input interface 22. For example, the reception function 214 accepts an operation of starting or ending the imaging process of the X-ray diagnostic apparatus 10 input by the operator. When the reception function 214 accepts the operation of starting or ending the imaging process, it sends to the imaging function 215 that the operation of starting or ending the imaging process has been accepted.

The imaging function 215 controls the entire X-ray diagnostic apparatus 10 based on the operation of starting or ending the imaging process received by the reception function 214, and executes the imaging process of imaging the subject P1. For example, the image pickup function 215 controls the rotation and movement of the C arm 15 and the movement of the top plate 14 by controlling the C arm / top plate mechanism control circuit 19. Further, the imaging function 215 controls the irradiation of the subject P1 with X-rays by controlling the X-ray high voltage device 11 and the aperture control circuit 20. Further, the imaging function 215 collects an X-ray image based on an electric signal converted from the X-ray by the X-ray detector 16. The imaging function 215 sends the collected X-ray image to the data acquisition function 211.

The learning data generation function 216 generates a plurality of learning data by cutting out a set of time-series data related to the progress of the procedure at different time lengths. The training data is data used by the learning function 217, which will be described later, to generate the trained model 90. The learning data includes learning image data and learning non-image data.

The generation of the learning data by the learning data generation function 216 will be described with reference to FIGS. 6 and 7.

FIG. 6 is a diagram showing an example of learning image data according to the present embodiment. For example, the image data group 701 shown in FIG. 6 is a set of time-series data in which a plurality of X-ray images are arranged in time series in the order of imaging. Further, the image data group 701 is image data in one case in which the time from the start to the end of the procedure (hereinafter referred to as the procedure time) is 100 minutes and the result of the procedure is "success". Each of the plurality of X-ray images included in the image data group 701 is associated with the elapsed time from the start of the procedure.

The learning data generation function 216 divides the image data group 701 into a combination of a plurality of learning image data 1071. For example, the learning data generation function 216 divides the image data group 701 at regular elapsed time intervals.

In the example shown in FIG. 6, the learning data generation function 216 uses the X-ray image from the start of the procedure to 100 minutes, that is, all the X-ray images included in the image data group 701 as the learning image data 1071a. Further, the learning data generation function 216 uses the X-ray image from the start of the procedure to 60 minutes as the learning image data 1071b. Further, the learning data generation function 216 uses the X-ray image from the start of the procedure to 10 minutes as the learning image data 1071c. Hereinafter, when the learning image data 1071a to 1071c are not particularly distinguished, they are simply referred to as learning image data 1071.

The elapsed time as a reference for the learning data generation function 216 to divide the image data group 701 is not particularly limited, and may be a fixed-length time unit such as 5 minutes or 10 minutes, or a variable-length time. It may be a unit.

Further, the learning data generation function 216 provides teacher data 1080a to 1080c corresponding to each of the divided learning image data 1071a to 1071c according to the elapsed time corresponding to the divided portion of the image data group 701 (hereinafter, referred to as Unless otherwise specified, simply generate teacher data (referred to as 1080).

The teacher data 1080a to 1080c is the time required for the procedure corresponding to each of the learning image data 1071a to 1071c and the success or failure of the procedure. The success or failure of the procedure is the same as the result of the procedure in the image data group 701. The time required for the procedure corresponding to each of the learning image data 1071a to 1071c is the time obtained by subtracting the elapsed time of each of the teacher data 1080a to 1080c from the procedure time in the image data group 701.

In the example shown in FIG. 6, the combination of the learning image data 1071a and the teacher data 1080a is the learning data set 1A, the combination of the learning image data 1071b and the teacher data 1080b is the learning data set 2A, the learning image data 1071c and the teacher. Let the combination of data 1080c be the training data set 3A.

Further, FIG. 7 is a diagram showing an example of non-image data for learning according to the present embodiment. The non-image data group 702 shown in FIG. 7 is a set of time-series data in which the non-image data generated in the case where the image data group 701 shown in FIG. 6 is imaged is arranged in time series.

The learning data generation function 216 divides the non-image data group 702 according to the elapsed time corresponding to the divided portion of the image data group 701. In the example shown in FIG. 7, the learning data generation function 216 is 100 minutes from the start of the procedure, 60 minutes from the start of the procedure, and 10 minutes from the start of the procedure, similarly to the image data group 701 shown in FIG. Non-image data for learning 1072a to 1072c corresponding to the elapsed time are generated. Hereinafter, when the individual learning non-image data 1072a to 1072c are not distinguished, they are simply referred to as learning non-image data 1072.

The learning data set 1B, which is a combination of the learning non-image data 1072a and the teacher data 1080a shown in FIG. 7, corresponds to the learning data set 1A of FIG. Further, the learning data set 2B, which is a combination of the learning non-image data 1072b and the teacher data 1080b, corresponds to the learning data set 1B of FIG. Further, the learning data set 2C, which is a combination of the learning non-image data 1072c and the teacher data 1080c, corresponds to the learning data set 1C of FIG.

The teacher data 1080a to 1080b shown in FIG. 7 is the same as that in FIG. That is, one learning image data 1071, one learning non-image data 1072, and one teacher data 1080 form one learning data set.

As described with reference to FIGS. 6 and 7, the learning data generation function 216 generates a plurality of learning data sets from the image data group 701 and the non-image data group 702 in one case. Further, the learning data generation function 216 executes the division processing as shown in FIGS. 6 and 7 on the plurality of image data groups 701 and the plurality of non-image data groups 702 in the plurality of cases, and the learning data generation function 216 executes the division processing as shown in FIGS. From each, a plurality of training data sets are generated.

Returning to FIG. 1, the learning function 217 generates the trained model 90 by learning the correspondence between the plurality of training data generated by the learning data generation function 216 and the time required for the procedure and the success or failure of the procedure. .. The generation process of the trained model 90 by the learning function 217 is also referred to as the production process of the trained model 90.

The learning function 217 learns by associating a plurality of learning image data 1071 included in a plurality of learning data sets, a plurality of non-image data 1072 for learning, and a plurality of teacher data 1080 in association with each other.

FIG. 8 is a diagram showing an example of a method for generating the trained model 90 according to the present embodiment. As shown in FIG. 8, the learning function 217 is based on a plurality of learning image data 1071a to 1071n, a plurality of learning non-image data 1072a to 1072n, and a plurality of teacher data 1080a to 1080n, such as RNN or LSTM. Deep learning is executed to generate a trained model 90.

The learning function 217 stores the generated learned model 90 in the storage circuit 24.

Next, a flow of processing for calculating the time required for the procedure executed by the X-ray diagnostic apparatus 10 of the medical information processing system S of the medical information processing system S of the present embodiment configured as described above and determining the success or failure of the procedure will be described.

FIG. 9 is a flowchart showing an example of a flow of processing for calculating the time required for the procedure according to the present embodiment and determining the success or failure of the procedure. The process shown in FIG. 9 starts, for example, when the image pickup process of the subject P1 by the image pickup function 215 is started. Further, it is assumed that the trained model 90 is generated by the learning function 217 and stored in the storage circuit 24 before the start of this flowchart.

First, the data acquisition function 211 acquires non-image data 72 from an external device, an input interface 22, or the like (S1). Further, the data acquisition function 211 acquires the image data 71, that is, the X-ray image of the subject P1 imaged by the X-ray diagnostic apparatus 10 (S2). The data acquisition function 211 sends the acquired image data and non-image data to the trained model operation function 212.

Next, the trained model operation function 212 reads the trained model 90 from the storage circuit 24, and inputs the X-ray image acquired by the data acquisition function 211 and the non-image data 72 into the trained model 90 (S3). ). The trained model 90 outputs the input X-ray image, the time required for the procedure corresponding to the non-image data 72, and the determination result of the success or failure of the procedure. The trained model operation function 212 sends the output result of the trained model 90 to the output function 213.

Next, the output function 213 outputs both or either of the time required for the procedure calculated by the trained model operation function 212 and the success or failure of the procedure determined by the trained model operation function 212 (S4). Specifically, the output function 213 displays the time required for the procedure on the display 23a installed in the examination room R1. Further, the output function 213 displays the time required for the procedure and the determination result of the success or failure of the procedure on the display 23b installed in the operation room R2. Further, the output function 213 displays the time required for the procedure on the display 301 of the terminal device 30 provided outside the examination room R1 and the operation room R2.

Next, the reception function 214 determines whether or not the operation for ending the procedure input by the operator has been accepted (S5). The operation for ending the procedure is, for example, an operation for ending the imaging process. When it is determined that the reception function 214 does not accept the operation for ending the procedure (S5 “No”), the process returns to the process of S1.

Further, when it is determined that the reception function 214 has accepted the operation of ending the procedure (S12 “Yes”), the processing of this flowchart ends.

Conventionally, in a procedure such as intervention treatment, the difficulty level varies greatly depending on the lesion site and shape, so that it may be difficult to estimate the time required for the procedure. For this reason, it may be difficult to schedule the procedure, and it may be difficult for the medical staff to systematically prepare the procedure for the next patient.

On the other hand, the X-ray diagnostic apparatus 10 of the present embodiment calculates the time required for the procedure based on the data related to the progress of the plurality of procedures acquired at different timings during the procedure. Therefore, according to the X-ray diagnostic apparatus 10 of the present embodiment, it is possible to support the medical staff in systematically preparing the procedure for the next patient.

Further, the X-ray diagnostic apparatus 10 of the present embodiment determines the success or failure of the procedure based on the data related to the progress of the plurality of procedures acquired at different timings during the procedure. Therefore, according to the X-ray diagnostic apparatus 10 of the present embodiment, the success or failure of the procedure can be estimated during the procedure.

For example, if the success or failure of the procedure is determined during the procedure, the operator P2 or the supervisor P3 can use it as a judgment material for determining whether or not to interrupt the procedure. In addition to the interruption of the procedure, the success or failure of the procedure can also be used as a basis for determining whether the doctor who is the current surgeon P2 is replaced by another doctor with a higher skill level or a change in the approach of the procedure. Therefore, according to the X-ray diagnostic apparatus 10 of the present embodiment, the operator P2 or the supervisor P3 is assisted in determining whether or not the procedure is continued, the operator P2 is replaced, or the approach of the procedure is changed. be able to.

Further, in the present embodiment, the data used for calculating the time required for the procedure or determining the success or failure of the procedure includes a medical image of the subject P1 to which the procedure is performed. Therefore, according to the X-ray diagnostic apparatus 10 of the present embodiment, the time required for the procedure or the success or failure of the procedure can be estimated with high accuracy based on the change in the state of the treatment target portion of the subject P during the procedure. it can.

More specifically, in the present embodiment, the X-ray diagnostic apparatus 10 calculates the time required for the procedure or determines the success or failure of the procedure based on the X-ray image of the subject P1. Since the X-ray image of the subject P1 depicts the state of a device such as a catheter or a stent or the treatment target portion of the subject P1, the X-ray diagnostic apparatus 10 uses the X-ray image captured during the procedure. , Information on the progress of the procedure, the speed of the procedure, the possibility of success, etc. can be obtained, and the time required for the procedure or the success or failure of the procedure can be estimated with high accuracy.

Further, the X-ray diagnostic apparatus 10 of the present embodiment calculates the time required for the procedure or determines the success or failure of the procedure based on the non-image data 72. More specifically, the X-ray diagnostic apparatus 10 of the present embodiment includes procedure information regarding the procedure, subject information regarding the subject P1, operator P2 information regarding the operator P2 performing the procedure, and measurement data measured during the procedure. Alternatively, the time required for the procedure is calculated or the success or failure of the procedure is determined based on the non-image data 72 including at least one of the device information regarding the device used for the procedure. The speed of the procedure and the possibility of success of the procedure differ depending on the skill level of the operator P2, the severity of the subject P1, and the like. The X-ray diagnostic apparatus 10 of the present embodiment calculates the time required for the procedure or determines the success or failure of the procedure based on such non-image data 72, thereby accurately determining the time required for the procedure or the success or failure of the procedure. Can be estimated to.

Further, according to the X-ray diagnostic apparatus 10 of the present embodiment, in order to output the time required for the procedure, the operator P2, the supervisor P3, the medical staff in charge of the procedure of the next patient, or the like can perform the procedure. It is possible to grasp the time until the end. Therefore, according to the X-ray diagnostic apparatus 10 of the present embodiment, it is possible to support the operator P2, the supervisor P3, the medical staff in charge of the procedure of the next patient, or the like to proceed with the procedure in the plan. it can.

Further, according to the X-ray diagnostic apparatus 10 of the present embodiment, in order to output the success or failure of the procedure, the operator P2 or the supervisor P3 or the like determines whether or not the procedure is continued, the change of the operator P2, or the approach of the procedure. It can help you make decisions about changes, etc.

Further, the X-ray diagnostic apparatus 10 of the present embodiment generates a plurality of learning data by cutting out a set of time-series data related to the progress of the procedure at different time lengths, and the learning data and the time required for the procedure. Learn the correspondence with. Therefore, according to the X-ray diagnostic apparatus 10 of the present embodiment, a plurality of learning data can be generated from one case, and the trained model 90 can be efficiently produced.

The trained model 90 in the present embodiment further updates the internal algorithm of the trained model 90 by acquiring user feedback on the time required for the procedure output by the X-ray diagnostic apparatus 10 and the success or failure of the procedure. It shall include a "self-learning model".

(Modification example 1)
In the above-described embodiment, the X-ray diagnostic apparatus 10 is used as an example of the medical information processing apparatus, but the medical information processing apparatus may be an information processing apparatus different from the X-ray diagnostic apparatus 10. For example, the medical information processing device may be a PC (Personal Computer) or a server device capable of communicating with the X-ray diagnostic device 10 via an in-hospital network or the like.

Further, the medical information processing apparatus may be various modalities for capturing a still image or a moving image of the subject P1 during the procedure. For example, the medical information processing device may be an endoscopic device. When this configuration is adopted, a medical image such as an in-vivo image of the subject P1 captured by the endoscope device is an example of the image data 71 and the learning image data 1071.

Further, the medical information processing device may be a telemedicine support system or the like that performs surgical treatment such as surgery by remote control. When this configuration is adopted, the captured image of the treatment target portion of the subject P1 imaged by the telemedicine support system is an example of the image data 71 and the learning image data 1071.

The image data 71 and the learning image data 1071 are not limited to those captured as still images, and may be frames included in the moving image.

(Modification 2)
Further, in the above-described embodiment, it is assumed that the learning data generation function 216 and the learning function 217 execute the generation of the training data and the generation of the trained model 90. However, the trained model 90 is another information processing apparatus. It may be generated by. When adopting this configuration, the X-ray diagnostic apparatus 10 acquires the learned model 90 from another information processing apparatus and stores it in the storage circuit 24.

(Modification example 3)
In the above-described embodiment, the trained model 90 is stored in the storage circuit 24, but the trained model 90 may be incorporated in the trained model operation function 212.

(Modification example 4)
In the above-described embodiment, the trained model 90 outputs both the time required for the procedure and the success / failure of the procedure, but only calculates the time required for the procedure or determines the success / failure of the procedure. It may be output.

Further, the X-ray diagnostic apparatus 10 may have a trained model for calculating the time required for the procedure and a trained model for determining the success or failure of the procedure as separate trained models. When this configuration is adopted, the trained model for calculating the time required for the procedure may be incorporated in the time calculation function, and the trained model for determining the success or failure of the procedure may be incorporated in the determination function.

(Modification 5)
In the above-described embodiment, the data related to the progress of the procedure includes both the image data 71 and the non-image data 72, but only one of the image data 71 and the non-image data 72 may be used.

(Modification 6)
In the above-described embodiment, the X-ray diagnostic apparatus 10 outputs the time required for the procedure and the determination result of the success or failure of the procedure, but the output information is not limited to these.

For example, the trained model 90 of the X-ray diagnostic apparatus 10 may output the probability of success of the procedure with a value of 0 to 100%. The output format is not limited to%, and the trained model 90 may output an evaluation value indicating the degree of success of the procedure.

Further, the trained model 90 may output the achievement level of the current procedure as a value of 0 to 100%. For example, in the procedure in which three stents were planned to be inserted, when one insertion is completed, the trained model 90 outputs a degree of achievement of 30%. Further, the trained model 90 may output an estimated value of achievement at the end of the procedure as a value of 0 to 100%.

(Modification 7)
In the above-described embodiment, the learning function 217 executes supervised learning by the teacher data 1080 generated by the training data generation function 216 when the trained model 90 is generated. For example, semi-supervised learning may be adopted.

(Modification 8)
Further, the trained model 90 may be constructed by an integrated circuit such as an ASIC or FPGA. Further, the X-ray diagnostic apparatus 10 uses a mathematical model, a look-up table, a database, or the like instead of the trained model 90 generated by the neural network to calculate the time required for the procedure or determine the success or failure of the procedure. You may.

According to at least one embodiment described above, it is possible to assist the healthcare professional in systematically preparing the procedure for the next patient.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

21 Processing circuit 23, 23a, 23b Display 24 Storage circuit 30 Terminal device 71, 71a to 71g Image data 72, 72a to 72l Non-image data 80 Output data 90 Learned model 211 Data acquisition function 212 Learned model operation function 213 Output function 214 Reception function 215 Imaging function 216 Learning data generation function 217 Learning function 301 Display 701 Image data group 702 Non-image data group 1071, 1071a-1071n Learning image data 1072, 1072a-1072n Learning non-image data 1080, 1080a-1080n Teacher Data P1 Subject P2 Operator P3 Supervisor R1 Laboratory R2 Operation Room S Medical Information Processing System

Claims (11)

A data acquisition unit that sequentially acquires data related to the progress of the procedure during the procedure, and a data acquisition unit.
A time calculation unit that calculates the time required for the procedure based on data related to the progress of a plurality of procedures acquired by the data acquisition unit at different timings during the execution of the procedure.
Medical information processing device equipped with. A determination unit for determining the success or failure of the procedure is further provided based on data related to the progress of the plurality of procedures acquired by the data acquisition unit at different timings during the execution of the procedure.
The medical information processing device according to claim 1. The data related to the progress of the plurality of procedures includes a medical image of the subject to which the procedure is performed.
The medical information processing device according to claim 1 or 2. The medical image is an X-ray image.
The medical information processing device according to claim 3. The data related to the progress of the plurality of procedures includes non-image information.
The medical information processing device according to any one of claims 1 to 3. The non-image information relates to procedure information relating to the procedure, subject information relating to a subject, operator information relating to an operator performing the procedure, measurement data measured during the procedure, or equipment used for the procedure. Includes at least one of the device information,
The medical information processing device according to claim 5. Further includes an output unit that outputs the time required for the procedure calculated by the time calculation unit.
The medical information processing device according to any one of claims 1 to 6. Further including an output unit that outputs the success or failure of the procedure determined by the determination unit.
The medical information processing device according to claim 2. A data acquisition unit that sequentially acquires data related to the progress of the procedure during the procedure, and a data acquisition unit.
A determination unit that determines the success or failure of the procedure based on data related to the progress of a plurality of procedures acquired by the data acquisition unit at different timings during the execution of the procedure.
Medical information processing device equipped with. During the procedure, a data acquisition unit that sequentially acquires X-ray images of the subject during the procedure, and a data acquisition unit.
A time calculation unit that calculates the time required for the procedure based on a plurality of X-ray images acquired by the data acquisition unit at different timings during the procedure.
X-ray diagnostic apparatus. A learning data generation step that generates multiple learning data by cutting out a set of time-series data related to the progress of the procedure at different time lengths.
A learning step for learning the correspondence between the plurality of learning data and the time required for the procedure, and
How to produce a trained model, including.

Images