JP6366898B2 - Medical device - Google Patents

Description

  Embodiments described herein relate generally to a medical apparatus and an X-ray diagnostic apparatus.

  2. Description of the Related Art Conventionally, a plurality of display devices that display various types of information are used in medical practice. For example, in an X-ray angiography system used in the diagnosis and treatment of a circulatory system such as the brain and heart, an examination room in which a procedure is performed on a subject, an operation room in which an X-ray diagnostic apparatus is operated, Each has a display. A plurality of observers in each of the examination room and the operation room observe the information displayed on each display. For example, in an examination room, an operator who performs a procedure observes a fluoroscopic image displayed on a display installed in the examination room. Further, for example, in the operation room, an operator who operates the X-ray diagnostic apparatus according to an operator's instruction observes various information displayed on a display installed in the operation room.

  In recent years, there has been known a multi-user medical information processing system in which a server device and a plurality of user terminals are connected via a communication network. As a medical information processing system, for example, a plurality of user terminals used in an examination room are connected to an operation terminal (for example, a workstation) arranged in an operation room, and the operation terminal is operated by the user terminal. Systems are known. However, in the above-described conventional technology, the operability of the operation using the user terminal may be deteriorated.

JP 2012-223500 A JP 2012-247905 A

  The problem to be solved by the present invention is to provide a medical apparatus and an X-ray diagnostic apparatus that can improve the operability of an operation using a user terminal.

The medical device of the embodiment, a touch panel, a display unit, a storage unit, and control means, Ru is used in X-ray diagnosis. The touch panel displays a GUI (Graphical User Interface) and accepts a touch operation by the user. The display unit is an apparatus different from the touch panel, and is installed in an examination room where imaging is performed by the X-ray diagnostic apparatus, and displays a medical image captured by the X-ray diagnostic apparatus . The storage unit stores a function for a medical image displayed on the display unit different from the touch panel in association with a predetermined operation by a plurality of fingers received by the touch panel . The control unit does not execute a function corresponding to an operation with one finger on the GUI displayed on the touch panel when the predetermined operation with the plurality of fingers is received by the touch panel, and is associated with the predetermined operation. The function is read from the storage means and executed.

FIG. 1A is a diagram for explaining an example of an X-ray angiography system according to the first embodiment. FIG. 1B is a diagram for explaining an example of the X-ray angiography system according to the first embodiment. FIG. 2 is a diagram illustrating an example of a configuration of an X-ray angiography system to which the user terminal according to the first embodiment is applied. FIG. 3 is a diagram illustrating an example of the configuration of the user terminal and the operation terminal according to the first embodiment. FIG. 4 is a diagram illustrating an example of a user terminal according to the first embodiment. FIG. 5 is a diagram illustrating an example of a screen to be controlled by the control unit according to the first embodiment. FIG. 6A is a diagram for explaining an example of processing by the conversion unit according to the first embodiment. FIG. 6B is a diagram for explaining an example of processing by the conversion unit according to the first embodiment. FIG. 6C is a diagram for explaining an example of processing by the conversion unit according to the first embodiment. FIG. 7 is a flowchart illustrating a processing procedure performed by the user terminal according to the first embodiment. FIG. 8 is a diagram illustrating an example of a configuration of a user terminal and an operation terminal according to the second embodiment. FIG. 9 is a diagram for explaining an example of a pointer operation based on processing by the calculation unit according to the second embodiment. FIG. 10 is a diagram for explaining an example of image display change based on processing by the calculation unit according to the second embodiment. FIG. 11 is a diagram for explaining an example of the alignment of the 3D road map based on the processing by the calculation unit according to the second embodiment. FIG. 12 is a flowchart illustrating a processing procedure performed by the user terminal according to the second embodiment.

  Hereinafter, embodiments of a medical device and an X-ray diagnostic apparatus will be described in detail with reference to the accompanying drawings. In the following, a case where the medical apparatus according to the present application is applied to an X-ray angio system will be described as an example.

(First embodiment)
First, an example of the X-ray angiography system according to the first embodiment will be described with reference to FIGS. 1A and 1B. 1A and 1B are diagrams for explaining an example of an X-ray angiography system according to the first embodiment. For example, in the X-ray angiography system, as shown in FIG. 1A, an apparatus main body including an arm and a top plate is arranged in an examination room R1 in which diagnosis and treatment of a circulatory system such as a brain and a heart are performed. In the operation room R2 shown in FIG. 1A, an operation terminal for performing an operation for controlling the apparatus main body is arranged.

  In the examination room R1 and the operation room R2, a plurality of examination room displays and a plurality of operation room displays are respectively installed. For example, the laboratory display is observed by a surgeon or nurse performing the procedure. The operation room display is observed by an operator who performs an operation for controlling the apparatus main body.

  Here, the medical device according to the first embodiment makes it possible to improve the operability of the operation using the user terminal. For example, in an X-ray angiography system as shown in FIG. 1A, in recent years, a technique of remotely operating a user terminal by connecting a user terminal to an operation terminal arranged in the operation room R2 has begun to be applied. For example, an application “Splashtop Touchpad” that accesses an operation terminal from a portable terminal with WiFi (registered trademark) and remotely operates a mouse and a keyboard of the operation terminal is being used.

  In this technology, for example, the mobile terminal and the operation terminal are connected by wireless communication, and the operation terminal receives the operation accepted by the mobile terminal via wireless, so that the mouse and keyboard of the operation terminal can be operated remotely. Is done. For example, an operator in the examination room R1 operates the pointer displayed on the examination room display by using the touch panel of the portable terminal as a touch pad, thereby allowing the GUI displayed on the examination room display to be displayed. Perform input operations.

  However, in such a system, the operability of the operation using the user terminal may deteriorate. For example, when an operator operates a portable terminal, a touch operation for operating a pointer displayed on the examination room display is performed while looking at the examination room display installed in the examination room R1. Here, the surgeon wears rubber gloves having a high coefficient of friction, and it is difficult to perform a smooth operation as in the case of bare hands, so the operability may be reduced. For example, when sliding in the direction of a finger, operability is reduced due to frictional resistance.

  As described above, in the conventional technique, the operability of the operation using the user terminal may be deteriorated. Therefore, the medical device according to the first embodiment can improve the operability of the operation using the user terminal by reducing the “upward drag operation”. Hereinafter, the details of one embodiment of the medical device according to the first embodiment will be described. In the following, a user terminal as a medical device will be described as an example. FIG. 2 is a diagram illustrating an example of a configuration of the X-ray angiography system 1 to which the user terminal according to the first embodiment is applied.

  As shown in FIG. 2, the X-ray angiography system 1 according to the first embodiment includes an apparatus main body 100 and an operation terminal 200. As shown in FIG. 2, the apparatus main body 100 includes a high voltage generator 11, an X-ray tube 12, an X-ray diaphragm device 13, a top plate 14, a C arm 15, an X-ray detector 16, and C The arm rotation / movement mechanism 17, the top plate movement mechanism 18, the C arm / top plate mechanism control unit 19, the aperture control unit 20, and the display unit 23 a are provided and arranged in the examination room R <b> 1. As illustrated in FIG. 2, the operation terminal 200 includes a system control unit 21, an input unit 22, a display unit 23 b, an image data generation unit 24, an image data storage unit 25, and an image processing unit 26. Arranged in the operation room R2. Then, the operation terminal 200 communicates with the user terminal 300 and executes processing according to the input operation accepted by the user terminal 300. Here, the user terminal 300 of this application improves the operativity of input operation by the structure demonstrated in detail below.

  Details of the user terminal 300 will be described later. In FIG. 2, one user terminal is arranged in the examination room R1, but the embodiment is not limited to this. For example, two or more user terminals are arranged. May be. In addition, the user terminal 300 may be disposed in the examination room R1 and the operation room R2, or may be disposed in the operation room. Although not shown, the X-ray angiography system 1 also includes an injector for injecting a contrast medium from a catheter inserted into the subject P.

  The high voltage generator 11 generates a high voltage under the control of the system control unit 21 and supplies the generated high voltage to the X-ray tube 12. The X-ray tube 12 generates X-rays using the high voltage supplied from the high voltage generator 11.

  The X-ray diaphragm 13 narrows the X-rays generated by the X-ray tube 12 under the control of the diaphragm controller 20 so that the region of interest of the subject P is selectively irradiated. For example, the X-ray diaphragm device 13 has four slidable diaphragm blades. The X-ray diaphragm 13 narrows down the X-rays generated by the X-ray tube 12 and irradiates the subject P by sliding these diaphragm blades under the control of the diaphragm controller 20. The X-ray tube 12 and the X-ray diaphragm device 13 are collectively referred to as an X-ray tube device. The top 14 is a bed on which the subject P is placed, and is placed on a bed (not shown). Note that the subject P is not included in the apparatus main body 100.

  The X-ray detector 16 detects X-rays that have passed through the subject P. For example, the X-ray detector 16 has detection elements arranged in a matrix. Each detection element converts the X-rays that have passed through the subject P into electrical signals and accumulates them, and transmits the accumulated electrical signals to the image data generation unit 24.

  The C arm 15 holds the X-ray tube 12, the X-ray diaphragm device 13, and the X-ray detector 16. The X-ray tube 12 and the X-ray diaphragm 13 and the X-ray detector 16 are arranged so as to face each other with the subject P sandwiched by the C arm 15.

  The C-arm rotating / moving mechanism 17 is a mechanism for rotating and moving the C-arm 15, and the top board moving mechanism 18 is a mechanism for moving the top board 14. The C arm / top plate mechanism control unit 19 controls the C arm rotation / movement mechanism 17 and the top plate movement mechanism 18 under the control of the system control unit 21, thereby rotating and moving the C arm 15 and the top plate 14. Adjust the movement. The aperture control unit 20 controls the irradiation range of the X-rays irradiated to the subject P by adjusting the aperture of the aperture blades of the X-ray aperture device 13 under the control of the system control unit 21. .

  The image data generation unit 24 generates image data using the electrical signal converted from the X-rays by the X-ray detector 16, and stores the generated image data in the image data storage unit 25. For example, the image data generation unit 24 performs current / voltage conversion, A (Analog) / D (Digital) conversion, and parallel / serial conversion on the electrical signal received from the X-ray detector 16 to generate image data. To do.

  Then, the image data generation unit 24 generates an X-ray image from the generated image data, and stores the generated X-ray image in the image data storage unit 25. The image data storage unit 25 stores the image data generated by the image data generation unit 24.

  The image processing unit 26 performs various image processing on the image data stored in the image data storage unit 25. For example, the image processing unit 26 generates a moving image by processing a plurality of X-ray images along the time series stored in the image data storage unit 25.

  The input unit 22 receives various instructions from an operator who operates the X-ray angiography system 1. For example, the input unit 22 includes a mouse, a keyboard, a button, a trackball, a joystick, and the like. The input unit 22 transfers the instruction received from the operator to the system control unit 21.

  The display unit 23a and the display unit 23b display a GUI (Graphical User Interface) for receiving an instruction from the operator, image data stored in the image data storage unit 25, and the like. For example, the display unit 23a is an examination room display, and the display unit 23b is an operation room display. The display unit 23a and the display unit 23b may each include a plurality of displays. Here, the same contents are displayed on the display unit 23a and the display unit 23b, respectively. For example, the display unit 23a and the display unit 23b display a real-time perspective image, a three-dimensional road map (3DRM), and the like. Note that 3DRM is an image in which a real-time perspective image is superimposed on a projection image generated from volume data collected by the apparatus main body 100.

  The system control unit 21 controls the operation of the entire X-ray angiography system 1. For example, the system control unit 21 controls the high voltage generator 11 in accordance with an operator instruction transferred from the input unit 22 and adjusts the voltage supplied to the X-ray tube 12, thereby irradiating the subject P. Controls X-ray dose and ON / OFF. Further, for example, the system control unit 21 controls the C arm / top plate mechanism control unit 19 in accordance with an instruction from the operator, and adjusts the rotation and movement of the C arm 15 and the movement of the top plate 14. Further, for example, the system control unit 21 controls the aperture control unit 20 in accordance with an instruction from the operator, and adjusts the aperture of the aperture blades of the X-ray aperture device 13 to irradiate the subject P. The X-ray irradiation range is controlled.

  Further, the system control unit 21 controls image data generation processing by the image data generation unit 24, image processing by the image processing unit 26, analysis processing, and the like according to an instruction from the operator. In addition, the system control unit 21 performs control so that a GUI for receiving an instruction from the operator, an image stored in the image data storage unit 25, and the like are displayed on the displays of the display unit 23a and the display unit 23b. Further, the system control unit 21 controls the injection of the contrast agent by transmitting signals of the start and end of the contrast agent injection to the injector.

  Here, in the operation terminal 200 of the X-ray angiography system 1 according to the first embodiment, each control described above is performed in accordance with an instruction received by the user terminal 300. That is, the system control unit 21 according to the first embodiment executes various controls in accordance with instructions accepted by the user terminal 300 in addition to operator instructions accepted by the input unit 22. Here, the user terminal 300 according to the first embodiment is configured to improve the operability of the operation by the user terminal 300.

  FIG. 3 is a diagram illustrating an example of the configuration of the user terminal 300 and the operation terminal 200 according to the first embodiment. As illustrated in FIG. 3, the operation terminal 200 includes a communication unit 210 and a storage unit 220 in addition to the input unit 22, the display unit 23 b, and the system control unit 21. In FIG. 3, the image data generation unit 24, the image data storage unit 25, and the image processing unit 26 are not shown, but actually, as shown in FIG. 24, an image data storage unit 25 and an image processing unit 26 are provided.

  As described above, the input unit 22 receives various input operations. The display unit 23b displays various information as described above. The communication unit 210 is a NIC (Network Interface Card) or the like, and communicates with the user terminal 300 via a network. Specifically, the communication unit 210 performs various communications with the communication unit 330 of the user terminal 300.

  The storage unit 220 is, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk, and stores information used by the system control unit 21. To do. Here, the storage unit 220 may be a case where the image data storage unit 25 is used.

  The system control unit 21 is, for example, an electronic circuit such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). Thus, overall control of the X-ray angiography system 1 is performed.

  As illustrated in FIG. 3, the user terminal 300 includes an input unit 310, a display unit 320, a communication unit 330, a storage unit 340, and a control unit 350. The user terminal 300 is, for example, a stationary information processing device such as a PC (Personal Computer), or a portable information processing device such as a tablet PC or PGA. Hereinafter, an example in which the user terminal is a tablet PC will be described.

  The input unit 310 is a touch panel or the like having a flat outer shape, and receives an input operation from an operator such as an operator or a nurse who operates the user terminal. For example, the input unit 310 receives various instructions by receiving a touch operation, a flick operation, a swipe operation, and the like. The display unit 320 displays display information including contents set by the control unit 350 described later. For example, the display unit 320 is a display device such as a liquid crystal panel, is formed in combination with the input unit 310, and displays a GUI for receiving an input operation by the input unit 310.

  FIG. 4 is a diagram illustrating an example of a user terminal according to the first embodiment. FIG. 4 shows an overview of the user terminal 300. As shown in FIG. 4, for example, the user terminal 300 is a tablet PC on which various operations are performed by a touch panel, and each is operated by an operator, an assistant, a nurse, or the like.

  For example, as shown in FIG. 4A, the touch panel of the user terminal 300 becomes a touch pad, and an operator in the examination room R1 operates the pointer displayed on the examination room display using the touch pad. As a result, an input operation is performed on the GUI displayed on the examination room display. Further, as shown in FIG. 4A, the user terminal 300 can also display “Image” or the like displayed on the examination room display or the operation room display.

  Further, the user terminal 300 includes an in-camera on the same surface as the touch panel as shown in FIG. 4A, and a rear camera on the opposite surface as shown in FIG. 4B. And in the user terminal 300, the image | video etc. which were image | photographed with each camera can be displayed on a touch panel.

  The communication unit 330 is a NIC or the like, and performs communication with the operation terminal 200 via a network. Specifically, the communication unit 330 performs various types of communication with the communication unit 210 of the operation terminal 200.

  The storage unit 340 is, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk, and stores information used by the control unit 350.

  The control unit 350 is, for example, an electronic circuit such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit), an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), and the user terminal 300 Perform overall control. As shown in FIG. 3, the control unit 350 includes a conversion unit 351 and a display control unit 352, and improves the operability of operations on the screen as shown in FIG.

  FIG. 5 is a diagram illustrating an example of a screen to be controlled by the control unit 350 according to the first embodiment. For example, as shown in FIG. 5, the control by the control unit 350 targets a plurality of image display areas displayed on the display unit 23 b of the operation terminal 200 and a screen including the operation area. For example, the control unit 350 performs control so that a vertical drag operation is converted into a horizontal drag operation in the operation region shown in the right region of FIG. Hereinafter, an example of processing by each unit will be described.

  Returning to FIG. 3, the conversion unit 351 converts a horizontal operation into a vertical operation in a predetermined input operation received by the touch panel. Specifically, the conversion unit 351 converts a horizontal drag operation into a vertical drag operation among the horizontal operations. FIG. 6A is a diagram for explaining an example of processing by the conversion unit 351 according to the first embodiment. For example, as illustrated in FIG. 6A, the conversion unit 351 converts a scroll bar that slides in the vertical direction into a scroll bar that slides in the horizontal direction.

  For example, the conversion unit 351 converts the upward movement of the scroll bar to be slid in the vertical direction into a right slide in the horizontal direction, and converts the downward movement into a left slide in the horizontal direction. That is, when the operator performs a drag operation in the right direction on the touch panel, the conversion unit 351 converts the operation information so that an upward slide similar to the distance of the executed drag operation is performed. . Similarly, when the operator performs a drag operation in the left direction on the touch panel, the conversion unit 351 converts the operation information so that a downward slide similar to the distance of the performed drag operation is executed. To do.

  The operation described above may be performed while an operator in the laboratory is performed while looking at the scroll bar that is slid in the vertical direction displayed on the laboratory display, or may be the vertical operation displayed on the touch panel of the user terminal 300. It may be executed while looking at a scroll bar that slides in the direction. That is, the display control unit 352 described later may display a scroll bar on the display unit 320. In addition, a horizontal scroll bar corresponding to the scroll bar to be slid in the vertical direction may be displayed on the display unit 320. In such a case, the display control unit 352 described later causes the display unit 320 to display a horizontal scroll bar. Then, the operator operates the displayed horizontal scroll bar in the horizontal direction to slide the display content in the vertical direction.

  Also, the conversion unit 351 converts the combo box so that the selection items included in the combo box displayed on the touch panel are arranged in the horizontal direction. FIG. 6B is a diagram for explaining an example of processing performed by the conversion unit 351 according to the first embodiment. For example, as illustrated in FIG. 6B, the conversion unit 351 converts a vertical combo box into a horizontal combo box. From this, for example, when the selection item “Cardiac” is selected in the combo box shown in FIG. 6B, the selection can be made by a drag operation in the right direction.

  In addition, the conversion unit 351 converts the display method of the entire display content so that the entire display content displayed on the touch panel is displayed in a tilted state of 90 degrees. FIG. 6C is a diagram for describing an example of processing by the conversion unit 351 according to the first embodiment. For example, as illustrated in FIG. 6C, the conversion unit 351 converts the display method of the entire display content including the vertical scroll bar by rotating 90 degrees to the right, and converts the display method into a horizontal combo box. To do. Thereby, for example, the vertical scroll bar and the selection area shown in FIG. 6C can be converted into a horizontal movement, and can be easily selected by a horizontal drag operation.

  Returning to FIG. 3, the display control unit 352 executes control for executing processing corresponding to a predetermined input operation received by the touch panel. Specifically, the display control unit 352 performs control so that the operation in the vertical direction converted by the conversion unit 351 is executed. In other words, the display control unit 352 executes processing corresponding to the vertical drag operation converted by the conversion unit 351.

  For example, the display control unit 352 moves and displays the corresponding table in the vertical direction in accordance with the slide of the horizontal scroll bar shown on the right side of FIG. 6A. Further, for example, the display control unit 352 causes the display unit 320 to display a horizontal scroll bar shown on the right side of FIG. 6A. In addition, the display control unit 352 displays a combo box in which the selection items illustrated in FIG. 6B are arranged in the horizontal direction. Further, as shown in FIG. 6C, the display control unit 352 controls to display the entire display content obtained by rotating the entire display content to the right by 90 degrees.

  In the above-described example, the case where the vertical drag operation is converted into the horizontal drag operation has been described. However, the embodiment is not limited to this. For example, the vertical flick operation, the pinch-in operation, and the pinch-out operation are converted to be executed by the horizontal flick operation, the pinch-in operation, and the pinch-out operation. It may be the case.

  Next, processing of the user terminal 300 according to the first embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating a processing procedure performed by the user terminal 300 according to the first embodiment. As shown in FIG. 7, when the input mode for converting the vertical operation into the horizontal operation is changed (Yes in step S101), the conversion unit 351 displays the horizontal movement performed on the touch panel on the screen. Is converted into a vertical motion (step S102). Note that the user terminal 300 accepts a normal input operation until the input mode is changed (No in step S101).

  When display switching is performed to change the display content in the horizontal direction (Yes at Step S103), the display control unit 352 changes the scroll bar, combo box, or the entire screen to display in the horizontal direction (Step S103). S104). In the above-described example, the case where the input mode and the display switching are respectively received has been described. However, the embodiment is not limited to this, and for example, the mode is changed by a single mode change operation. Also good. Note that the user terminal 300 displays the display content in the normal display direction (vertical direction) until display switching is executed (No in step S103).

  As described above, according to the first embodiment, the input unit 310 accepts a predetermined input operation for operating a communication destination operation terminal. The control unit 350 performs control for executing processing corresponding to a predetermined input operation received by the input unit 310. Therefore, the user terminal 300 according to the first embodiment can execute processing according to the input operation, and can improve the operability of the operation by the user terminal.

  Further, according to the first embodiment, the conversion unit 351 converts a horizontal operation into a vertical operation in a predetermined input operation received by the input unit 310 that is a touch panel. The control unit 350 performs control such that the vertical operation converted by the conversion unit 351 is executed. Therefore, the user terminal 300 according to the first embodiment can execute the vertical processing by the horizontal operation, and can further improve the operability of the operation by the user terminal. For example, even in the case of operating the covered touch panel with a finger wearing rubber gloves having a high friction coefficient, it is possible to easily execute the vertical processing.

  Further, according to the first embodiment, the conversion unit 351 converts a horizontal drag operation into a vertical drag operation among the horizontal operations. The control unit 350 performs control so that processing corresponding to the vertical drag operation converted by the conversion unit 351 is executed. Therefore, the user terminal 300 according to the first embodiment can execute the vertical processing by the horizontal drag operation, and can further improve the operability of the operation by the user terminal.

  According to the first embodiment, the conversion unit 351 converts the combo box so that the selection items included in the combo box displayed on the touch panel are arranged in the horizontal direction. Therefore, the user terminal 300 according to the first embodiment can execute the combo box selection operation by a horizontal drag operation, and can further improve the operability of the operation by the user terminal.

  Further, according to the first embodiment, the conversion unit 351 converts the display method of the entire display content so that the entire display content displayed on the touch panel is displayed in a tilted state of 90 degrees. Accordingly, the user terminal 300 according to the first embodiment can execute all the vertical operations displayed on the touch panel by the horizontal operation, and can further improve the operability of the operation by the user terminal. To.

(Second Embodiment)
In the above-described first embodiment, the case where the vertical processing is executed by the horizontal operation has been described. In the second embodiment, a case will be described in which a pointer operation, an image display change, and a 3D road map alignment are executed without performing an operation on the touch panel. FIG. 8 is a diagram illustrating an example of a configuration of a user terminal and an operation terminal according to the second embodiment. As illustrated in FIG. 8, the user terminal 300 according to the second embodiment is different from the user terminal 300 according to the first embodiment in that a camera 360 and a calculation unit 353 are newly included. Hereinafter, these will be mainly described.

  The camera 360 accepts a predetermined input operation for operating information processing of a communication destination. For example, the camera 360 is an in-camera illustrated in FIG. 4A or a rear camera illustrated in FIG. 4B, and accepts an image capturing operation. Based on the image photographed by the camera 360, the calculation unit 353, which will be described in detail below, executes pointer operation, image display change, and 3D roadmap alignment processing. In the following, pointer operations, image display changes, and 3D roadmap alignment will be described in order.

  First, the pointer operation will be described. In such a case, the calculation unit 353 calculates the amount of movement of the own device before and after shooting based on the image shot by the camera 360. FIG. 9 is a diagram for explaining an example of a pointer operation based on processing by the calculation unit 353 according to the second embodiment. For example, as illustrated in FIG. 9A, the calculation unit 353 calculates the movement amount of the user terminal 300 in the real space based on an image captured by the camera 360.

  Here, the calculation unit 353 calculates the amount of movement of the user terminal 300 with reference to an object included in an image captured by the camera 360 (for example, a rear camera). For example, the calculation unit 353 calculates the movement amount of the user terminal 300 with reference to a focused object among the top plate, the arm, and the examination room display in the examination room R1. That is, the calculation unit 353 calculates the movement amount of the coordinates of the reference object in the coordinates of the real space photographed by the camera 360. The amount of movement here corresponds to the direction and distance of movement. Then, the calculation unit 353 calculates a pointer movement amount according to the calculated movement amount, and notifies the display control unit 352 of the calculated movement amount. The movement amount of the pointer may be the case where the movement amount of the coordinates of the reference object is used as it is, or the case where a predetermined value is added to the movement amount of the coordinates of the reference object.

  Here, in the operation of the pointer, the calculation unit 353 eliminates the influence of the tilt of the user terminal 300 using the gyro sensor. That is, when the user terminal 300 moves in parallel with the reference object, an appropriate amount of movement on the two-dimensional plane can be calculated, but when the user terminal 300 moves with an inclination, the camera 360 The exact amount of movement of the reference object to be photographed cannot be calculated. Therefore, the calculation unit 353 acquires information on the inclination of the user terminal 300 at the time of photographing with the gyro sensor of the inclination of the user terminal 300, corrects the inclination of the user terminal 300 using the acquired inclination, and then calculates the movement amount. calculate.

  The display control unit 352 according to the second embodiment moves the pointer according to the movement amount notified from the calculation unit 353. For example, as shown in FIG. 9B, the display control unit 352 moves the pointer 40 displayed on the display unit 23b of the operation terminal 200 according to the notified movement amount. That is, in the above-described operation of the pointer, the pointer moves in the same manner as the movement of the reference object in the image taken by the camera 360. For example, in the display unit 23a arranged in the examination room R1, the user terminal 300 is observing the display unit 23a while the operator of the user terminal 300 is observing the same screen as the display unit 23b of the operation terminal 200. The pointer 40 displayed on the display unit 23b is moved by moving the pointer 40 displayed on the display unit 23a.

  The operation of the pointer by moving the user terminal 300 is not limited to the above-described example. For example, an acceleration sensor may be used, or a gyro sensor may be used. That is, the pointer may be operated using the acceleration or inclination of the user terminal 300.

  Next, image display change will be described. In such a case, the calculation unit 353 calculates the movement amount in the front-rear direction and the inclination amount of the user terminal 300 using information acquired by the camera 360, the gyro sensor, the acceleration sensor, and the like. Then, the display control unit 352 changes the generation position of the medical image generated from the three-dimensional medical image data and displayed on the display unit 320 by the amount of movement calculated by the calculation unit 353, and the three-dimensional medical image. Change to a medical image generated from image data. FIG. 10 is a diagram for explaining an example of image display change based on processing by the calculation unit 353 according to the second embodiment. In FIG. 10, predetermined three-dimensional MPR images (axial plane, coronal plane, sagittal plane) generated from three-dimensional image data collected by the X-ray angiography system 1 and predetermined three-dimensional image data. A projected image (lower right) projected from the viewpoint position is shown.

  As illustrated in FIG. 10A, the calculation unit 353 calculates the movement amount including the inclination of the user terminal 300. Here, the calculation unit 353 calculates not only the movement amount in the two-dimensional plane of the XY plane but also the movement amount in the Z-axis direction. Note that the amount of movement in the Z-axis direction is calculated based on the degree of enlargement and reduction of the image. And the control part 350 produces | generates the MPR image of 3 orthogonal cross sections which changed the production | generation position with the calculated movement amount. That is, the control unit 350 uses the generation position of the MPR image of the three orthogonal cross sections of the axial plane, coronal plane, and sagittal plane shown in FIG. 10B as the position of the user terminal 300 before the movement, and generates the three-dimensional image data. Control is performed so that an MPR image having three orthogonal cross sections is generated at a position shifted from the position by the movement amount calculated by the calculation unit 353. Then, the display control unit 352 updates each MPR image shown in FIG. 10B to an MPR image having three orthogonal cross sections generated under the control of the control unit 350.

  In addition, the control unit 350 performs control so as to generate a projection image in which the line-of-sight position is changed by the amount of inclination calculated by the calculation unit 353. That is, the control unit 350 moves the viewpoint position when the projection image shown in the lower right region of the display unit 23b in FIG. 10B is generated by the calculated inclination, and then moves the position after the movement. Control is performed to generate a projection image in which the position is projected as the viewpoint position. Then, the display control unit 352 updates the projection image shown in FIG. 10B to a projection image generated under the control of the control unit 350. Here, 360 ° observation can be performed by using the projection image inversion function. For example, in the display unit 23a arranged in the examination room R1, the user terminal 300 is observing the display unit 23a while the operator of the user terminal 300 is observing the same screen as the display unit 23b of the operation terminal 200. Is moved to change the position of the projected image displayed on the display unit 23a, thereby changing the projected image displayed on the display unit 23a (and the display unit 23b) to a desired image.

  Next, alignment of the 3D road map will be described. The alignment of the 3D road map means the alignment when superimposing the 3D image captured in advance with the fluoroscopic image. In such a case, the camera 360 captures the three-dimensional image displayed on the display unit 320 of the user terminal 300 and the fluoroscopic image displayed on the display unit 23b of the operation terminal 200 or the subject on which the fluoroscopic image is captured. Are received for alignment operation within the display unit 320 of the user terminal 300. Specifically, the camera 360 receives an operation for photographing a fluoroscopic image or a subject displayed on the display unit 23a or the display unit 23b. Hereinafter, a case where the fluoroscopic image displayed on the display unit 23b is captured and aligned will be described.

  FIG. 11 is a diagram for explaining an example of alignment of a 3D road map based on processing by the calculation unit 353 according to the second embodiment. As shown in FIG. 11A, first, the display unit 23b displays a still image of a fluoroscopic image. For example, the display unit 23b displays LIH (Last Image Hold). Here, the system control unit 21 transmits to the user terminal 300 information on the LIH displayed on the display unit 23b (information such as an angle taken with respect to the subject). Furthermore, the system control unit 21 transmits a 3D image captured in advance to the user terminal 300. At this time, the system control unit 21 transmits simultaneously the information of the transmitted three-dimensional image (information such as from which direction the image is generated with respect to the three-dimensional image data).

  Then, the display control unit 352 displays a three-dimensional image on the display unit 320 as illustrated in FIG. Thereafter, the operator of the user terminal 300 captures the LIH displayed on the display unit 23b by the camera 360. Here, the display control unit 352 fixes the three-dimensional image in the virtual space in the display unit 320, and translates or rotates the user terminal 300 with respect to the fixed three-dimensional image. Then, control is performed so that the photographed LIH is matched with the three-dimensional image. That is, as shown in FIG. 11B, the operator translates or rotates the user terminal 300 with respect to the three-dimensional image fixed in the virtual space in the display unit 320. Perform alignment. Here, the photographed LIH moves in conjunction with the movement of the user terminal 300.

  The calculation unit 353 calculates a correction value for aligning the three-dimensional image and the LIH in the above-described alignment. Specifically, the calculation unit 353 first calculates the positional relationship between the position of the display unit 23b when the LIH is captured and the user terminal 300. Here, for example, the position of the display unit 23b in the imaging space is calculated by attaching an AR marker to the display unit 23b or performing pattern matching.

  At this time, since the LIH information displayed on the display unit 23b is received, the calculation unit 353 can calculate the position and angle of the LIH in the imaging space (the position and angle of the LIH with respect to the user terminal 300). it can. Furthermore, since the calculation unit 353 receives the information of the three-dimensional image displayed on the display unit 320, the calculation unit 353 refers to the state of the three-dimensional image displayed on the user terminal 300, so that LIH The current positional relationship with the three-dimensional image (when the fluoroscopic image is taken) can be calculated. Thereby, the following alignment can target the LIH and the three-dimensional image that are generated first.

  Then, the calculation unit 353 determines how much the user terminal 300 (the associated fluoroscopic image) is moved when the fluoroscopic image and the three-dimensional image are aligned by an operation (parallel movement or plane rotation) of the operator. To calculate a correction value for aligning the fluoroscopic image and the three-dimensional image. Note that the movement of the user terminal 300 is calculated by the camera 360, the gyro sensor, or the like as described above.

  The calculation unit 353 transmits the calculated correction value to the operation terminal 200, thereby enabling alignment on the operation terminal 200. Further, the user terminal 300 can execute alignment with the fluoroscopic image collected in real time by using the calculated correction value. Note that the information on the fluoroscopic image is information on a mechanical system at the time of photographing (such as a fluoroscopic angle). That is, in the case of real time, information on the current mechanical system is transmitted, and in the case of a past fluoroscopic image, information on the mechanical system at that time is transmitted.

  In the above-described embodiment, the case where the fluoroscopic image displayed on the display unit 23b is captured has been described. However, the embodiment is not limited to this, and for example, the subject may be captured. . Even in such a case, similarly to the above, the positional relationship between the user terminal 300 and the subject is calculated, and alignment based on the captured image is executed based on the calculated positional relationship.

  Next, processing by the user terminal 300 according to the second embodiment will be described with reference to FIG. FIG. 12 is a flowchart illustrating a processing procedure performed by the user terminal 300 according to the second embodiment.

  As shown in FIG. 12, in the pointer operation mode (Yes at Step S201), the calculation unit 353 calculates the movement amount of the own apparatus from the video imaged by the camera 360 (Step S202), and the calculated movement. The pointer is moved according to the amount (step S203).

  When it is not the pointer operation mode (No at Step S201), the calculation unit 353 determines whether or not it is the alignment mode (Step S204). Here, in the case of the alignment mode (Yes at Step S204), the display unit 320 displays a 3D image (Step S205). Then, the calculation unit 353 determines whether or not the display on which the fluoroscopic image is displayed has been photographed (Step S206).

  Here, when the display on which the fluoroscopic image is displayed is photographed (Yes at Step S206), the calculation unit 353 acquires information on the mechanical system at the time of photographing the fluoroscopic image (Step S207), and calculates the deviation. Then, alignment is executed (step S208). Note that the user terminal 300 is in a standby state until a display on which a fluoroscopic image is displayed is photographed (No at Step S206).

  On the other hand, when it is not the alignment mode in step S204 (step S204), the calculation unit 353 calculates the movement amount of the own apparatus from the video imaged by the camera 360 (step S209), and the calculated movement amount is obtained. The display is changed accordingly (step S210).

  As described above, according to the second embodiment, the calculation unit 353 calculates the movement amount of the user terminal 300 before and after shooting based on the image shot by the camera 360. The control unit 350 controls to execute processing corresponding to the movement amount calculated by the calculation unit 353. Therefore, the user terminal 300 according to the second embodiment can execute the process only by moving the user terminal 300 without using the touch panel, and can further improve the operability.

  Further, according to the second embodiment, the control unit 350 moves the pointer displayed on the display unit 320 by the movement amount calculated by the calculation unit 353. Therefore, the user terminal 300 according to the second embodiment can easily move the pointer.

  Further, according to the second embodiment, the camera 360 captures a three-dimensional image displayed on the display unit 320 of the user terminal 300 and a fluoroscopic image or a fluoroscopic image displayed on the display unit 23b of the other device. An alignment operation for aligning the target object within the display unit 320 of the user terminal 300 is accepted. The calculation unit 353 calculates a deviation between the three-dimensional image and the fluoroscopic image at the time of the alignment operation received by the camera 360. The display control unit 352 corrects the deviation calculated by the calculation unit 353 and displays a superimposed image in which the three-dimensional image and the fluoroscopic image are superimposed. Therefore, the user terminal 300 according to the second embodiment makes it possible to perform alignment of the 3D road map without performing a touch operation.

  The control unit 350 generates the medical image generated from the three-dimensional medical image data and changes the generation position with the movement amount calculated by the calculation unit 353 from the three-dimensional medical image data. Change to the generated medical image. Therefore, the user terminal 300 according to the second embodiment makes it possible to easily change the display of the three-dimensional image.

(Third embodiment)
Although the first and second embodiments have been described so far, the present invention may be implemented in various different forms other than the first and second embodiments described above.

  In the second embodiment described above, the case where the display of the three-dimensional image is changed based on the movement amount of the user terminal 300 has been described. However, the embodiment is not limited to this. For example, the display of a three-dimensional image may be changed based on an image captured by the camera 360.

  For example, the camera 360 receives an operation for photographing the arm. Then, the calculation unit calculates the angle information of the arm based on the image of the arm taken by the camera 360. The display control unit 352 controls to generate a medical image from the three-dimensional medical image data at an angle corresponding to the arm angle information calculated by the calculation unit 353. Accordingly, the user terminal 300 according to the third embodiment can easily display a three-dimensional image at an angle that is currently viewed through.

  In the first and second embodiments described above, in order to improve the operability of the operation by the user terminal 300, the amount of movement of the user terminal 300 or the image taken by the camera 360 is used. explained. However, the embodiment is not limited to this. For example, the user terminal 300 can be operated without looking at the display surface.

  In such a case, the storage unit 340 stores a predetermined function in association with each predetermined input operation received by the touch panel. Then, when a predetermined input operation is received by the touch panel, the control unit 350 controls to execute a predetermined function corresponding to the predetermined input operation stored in the storage unit 340.

By way of example, the storage unit 340, and pinch, pinch-out, in association like swipe by a plurality of fingers, the image storage and the switching of the image, and stores and the expansion of the region of interest. Then, the touch panel, or pinch, the pinch-out, when such swipe is performed by a plurality of fingers, the control unit 350 controls to execute the corresponding processing.

Here, for example, when or pinch, the pinch-out, such as swipe by a plurality of fingers is performed, the control unit 350, GUI operation being displayed at present is ignored. For example, if a swipe with a plurality of fingers is executed when any button is displayed, the corresponding function is not executed even if one finger touches the button. In addition, a predetermined operation to which each function is assigned is an operation that is not used in a normal operation. Thereby, the operator can perform a predetermined | prescribed process in the state which observed the image displayed on the laboratory display, without looking at a touch panel.

  In the above-described embodiment, the case where there is one user terminal 300 has been described. However, the embodiment is not limited to this, and for example, two or more units may be used.

  In addition, the configurations of the user terminal 300 and the operation terminal 200 according to the above-described embodiment are merely examples, and the integration and separation of each unit can be performed as appropriate. For example, the conversion unit 351 and the calculation unit 353 can be integrated.

  As described above, according to the first, second, and third embodiments, the medical apparatus and the X-ray diagnostic apparatus of the present embodiment can improve the operability of the operation using the user terminal. .

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 X-ray angiography system 200 Operation terminal 300 User terminal 310 Input part 320 Display part 351 Conversion part 352 Display control part 353 Calculation part 360 Camera

Claims (2)

A touch panel that displays a GUI (Graphical User Interface) and receives a touch operation by a user;
A display unit that is a device different from the touch panel, is installed in an examination room where imaging is performed by an X-ray diagnostic apparatus, and displays a medical image captured by the X-ray diagnostic apparatus ;
Storage means for storing a function associated with a medical image displayed on the display unit different from the touch panel in association with a predetermined operation by a plurality of fingers received by the touch panel;
When accepting a predetermined operation with the plurality of fingers by the touch panel, the function corresponding to the operation with one finger on the GUI displayed on the touch panel is not executed, and the function associated with the predetermined operation is Control means for reading from the storage means and executing;
A medical device used for X-ray diagnosis .   The medical apparatus according to claim 1, wherein the function for the medical image displayed on the display unit is any one of image storage, image switching, and region of interest expansion.

Images