WO2017034020A1 - Medical image processing device and medical image processing program - Google Patents

Abstract

This image processing device (301) is equipped with: a display (361); a control unit (350) which is connected to the display; a speech input device (370); and a motion sensor (380). The control unit (350) has: a: an image display part for displaying a three-dimensional medical image on the display; b: a mode selection part for switching, upon recognizing speech input through a speech recognition device (170), the mode of display related to the three-dimensional medical image in response to the speech; and c: a display processing part for changing, upon recognizing via the motion sensor a motion input by an operator, the display of the three-dimensional medical image in response to the motion.

Description

Medical image processing apparatus and medical image processing program

The present invention relates to a medical image processing apparatus and a medical image processing program. The present invention particularly relates to a medical image processing apparatus and the like that can display a three-dimensional medical image obtained by imaging a patient and perform various display processes with good workability and while maintaining the cleanliness of an operator. .

Currently, CT (Computed Tomography) devices, MRI (Magnetic Resonance Imaging) devices, PET (Positron Emission Tomography) devices, ultrasound diagnostic devices, angiographic imaging devices, etc. are known as medical diagnostic imaging devices. Yes.

In recent years, for example, a simulation may be performed before performing an operation on an organ such as a liver in which blood vessels are intertwined in a complicated manner. The simulation is performed, for example, by performing a contrast CT examination, preparing a fluoroscopic image of a site to be treated, and confirming it on a display. Such simulations are useful for studying treatment plans. For example, Patent Document 1 discloses a technique for simulating a surgical operation by displaying an organ such as a liver on a tablet terminal.

JP 2014-54358 A

The technique of Patent Document 1 uses a tablet terminal, so it is useful in that it can be carried anywhere and can perform preoperative simulation. In addition, since it can be brought into the operating room, it is also useful in that some confirmation can be performed during the operation.

However, if the operator who uses the terminal is not easy to use, the functions provided may not be used. Moreover, it is desirable that the operation can be performed while keeping the operator clean. This is because a medical image processing apparatus may be operated by a doctor or the like during surgery.

The present invention has been made paying attention to such problems. Its purpose is to display a three-dimensional medical image obtained by imaging a patient, a medical image processing apparatus and an image processing program capable of performing various display processes with good workability and while keeping the operator clean. Is to provide.

A medical image processing apparatus according to an embodiment of the present invention for solving the above-described problems is as follows:
Display,
A control unit (processor) connected to the display;
An audio input device;
A motion sensor,
A medical image processing apparatus comprising:
The control unit (processor)
a: an image display unit for displaying a three-dimensional medical image on the display;
b: a mode selection unit for recognizing a voice input using the voice recognition device and switching a mode related to display of the three-dimensional medical image in accordance with the voice;
c: a display processing unit for recognizing an operator's motion input via the motion sensor and changing the display of the three-dimensional medical image accordingly;
A medical image processing apparatus.

In other words, in the apparatus according to an embodiment of the present invention, the control unit (processor)
-Display a 3D medical image on the display;
Recognizing speech input using the speech recognition device and switching modes relating to the display of the 3D medical image accordingly;
-Configured to recognize an operator's motion input via the motion sensor and change the display of the 3D medical image accordingly.

(Explanation of terms)
“Anatomical structure” refers to an object (for example, an organ, bone, blood vessel, etc.) that can be recognized in a subject, and includes fat, lesions such as a tumor, and the like.
“Terminal” refers to an information processing apparatus that is connected to a network or used in a standalone manner to perform data processing. An arbitrary peripheral device may be connected to the information processing apparatus. Basically, it is also preferable that various functions are provided in one device such as a tablet terminal or a laptop computer. However, in some cases, a part of the functions may be arbitrarily set depending on, for example, a load. It can also be configured to be functionally or physically distributed in units.
“Connected” means that, in addition to the case where two elements are directly connected, one element and another element are indirectly connected via some intermediate element without departing from the spirit of the present invention. This includes cases where It also includes both wired and wireless connections.

In the present specification, the component expressed as “function” + “part” corresponds to a functional block that performs a predetermined function. A functional block does not necessarily indicate a division between hardware circuits. Thus, for example, one or more functional blocks can be implemented with a single hardware, but can also be implemented with multiple hardware.

According to the present invention, a medical image processing apparatus and the like that can display a three-dimensional medical image obtained by imaging a patient and perform various display processes with good workability and while maintaining the cleanliness of an operator are provided. can do.

It is a figure which shows the medical image processing apparatus of one Embodiment. It is a figure which shows the example of the block diagram of the image processing apparatus of FIG. It is a figure which shows an example of the component of a hospital system. 3 is a flowchart of an operation example of the image processing apparatus in FIG. 1. It is a figure which shows the example of the three-dimensional medical image which displayed the liver and the blood vessel of the circumference | surroundings. It is an example of the graphical image at the time of recognizing the position of an operator's hand. It is a figure which shows an example of a three-dimensional medical image. It is a figure for demonstrating a mode that 2 points | pieces are designated with respect to the arbitrary objects on a three-dimensional medical image. It is a figure for demonstrating a mode that the position of the designated point is finely adjusted. It is a figure which shows the example which divides | segments a liver. It is a figure for demonstrating a mode that a part of divided | segmented liver is touched. It is a figure which shows the state which hidden only the part of the divided | segmented liver. It is a flowchart which shows the flow of the operation procedure as an example. It is a figure for demonstrating the procedure of area | region designation. It is a figure for demonstrating the procedure (an example) of an area | region designation | designated. It is a three-dimensional medical image which shows the example which displayed the blood vessel with a different threshold value. It is a figure which shows the example of the screen which displays a stereo image. It is a figure which shows the example of the screen regarding concealment of patient information. It is an operation | movement flow which displays patient information in a predetermined case. It is a table which shows that each site | part of a patient's fluoroscopic image is separately managed separately beforehand in a data server. It is a figure which shows an example of arrangement | positioning of a foot switch typically. It is an example of the screen displayed in the state where voice input is ON.

Embodiments of the present invention will be described below with reference to the drawings.
[Section A: Medical image processing apparatus capable of performing various display processes with good workability and keeping the operator clean]
1. Configuration The medical image processing apparatus 301 of the present embodiment is a portable computer device such as a tablet terminal as an example. Alternatively, it may be a laptop PC (notebook PC) having a touch panel display. FIG. 1 shows an example of a tablet terminal, which may be configured by installing an image processing program according to one embodiment of the present invention in a commercially available tablet terminal. Hereinafter, the medical image processing apparatus will be described simply as an image processing apparatus. Although it does not specifically limit regarding a tablet terminal or a notebook PC, It is preferable in one form that a screen size is 9 inches or more or 10 inches or more. In one embodiment, the thickness is preferably 20 mm or less or 15 mm or less. In one embodiment, the mass is preferably within 2 kg or within 1.5 kg.

As shown in FIG. 1, in this example, the image processing apparatus 301 has a thin casing 301a, and a touch panel display 360 is provided on one surface thereof. The touch panel display 360 includes a display 361 (see FIG. 2) and a touch panel 363 (see FIG. 2).

Such an image processing apparatus 301 can be connected to a network of a hospital system, for example, as shown in FIG. The hospital system of this example includes the following devices connected to the network 30: an imaging device 1, a chemical injection device 10, a hospital information system HIS (Hospital Information System) 21, and a radiology information system. A certain RIS (Radiology Information System) 22, a PACS (Picture Archiving and Communication Systems) 23, which is an image storage communication system, a workstation 24, a printer 25, and the like. Note that not all of these are essential components, and some of them can be omitted. Each of the above elements may be only one or plural. Of course, the connection to the network may be a wired connection or a wireless connection.

Examples of the imaging device 1 include imaging devices such as a CT device, an MR device, and an angiography device. Other types of imaging devices may be used, or a plurality of imaging devices of the same type or different types may be used. A three-dimensional medical image, which will be described later, may be created using a plurality of modality images, such as combining an image captured by a CT apparatus and an image captured by an MR apparatus.

The drug solution injection device 10 may be a contrast agent injection device that injects at least a contrast agent. Specifically, the drug solution injection device 10 includes a drive mechanism that pushes a drug solution from a container (for example, a syringe) filled with the drug solution, and an operation thereof. It may include a control circuit for controlling. As an example, a contrast medium injection device including an injection head and a console can be used. The drive mechanism may be a piston drive mechanism, a roller pump, or the like.

Refer to the block diagram in FIG. The image processing apparatus 301 includes a display 361, a touch panel 363, an input device 365, a communication unit 367, an interface 368, a slot 369, a control unit 350, a storage unit 359, and the like. Note that all of these are not essential, and some of them may be omitted.

Examples of the display 361 include devices such as liquid crystal panels and organic EL panels. A touch panel display in which the touch panel 363 is integrally provided can also be used. As the touch panel, a system such as a resistive film, capacitance, electromagnetic induction, surface acoustic wave, infrared ray, or the like can be used. As a specific example, a multi-touch that is a touch at a plurality of positions, such as a capacitance method, may be detected. The touch operation can be performed using a user's finger or a touch pen. The touch panel may detect the start of a touch operation on the touch panel, the movement of the touch position, the end of the touch operation, and the like, and output the detected touch type and coordinate information.

Note that the image processing apparatus 301 of the present embodiment can perform operations related to image display using voice input or motion input, as will be described later. Therefore, the touch panel 363 may be omitted depending on circumstances.

As the input device 365, for example, a general device such as a keyboard or a mouse can be used.

The storage unit 359 includes a hard disk drive (HDD: Hard Disk).
Drive (Solid State Drive) and / or memory, etc., and an OS (Operating System) program or a medical image processing program according to an embodiment of the present invention (Including algorithm data and graphical user interface data) may be stored.

In addition, other programs used for various processes, tables, databases, etc. are stored as necessary. The computer program is executed by being loaded into the memory of the control unit, and cooperates with hardware such as a CPU, thereby constituting a control unit having functions as in the present embodiment. .

The computer program may be downloaded in whole or in part from an external device when necessary via an arbitrary network. The computer program may be stored in a computer-readable recording medium. The “recording medium” is a memory card, USB memory, SD card (registered trademark), flexible disk, magneto-optical disk, ROM, EPROM, EEPROM, It shall include any “portable physical media” such as CD-ROM, MO, DVD, and Blu-ray (registered trademark) Disc. The medical image processing apparatus according to the present embodiment may be provided with a slot 369 for reading the storage medium as described above.

The communication unit 367 is a unit for enabling communication with an external network or device in a wired or wireless manner. The communication unit 367 may include a transmitter for sending data to the outside, a receiver for receiving data from the outside, and the like. The interface 368 is for connecting various external devices and the like, and only one interface is shown in the drawing, but a plurality of interfaces may naturally be provided.

The slot 369 is a part for reading data from a computer readable medium. The interface 368 is a part for connecting an external device or the like.

The image processing apparatus 301 of this embodiment includes a microphone 370 as an audio input device. A microphone built in the housing may be used, or an external microphone that is separate from the housing and is connected to the terminal by wire or wirelessly. It is also possible to use a device in which a motion sensor 380 and a microphone described below are integrated.

In order to perform voice recognition, voice recognition software is installed in the image processing apparatus 301, and thereby a voice recognition unit 351 is configured.

(Motion sensor)
The motion sensor 380 is a sensor that three-dimensionally detects the movement of at least a part of the operator's body in a non-contact manner. Motion recognition software is installed in the image processing apparatus 301, and thereby a motion recognition unit 353 is configured.

As the motion sensor 380, for example, a leap motion controller (manufactured by Leap Motion, “Leep Motion” is a registered trademark) can be used. The leap motion controller is an input device that can recognize the position, shape, and movement of the operator's finger and / or the position and movement of the palm in real time without contact. The leap motion controller is configured as a sensor unit including an infrared irradiation unit and a CCD camera. The upper area of the sensor unit is a recognition area. This sensor unit is used by connecting to a tablet terminal or a laptop PC by wire or wireless.

As the motion sensor 380, for example, Kinect (manufactured by Microsoft Corporation, registered trademark) can be used. The motion sensor 380 may include one or more cameras and one or more distance sensors. Or you may provide only one of them. The unit of the motion sensor 380 may have a built-in microphone. For example, the accuracy of the detection target (for example, a hand) of the motion sensor 380 is preferably one that can be detected with an accuracy of 5 mm or less, and more preferably one that can be detected with an accuracy of 1 mm or less.

The detection principle of the motion sensor 380 is not limited to a specific one. As one aspect, a system called Light Coding can be used. In this method, a large number of dot patterns are irradiated from the infrared light emitting unit, and the amount of change (distortion) when the dot pattern hits the detection target (person) is read by the camera. As another aspect, a system called TOF (Time Of Flight) can be used. This is suitable for a method in which the recognition range is relatively close and sensing fine movements of fingers. In the TOF method, the distance until the irradiated infrared rays hit the object and return is measured. In general, the recognition accuracy is higher than that of the Light Coding method, and the deterioration of accuracy due to distance is small. As yet another aspect, a system in which reflection of light irradiated on an object by an infrared LED is photographed by two cameras and movement is recognized may be used.

(Control part)
Refer to FIG. 2 again. The control unit 350 includes hardware such as a central processing unit (CPU) and a memory, and a computer program is installed to perform various arithmetic processes. As a conceptual configuration, the control unit 350 includes an image display unit 355a, an operation determination unit 355b, a display processing unit 355c, and a mode selection unit 355d. Further, as described above, the voice recognition unit 351 and the motion recognition unit 353 are included.

The image display unit 355a displays a medical three-dimensional medical image on the display 361. For example, the image display unit 355a displays each of anatomical structures such as the liver and blood vessels as independent objects. In addition, each anatomical structure may be displayed in a different color. The color in which each anatomical structure is displayed may be manually input and set by the operator, but is not necessarily limited thereto. As will be described later, in the case where color assignment or the like is performed in advance on a predetermined data server side (by a table or the like), the display may be performed in accordance therewith.

The operation determination unit 355b receives an input operation on the input device 365 and the touch panel 363.

The display processing unit 355c performs various image processing. For example,
-Rotation of 3D image,
-Translation of 3D images
-3D display and enlargement / reduction of images,
-Change the transparency of images displayed in 3D,
-Switching the display / non-display of a given object,
-Cut (split) function of a given object,
-A function for specifying an area of a predetermined object, etc.
It is. The specific contents of these functions will be described in detail in a series of operations described later.

The voice recognition unit 351 performs various kinds of voice recognition. For example, the following words are recognized.
-"Rotate" command for changing display mode
-"Move" command for changing display mode
-"Multi" command for changing display mode
-"Stop" command for changing display mode
-"Cut" command for processing
-"Box" command for processing
-The name of the anatomical structure (for example, an organ name such as "liver" or a blood vessel name such as "portal vein" or "hepatic artery").

The motion recognition unit 353 performs various motion recognition processes. For example, assuming that the motion sensor detects a hand, the position or movement of the hand (finger) in the detection space is detected.

It should be noted that regarding the invention having main features in image processing and other data processing, the hardware configuration is not limited to the specific one disclosed in the above-described embodiment, and various modes can be used. Therefore, it should be noted that, for example, processing performed by other computer means is not limited to a tablet terminal and a notebook PC, and can be an object of one embodiment of the present invention. In the following, it will be understood that the invention disclosed mainly as the description of “operation” can be understood by those skilled in the art as an invention of a product or an invention of a computer program with different category expressions. . Accordingly, the present specification also discloses such an invention.

2. Operation Next, an operation example of image display in the image processing apparatus 301 of the present embodiment will be described. Hereinafter, an example in which a display operation of a three-dimensional medical image as illustrated in FIG. 5 will be described as an example. This three-dimensional medical image includes a liver 371 and a blood vessel 375.

First, as shown in the flowchart of FIG. 4, three-dimensional medical image data is acquired in step S11. The “three-dimensional medical image data” may be created based on data obtained by tomographic imaging of a patient with an imaging device. In particular, volume data by volume rendering may be used. Note that the data format of the three-dimensional image is not particularly limited, and various types can be used. For example, an STL (Standard Triangulated Language) file format can be used.

The image data may be stored in a predetermined data storage area such as a predetermined database server, PACS, DICOM server, or workstation. As an example, the image processing apparatus 301 reads the data from a predetermined data storage area on the network and stores it in the storage unit 359 in the apparatus.

Next, the image processing apparatus 301 displays a three-dimensional medical image on the display 361 (step S12). The creation of a three-dimensional medical image can basically be performed using a known method. An image creation flow according to an embodiment of the present invention will be described later with reference to the drawings. The generated three-dimensional medical image data may be stored in the image processing apparatus 301 and / or stored in an external server (for example, a server on the cloud).

Here, various display modes are prepared for the display of medical images. For example,
-Display a given anatomical structure in a translucent state;
-Display a given anatomical structure in an opaque state,
-Color display of a given anatomical structure,
-Display a given anatomical structure with a shadow,
At least one of displaying an image of a three-dimensional coordinate axis (or an equivalent thereof, for example, a cube) on the screen, and the like.

Regarding permeation, for example, the liver is displayed in a translucent state and the blood vessel is displayed in an opaque state. When there is a tumor, the tumor may be displayed in an opaque state. According to such a display mode, it is possible to confirm the position and running state of an internal blood vessel that is originally hidden behind the liver and cannot be visually recognized by making the liver translucent display. The ability to perform such confirmation is very useful, for example, in that the positional relationship between the liver, blood vessels, tumors, and the like can be well confirmed, particularly in surgery where a portion of the liver is removed by laparoscopic surgery. The image processing apparatus 301 according to the present embodiment is portable, and therefore it is possible to check a three-dimensional medical image by operating the apparatus in the operating room.

Regarding color coding, for example, the liver and blood vessels may be displayed in different colors. If there is a tumor, the tumor may be displayed in a different color. More specifically, regarding the blood vessels, the liver, portal vein, and hepatic artery may be displayed in different colors. When blood vessels are grouped, they may be displayed together in the same color.

Next, in step S13, the image processing device 301 detects the position of the hand so that the distance between the motion sensor 380 and the operator's hand is appropriate. Specifically, the operator places the hand above the motion sensor 380. An appropriate distance (height from the sensor to the hand) between the sensor and the operator's hand is set in advance, for example, in a range of h1 (mm) to h2 (mm). The reason why the appropriate range is set in advance is that there is a possibility that the movement of the hand cannot be recognized well when the operator's hand is too close or too far from the sensor.

Regarding the detection of the hand position, what kind of image is displayed on the screen is not particularly limited, but it may be as shown in FIG. 6 as an example. In this example, a reference circle (first circle) 391 having a predetermined size is displayed in the approximate center of the screen. The first circle 391 is always displayed in a fixed size regardless of the position of the operator's hand. On the other hand, a second circle 393 is also displayed on the screen.

The center of the second circle 393 corresponds to the position of the operator's hand. That is, when the operator's hand is directly above the motion sensor 380 (an example), the center of the second circle 393 is the same as the center of the circle 391 at the second position. That is, the two circles 391 and 393 are displayed as concentric circles.

When the operator's hand is displaced in a predetermined direction (as an example on the right) from the position directly above the motion sensor 380, the second circle 393 is also displaced in the same direction (as an example on the right) and displayed in real time. Is done.

With such a display mode, the operator confirms whether his / her hand is in an appropriate position (horizontal position) with respect to the motion sensor 380 while viewing the positional relationship between the two circles 391 and 393 on the screen. can do.

∙ The position in the height direction can be confirmed as follows. That is, the diameter of the second circle 393 corresponds to the height of the operator's hand. For example, when the height of the hand is the reference height (in the example, (h1 + h2) / 2), the second circle 393 is formed so that the diameter of the second circle 393 is the same as the diameter of the first circle 391. Is displayed. As the hand position increases, the size of the second circle 393 decreases accordingly, and conversely, as the hand position decreases, the size of the second circle 393 increases accordingly. With such a display mode, the operator can check whether the height of his / her hand is appropriate while viewing the magnitude relationship between the two circles 391 and 393 on the screen.

In order to make it easier to confirm that the position is appropriate, the following display may be used. That is, the second circle 393 may be displayed in a special display when the horizontal position, the height position of the hand, or a combination thereof enters a predetermined appropriate position. For example, the second circle 393 may be displayed in different colors depending on whether it is outside the proper range as shown in FIG. 6 (a) or within the proper range as shown in FIG. 6 (b). In one embodiment, it is preferable to switch between blinking display and lighting display.

As described above, the step for making the distance between the motion sensor and the operator's hand appropriate is completed (step S13).

Next, in step S14, a voice input for selecting a display mode is received. As an example of speech input, the following words may be recognized:
-"Move"
-"Stop"
-"rotation"
-"Multi"

Note that the voice recognition function may be turned on as a trigger when the position of the operator's hand enters a predetermined appropriate range in step S3. In this configuration, when the position of the operator's hand is not within the predetermined appropriate range, the voice recognition function is OFF and is ON only when it is within the appropriate range. As described above, according to the configuration in which the voice recognition function is turned on only under a predetermined condition, it is possible to prevent voice input due to erroneous recognition that is not intended by the operator.

As schematically shown in FIG. 1, in order to notify the operator that the voice recognition function is ON, for example, a display such as “During voice recognition” may be displayed on the screen. preferable.

Note that, in one aspect of the present invention, the step of detecting the hand position in S13 may be omitted.

<Zoom (enlarge and reduce) / pan>
To change the size or position of the displayed three-dimensional medical image, the following procedure is used.

First, the operator utters “move” with the voice recognition function ON. The image processing apparatus 301 analyzes the voice input from the microphone 370 by the voice recognition unit 351 and recognizes the word “move”. In response to this, a transition is made to the “zoom / pan” mode (step S15).

In the “zoom / pan” mode, the image processing apparatus 301 then waits for input of motion by the operator's hand. The image processing apparatus 301 uses the motion sensor 380 and the motion recognition unit 353 to recognize the position and movement of the operator's hand in real time. When the operator moves the hand upward (that is, when the hand moves from the initial height h ₀ to a higher h _h ), the image is gradually reduced in accordance with the movement. On the other hand, when the operator moves his / her hand downward (when moving from the initial height h ₀ to a lower h _L ), the medical image is gradually enlarged in accordance with the movement.

Also, when the hand is moved horizontally, the three-dimensional medical image is panned (translated) in accordance with the moving direction and moving amount.

As described above, in this mode, the image is reduced or enlarged by moving the hand up and down, and the image is moved in parallel by moving the hand horizontally. According to the configuration of the present embodiment, compared with input methods such as voice recognition and numerical input, zoom / pan of an image (and further described below) is performed using motion input capable of intuitive analog input. Rotation, etc.). Therefore, it is intuitive for the operator, excellent in operability, and contributes to practical use.

Further, since the motion sensor 380 can input without touching the device, the input can be performed while keeping the operator's hand clean. Such a configuration is very advantageous in that, for example, an intraoperative doctor can check an image using an image processing apparatus in an operating room.

Especially in the case of the liver, a plurality of blood vessels are present in the liver in a branch shape. Therefore, when part of the liver parenchyma is excised so as not to damage the blood vessel more than necessary, it is necessary to sufficiently confirm the positional relationship between the blood vessel and the tumor. In this regard, according to the image processing apparatus of the present embodiment, the positional relationship of blood vessels and the like can be confirmed while viewing a three-dimensional medical image during the operation. In a three-dimensional medical image, for example, a blood vessel may exist on the front side and a tumor may be hidden behind the blood vessel (the tumor is not shown in the figure, but refer to FIG. 5 for reference). Even in such a case, the image processing apparatus of the present embodiment can check the tumor by rotating the image in the “rotation” mode. In particular, in the configuration of the present embodiment, rotation is not performed at every predetermined angle, but it can be freely rotated (steplessly) by an arbitrary angle by motion input, so that good observation is performed. It becomes possible.

In one embodiment, it is preferable that the “zoom / pan” mode is configured not to perform “rotation” (details below) of the three-dimensional medical image. When using this mode, it is often the case that only enlargement / reduction or parallel movement is desired. Therefore, it is easier for the operator to perform rotation, enlargement, reduction, and parallel movement while maintaining the desired posture.

In the above description, a mode in which both zooming and panning can be performed has been described. However, the present invention is not limited to this. It is also possible to set a mode in which only one of them can be performed.

<Rotation>
If you want to rotate the displayed 3D medical image, do the following:
First, the operator utters “stop” to cancel the “zoom / pan” mode. The image processing apparatus 301 accepts this by the voice recognition function, cancels the “zoom / pan” mode, and transitions to a state of accepting another mode.

In this state, the operator says “Rotate”. The image processing apparatus 301 accepts this through the voice recognition function and transitions to the “rotation” mode. Next, the image processing apparatus 301 waits for input of motion by the operator's hand.

In the “rotation” mode, the image processing apparatus 301 recognizes the position and movement of the operator's hand in real time. Then, the image processing apparatus 301 rotates the three-dimensional medical image around a predetermined rotation axis (X axis, Y axis, Z axis) in accordance with the movement of the operator's hand. Specifically, the movement of the operator's hand in the horizontal direction or the movement of moving the hand along the surface of the virtual sphere is recognized. Then, the three-dimensional medical image is rotated by a predetermined angle corresponding to the moving direction, moving speed, and moving amount.

Also in this rotation mode, it is preferable in one embodiment that only rotation is allowed and panning (parallel movement) and zooming (enlargement / reduction) are prohibited. Accordingly, for example, it is possible to rotate the image in a desired direction while maintaining a predetermined image size, and perform subsequent predetermined image processing and observation.

* Even when canceling the “rotation mode”, the operator speaks “stop” as described above.

In the above description, when one of “move” and “rotate” is performed, the other is not performed. However, the “multi” mode is used so that both inputs can be performed simultaneously. May be prepared. In this mode, “zoom”, “pan”, and “rotation” are all performed according to the movement of the hand of the operator.

Specifically, when the operator utters “multi”, the image processing apparatus 301 recognizes this and shifts to the “multi” mode. The three-dimensional medical image is rotated, moved, enlarged / reduced according to the input of the motion of the operator's hand.

It should be noted that a function capable of rotating a three-dimensional medical image only by voice input instead of motion input may be implemented. For example, the image processing apparatus 301 recognizes this by saying “rotation”, “left”, and “15 °”. Then, the image is rotated by 15 ° around a predetermined rotation axis (for example, the Z axis extending in the vertical direction of the screen). If it is desired to rotate 15 ° upward around the axis extending in the horizontal direction, for example, “rotation”, “up”, and “15 °” may be input as voices.

<Other voice input>
The image processing apparatus 301 according to the present embodiment can select an anatomical structure portion or change the transmittance of the selected one by voice input. This will be described again after explaining the operation through the touch panel.

(About various functions through touch panel input)
The image processing apparatus 301 displays anatomical structures in the three-dimensional medical image as independent objects. Thereby, each can be selected individually or display can be switched on and off. For example, hepatic arteries, portal veins, and hepatic veins may be grouped and selected at once, or may be selected individually.

(Rotation function)
The rotation of the three-dimensional medical image can also be performed by an operation on the touch panel. When the operator touches the touch panel and moves his / her finger, the image processing apparatus 301 rotates the three-dimensional medical image accordingly.

(Enlarge / reduce function)
The image processing apparatus 301 also responds when the operator touches two points on the screen and performs an operation (a pinch-out operation or a pinch-in operation) to increase or decrease the distance between the two points. The image may be enlarged or displayed.

(Display transparency switching function)
The image processing apparatus 301 may change the display density when an operator touches any anatomical structure (for example, liver). Specifically, switching between two states of a normal opaque display state and a semi-transparent state may be performed. That is, as an example. It may be a semi-transparent state when touched once and return to a normal display state when touched again.

As another aspect, for example, the transparency is set to a plurality of stages such as 0%, 30%, 70%, and 100% (non-display), and the display density is sequentially switched in a loop shape every time the touch is performed. It may be. In this case, the transparency of 100% (that is, the non-display state) may be excluded from this loop. Naturally, the specific numerical value of the transmittance can be changed as appropriate. In short, it is only necessary that the transparency is set to at least a plurality of stages, and they are switched.

According to such a display transparency switching function, the display transparency switching function can be exhibited simply by touching an arbitrary anatomical structure. Therefore, the operation can be performed simply and intuitively as compared with a method in which some icon or the like is separately selected or a command needs to be selected in order to switch the transparency.

Also, when the display is switched in a loop shape as described above, it is preferable in that it is not necessary to separately select an icon or the like in order to return to the original display state. Furthermore, such a loop-like display switching can be realized only by changing the display color of the selected object, which is preferable in that the image processing can be simplified and the arithmetic processing can be performed with a small amount of memory.

Gestures for changing the transparency are not limited to those described above. For example, when a finger (for example) is swiped in the vertical direction or the horizontal direction, the image processing apparatus accepts the input, and the transparency is changed accordingly. It is good also as a structure changed. In this case, the transparency may be set in several steps, for example, 0%, 30%, 70%, 100% (non-display), or instead, in a stepless manner (continuously). It is good also as a structure which the transmittance | permeability changes.

(Display / non-display switching function)
When the operator has touched a predetermined anatomical structure for a certain period of time (an example), the image processing apparatus 301 sets the anatomical structure as “selected”. In order to indicate the “selected state”, the anatomical structure (for example, the liver) may be displayed in a color different from the initial state or may be blinked.

When the operator moves the fingertip (one example) toward the screen edge while touching the selected anatomical structure (for example, liver) (swipe operation, drag operation, etc.) ) And hide the anatomical structure. In this example, the liver is not displayed, and only a three-dimensional image such as a blood vessel remains.

This function is useful when the operator wants to see only the desired anatomical structure. In addition, according to the method in which the anatomical structure can be directly selected and moved only by being moved as in the present embodiment, the display is turned on / off only when, for example, some icon is selected. Compared to the mode, the operation can be performed easily and intuitively.

(Cut function)
The cutting function is performed as follows. An example in which the liver is cut and then a part thereof is not displayed will be described below. FIG. 8 is a flowchart of a series of operations.

The image processing apparatus 301 first displays a three-dimensional medical image as shown in FIG. 7A as step S1. Then, when the operator touches two points on an arbitrary anatomical structure (here, the liver 71) as shown in FIG. 7B, the state, that is, the two points touch. Is determined (step S2). In addition, as timing, two points may be touched simultaneously or substantially simultaneously.

Next, in order to perform the function in the case of a so-called long press, the image processing apparatus 301 determines whether or not the state where the two points are touched has continued for a certain period of time (step S3).

The image processing apparatus 301 displays on the screen in a predetermined display manner that the two touched points P1 and P2 are designated when it is determined in step S3 that the continuation is longer than a certain time. Any "predetermined display mode" may be used. For example, (i) both the points P1 and P2 and the line L1 connecting them are displayed, or (ii) only the points P1 and P2 are displayed. Alternatively, only the line L1 may be displayed. Regarding the points P1 and P2, not only a small dot but a slightly larger graphical image as shown in FIG. 7B (for example, any shape such as a circle, a rectangle, a polygon, a star, etc.) so that the designated position can be clearly understood. However, it is also possible to display such as a circle).

The image processing apparatus 301 may display the designated points P1 and P2 as they are as shown in FIG. 7C even after the operator releases the hand from the screen. Moreover, you may be comprised so that the fine adjustment of the position of the points P1 and P2 may be received. In this way, for example, the circular graphical image of P1 and P2 and / or the line L1 may be blinked so that it can be understood that the mode is a mode for accepting fine adjustment. FIG. 7C illustrates a state in which the point P2 is slightly moved and finely adjusted to the point P2 ′ as an example.

This fine adjustment may be performed by the operator moving the graphical images of the points P1 and P2 with a finger, for example (operation on the touch panel). As another aspect, motion input may be used to finely adjust the positions of the points P1 and P2 without contacting the device. Note that displaying the cutting reference line L1 by voice input will be described later again. Here, the cut function and the like of the present embodiment will be described first on the premise of touch panel operation.

After the designation of the points P1 and P2 is completed in this way, in step S4, for example, the operator touches a predetermined icon (for example, an “OK” input icon) on the screen. Then, the liver is cut by the line L1 connecting the points P1 and P2 as shown in FIG. 7D by the cutting function (step S5).

The first part 71-1 and the second part 71-2 divided into two with the line L1 in between can be operated as independent anatomical structures. As a method other than the above operation, for example, (i) not touching an icon, but touching a predetermined area on the screen, or (ii) touching multiple times (double tap in one example) The above functions may be executed by performing an external operation. It may be by voice input.

Since it can be operated as an independent anatomical structure, for example, when the first part 71-1 is touched (step S6), the part as shown in FIG. Only selected. Then, the display density is switched. Specifically, only the first part 71-1 is translucently displayed. Touch again to return to the original display.

Further, for example, when the first part 71-1 is long pressed and swiped or dragged to the periphery of the screen, the part is not displayed, and the second part 71-2 and the blood vessel 73, Only 75 remains. The non-displayed image may be displayed as a thumbnail image 66 as illustrated in FIG. 7F.

(Area specification)
The image processing apparatus 301 designates a part of an anatomical structure by the following operation of the operator. FIG. 9A shows a state in which two points P1 and P2 are touched as in the operation described with reference to FIG. 7B (note that the operator's finger remains touching two points on the screen. (The illustration is omitted).

Next, from this state, for example, as shown in FIG. 9B, when the operator moves two fingers (but not limited to two fingers simultaneously), the image processing apparatus 301 moves. The positions of the latter two points P1 ′ and P2 ′ are specified, and a substantially rectangular area is designated based on the positions. Specifically, a quadrangle surrounded by four points, two points P1 and P2 before movement and two points P1 ′ and P2 ′ after movement, is designated as an area.

Here again, as described above, even if the operator releases his hand, a substantially square of the designated area remains on the screen, and the position of each point P1, P2, P1 ', P2' is moved individually. It is also preferable that the image processing apparatus 301 is configured so that fine adjustment can be performed. As a method of determining the designated area, for example, the operator may touch a predetermined icon (for example, an “OK” input icon) on the screen.

As shown in FIG. 9B, when the positions of the four points P1, P2, P1 ′, and P2 ′ are designated, the circles of the points P1, P2, P1 ′, and P2 ′ can be finely adjusted (an example). You may make it display the graphical image and the line which connects them blinking.

The designated area Sa1 (see FIG. 9B) is divided from other parts and can be operated as an independent object. Therefore, it is possible to change the display density of only the region or switch the display on and off. According to such a function, for example, by not displaying only the region Sa1, it is possible to observe the inner blood vessels 73 and 75 and to confirm the relationship between the blood vessels 73 and 75 and the liver 71.

It should be noted that the area designation is not necessarily performed with a quadrangle, and a triangle or a polygonal shape of a pentagon or more may be area designated.

In the above description, medical images of the liver and its surroundings are taken as an example, but of course, the anatomical structure is not limited to a specific one in the present invention. For example, a medical image of the examiner's head may be displayed and various image processing may be performed on the medical image.

[Other functions by voice input]
(Display / emergency switching)
It is also preferable that the display non-display switching function and the cut function as described so far can be executed only by voice input or the like without performing input to the touch panel.

First, regarding the selection of a predetermined anatomical structure, selection is performed when a target is uttered by voice instead of touching and recognized. For example, if the operator says “liver”, the image processing apparatus 301 recognizes it and sets the liver to the selected state. In order to indicate the “selected state”, the anatomical structure (for example, the liver) may be displayed in a color different from the initial state, or may be blinked.

And, when it is desired to change the permeability of the liver, for example, the operator utters “Transparent”. The image processing apparatus 301 recognizes the voice and switches the display to a semi-transparent display. In this discharge, other anatomical structures (in one example, blood vessels and tumors) continue to appear impermeable. According to such a configuration, it is possible to confirm blood vessels and the like that are otherwise hidden behind the liver and cannot be seen.

As one embodiment, for example, the transparency may be changed according to the distance from the motion sensor to the operator's hand. That is, the transmittance gradually increases (or decreases) when the hand is brought closer to the motion sensor, and conversely, the transmittance gradually decreases (or increases) when the hand is moved away from the motion sensor. . Specifically, when the operator speaks “transparent” and the image processing apparatus 301 recognizes the voice, the mode is set to accept the motion input as described above. The device then detects the distance from the motion sensor to the operator's hand and changes the transparency accordingly.

(Line cut / box cut)
When performing line cut, for example, the operator utters “line cut”. The image processing apparatus 301 recognizes it and displays a cutting reference line on the screen. This reference line may be like the line L1 in FIG. 7B.

Then, the image processing apparatus 301 waits for motion input. The operator can change the position, length, orientation, etc. of the cutting reference line by motion input. Thereby, the reference line can be set at a predetermined position without contact.

Next, for example, “right cut”, the right region of the reference line is removed. Instead, the left region of the reference line is removed by saying “left cut”. Instead of removal, a semi-transparent display may be used. FIG. 5 (b) shows an example of this, and the portion 371-2 on the right side of the reference line remains opaque and the portion 371-1 on the left side is semi-transparent.

When performing box cut, for example, the operator utters “box cut”. Although detailed illustration is omitted, the image processing apparatus 301 recognizes the voice and displays a rectangle (one example) as a reference for excision on the screen. The size of the rectangle may be only one predetermined size, or a plurality of sizes such as large, medium, and small may be prepared.

This box cut is to cut the target anatomical structure at a predetermined depth. The image processing apparatus 301 waits for motion input while displaying a rectangle serving as a reference for excision on the screen. The size and shape of the quadrangle may be fixed, but may be changeable. For example, the size and shape of the quadrangle can be changed by moving the corners of the default quadrangle displayed first. Motion input can be used to move the corner position.

In this state, for example, when the operator moves his hand closer to the motion sensor 380, a substantially rectangular parallelepiped hole having a predetermined depth corresponding to the moving distance of the hand is formed in the liver correspondingly to the rectangle as an outline. It will be done. As a result, it is possible to obtain a medical observation image such that a part of the liver is excised and an internal blood vessel is not excised.

Note that it is also possible to rotate the entire three-dimensional medical image by a predetermined angle in a state where the excision part having a predetermined depth is formed in this way. For example, when the operator utters “up”, “15 °”, or the like in the rotation mode, the image processing apparatus 301 recognizes it and rotates the medical image in which the hole is formed by 15 °. . According to such a configuration, the internal configuration of the hole (for example, a part of the liver is excised but the blood vessel is displayed) can be observed from different angles, which is useful.

In the above description, the removal is performed using a rectangular outline, but it is needless to say that the outline may be defined by a triangular, polygonal, circular, elliptical or other arbitrary geometric shape.

In the above description, the area designated as a box is cut out, but conversely, only the area designated as a box may be left and the other areas may be hidden.

[Section B: Collective handling of multiple anatomical structures]
1. Problems of the Invention of this Section As described above, volume data or the like is used as a three-dimensional medical image as illustrated in FIG. By the way, in such a three-dimensional medical image (that is, an image including several different types of anatomical structures), CT values (signal values) of the data are different between the liver and blood vessels. Yes. Even in the same blood vessel, for example, the CT value (signal value) is different for an artery, vein, portal vein, or the like.

This is because in fluoroscopic imaging using a contrast agent, image data is obtained by performing fluoroscopic imaging after a predetermined time after injecting the contrast agent, depending on the difference in the arrival time of the contrast agent, etc. This is because there is a difference in CT values (signal values) of each part such as an artery, vein, portal vein, and liver parenchyma. In the conventional technique, threshold data are set and filtered for each blood vessel or organ to create volume data for each part.

However, even if the blood vessels are the same, for example, when the data of arteries, veins, and portal veins are individually registered due to differences in CT values (signal values), the following operations are relatively time-consuming. May be required. That is, in the viewer function of a three-dimensional medical image, for example, it is possible to switch between displaying with a blood vessel emphasized and displaying without emphasis. By being able to do this, for example, the peripheral portion of the blood vessel (the CT value (signal value) is low) can be switched as necessary or not, and observation can be performed as necessary. (See FIG. 10). However, in the case of a three-dimensional medical image including a plurality of different types of blood vessels, the display mode of all blood vessels is uniformly changed for each blood vessel unless the display threshold is reset or filtered. Therefore, there is a problem that it is difficult to easily change the display.

On the other hand, in actual observation, for example, it may be preferable that the vasculature can be handled by dividing it into a vascular system and a parenchymal system. Therefore, in the invention of this section, the image processing apparatus has the following functions.

2. Function and Operation The image processing apparatus according to the present embodiment reads volume data based on information obtained by imaging a patient (see also step S1 in FIG. 3).

Then, the signal value, CT value, and standard deviation (SD) in the volume are analyzed. For example, if the average CT value is 300 HU or more, it is automatically determined as an artery, and if the average CT value is 100 HU or less, it is automatically determined as an organ. In addition to the CT value, the histogram shape of the CT value is also recognized. Generally, arteries tend to have a high peak and a narrow width (distribution), and portal veins and veins have a low peak and a wide width (distribution). Therefore, automatic recognition of the blood vessel type can be realized based on such elements.

As described above, it is possible to automatically distinguish and register data of arteries, veins, portal veins, and the like that originally have different CT values (signal values) by the image processing apparatus. In addition, different colors may be automatically assigned to arteries, veins, portal veins, etc., and displayed in different colors.

In addition, a histogram of blood vessels originally exists for each artery, vein, portal vein, etc., but each is integrated and normalized, and all blood vessels (in another embodiment, arbitrary blood vessels are normalized). It may be possible to represent two or more. Specifically, as an example, the average value and the center of gravity of each histogram may be calculated, and one histogram may be created by shifting the whole so that the lower one matches the higher one.

In this way, when a plurality of blood vessels (other anatomical structures may be combined) are expressed in a single histogram, an operation is not performed on the histogram of each blood vessel, but only by an operation on one histogram. The display can be changed automatically. That is, for example, when it is not necessary to display the peripheral portion of the blood vessel, the image processing apparatus accepts a predetermined input from the operator and does not display a portion below a certain reference value (or below a certain reference value). By performing processing such as “do not display this part”, peripheral parts such as arteries, veins, and portal veins can be collectively hidden (see, for example, FIG. 10B). On the other hand, when it is desired to emphasize and display the peripheral portion of the blood vessel, the lower limit value of the CT value to be displayed may be set lower as shown in FIG. 10A (the threshold value is 130 HU here). .

According to this configuration, since it is necessary to change the display modes of the arteries, veins, portal veins, etc., the operability is very good and contributes to practical use.

The above-described “predetermined input from the operator” may be performed by operating an image button such as an icon, a cursor, or a slider on the screen, for example. Alternatively, a predetermined gesture of an operator's finger on the touch panel may be recognized and performed based on it.

For example, in a state in which the display of the blood vessel is changed, the blood vessel display may be changed by the operator touching the touch panel with several fingers and simultaneously moving the fingers in a predetermined direction. . More specifically, when several fingers are simultaneously moved upward (in the first direction), the peripheral part of the blood vessel is displayed, and conversely, the fingers are moved downward (in the second direction). When moved, the peripheral portion of the blood vessel (more precisely, the vicinity of the outer edge of the thick portion of the blood vessel) may be removed.

It is also preferable that the above-described operation can be performed by a motion input via the motion sensor 380 instead of an operation on the touch panel. That is, in this configuration, the display of blood vessels (for example, other anatomical structures may be used) can be switched only by voice input and motion input, so it is not necessary to touch the touch panel, and the cleanliness is maintained. The three-dimensional medical image can be observed as it is.

[Section C: Other functions]
(1) 3D-Pointer As a usage mode of the three-dimensional medical image as illustrated in FIG. 5, for example, a plurality of medical workers confirm the running state of blood vessels (example) before the operation and simulate an actual operation. And so on.

In the case of a two-dimensional medical image, for example, when a pointer appears as an image on the screen and is positioned at a predetermined part, the part can be watched. However, in the three-dimensional medical image, the pointer needs to be arranged at an arbitrary part in the three-dimensional space, not in the plane. Such an operation of moving the pointer to an arbitrary part in the dimensional space is relatively difficult to perform with an input interface such as a mouse or a touch panel.

Therefore, in this embodiment, the pointers may be arranged three-dimensionally using motion input. Specifically, the medical image processing apparatus 301 first receives an input of a “pointer” (an example) using a voice recognition function. Then, as an example, a three-dimensionally displayed pointer is displayed on the screen.

In the vertical direction and the horizontal direction on the screen, the pointer may be moved in accordance with the movement of the operator's hand in the horizontal plane. With respect to the depth direction, the pointer may move in the depth direction of the three-dimensional medical image when the operator brings his hand close to the motion sensor 380, and the pointer may move in the near direction when the operator moves away.

Regarding the display of the pointer, a display mode in which the display size gradually decreases as it moves in the back direction and gradually increases as it moves in the front direction may be adopted.

(2) Writing of a schema image In diagnosis and examination, a two-dimensional picture called a schema image representing a body part of a patient may be used. Therefore, an image processing apparatus according to an aspect of the present invention may include a schema image writing function.

For example, when there is a predetermined input by the operator, the image processing apparatus creates a schema image corresponding to the data using data of a three-dimensional medical image (see, for example, FIG. 5). The schema image may be any two-dimensional image such as a line drawing, a monochrome image, or a color image. Examples of the predetermined input by the operator include various inputs such as an input by touching an icon on the screen, a voice input, or a predetermined gesture input via a motion sensor.

As an example of using the data of a three-dimensional medical image and creating a corresponding two-dimensional schema image, for example, a medical image displayed in an orientation (example) as shown in FIG. 5 is currently displayed. It is also possible to write the data by converting the image into two dimensions as it is. At this time, a line drawing may be created by performing contour extraction processing. The data format may be any format, but for example, a PDF (Portable Document Format) format or any other image format such as GIF, PNG, or JPEG can be used. A doctor can write a sketch, a finding, or the like on the schema image created in this way, for example, with a touch pen or a finger.

The schema image created by the image processing apparatus can be sent out from the apparatus and stored in a predetermined storage area connected on the network (see FIG. 3). For example, it may be imported as a part of the electronic medical record.

(3-1) Concealment The image processing apparatus according to the present embodiment is a portable type such as a tablet terminal, and can be taken out of the hospital and used in some cases. Such a configuration may be useful, for example, when performing a simulation of a procedure while confirming a three-dimensional medical image of a specific patient outside the hospital. However, it is necessary from the viewpoint of security that the internal information is kept secret when it is taken out of the hospital.

Therefore, the image processing apparatus according to an embodiment of the present invention preferably has the following functions: (a) a function for recognizing the current position of the apparatus, and (b) the apparatus based on the outside of the hospital. A function for determining whether or not the device exists (or whether or not the device does not exist inside the hospital), and (c) if determined to be outside the hospital (or determined not to exist inside the hospital) Function to automatically hide the predetermined information held in the device.

As the information to be hidden, for example, information (identification information) including at least information that can identify a patient corresponds to this. Note that, instead of simply displaying, the target information may be encrypted, or access to the information may be prohibited.

As a method for determining whether the hospital is inside or outside, for example, it may be based on whether it is within the range of the wireless network system in the hospital.

Thus, according to the configuration in which predetermined information is automatically concealed when the device is taken out of the hospital, it contributes to prevention of leakage of patient information and the like, which is more preferable from the viewpoint of security.

It should be noted that the operational effects of concealment as described above are not necessarily limited to being inside or outside the “hospital”. Such concealment may be performed in a target specific area (any facility or place may be used).

(3-2)
Regarding the concealment, the following functions may be further provided. FIG. 12 is a diagram schematically illustrating a state in which concealment is performed. In this screen, all information that can identify the patient is concealed. An icon 441 is displayed on the screen.

By the way, it may be assumed that a doctor who is viewing a three-dimensional medical image may want to temporarily confirm patient information, for example, when it is desired to confirm which patient the image belongs to.

Therefore, the medical image processing apparatus of this example first determines that the icon 441 has been pressed (FIGS. 12 and 13), and then displays patient information. The patient information may be data stored inside the apparatus, or may be data obtained by accessing an external server (a server in a hospital system, for example).

More specifically, it is also preferable that the conditions under which such patient information can be displayed are limited to certain conditions. For example, this is a case where the fingerprint authentication of the operator is performed on the apparatus. Of course, it may be a case where the identity authentication is performed by another authentication method.

It is also preferable that the communication with the external server is an example, and is configured to be possible only under secure communication conditions such as VPN (Virtual Private Network).

As the information to be displayed, for example, one, two, or three or more of the patient's initial, date of birth, sex, age, address, operation date, doctor in charge, etc. may be used. Patient ID, examination ID, etc. may be displayed.

When confirming patient information under the above circumstances, only the minimum necessary information about the patient etc. (for example, contents that can identify the patient and identify the operation date or doctor) is displayed. In one form, it is preferable.

The displayed patient information may be automatically hidden again after a certain period of time. Alternatively, once displayed, the display may be continued during the operation (in one example, the display is continued until the operation is completed (logout)).

As described above, the minimum patient information and / or operation information can be confirmed as necessary, so that the displayed 3D medical image is mistaken for the patient who is actually operated on. The possibility of occurrence of such problems can be reduced.

In addition, regarding the above features, the present specification discloses not only an apparatus invention but also a method and program invention corresponding to the above contents.

(4) Stereo Image The medical image processing apparatus according to an aspect of the present invention may have a function of displaying a stereo image as described below.

As shown in FIG. 11, an image including the first image 431L and the second image 431R may be displayed so that the operator can view stereoscopically. The first image 431L and the second image 431R display the same subject with a predetermined parallax. Sometimes referred to as a left-eye image and a right-eye image.

In the screen including the images 431L and 431R, an operation pad area 433 may be displayed as shown in FIG. The operation pad area 433 is an area for changing the display angle of the subject. When the operator touches and moves his / her finger in the area 433, the medical image processing apparatus changes the display angle of the subject (two images) at the same time accordingly. That is, the direction of the subject can be changed in conjunction with the movement of the finger.

In addition, although said operation assumes operation with respect to a touch panel, it is not limited to it as an input method, A various thing can be utilized. For example, one or more of the contactless input methods disclosed herein can be utilized.

As shown in FIG. 11, in the case where one operation pad area 433 is displayed together with the first image 431L and the second image 431R, the operator operates the orientation of the subject image by operating it. You can intuitively understand that you can change.

The subject image is not particularly limited, but may be a blood vessel contrast image.

As described above, according to the configuration in which the first image 431L and the second image 431R are displayed, the doctor can more accurately grasp the three-dimensional structure of the subject through stereoscopic vision.

The apparatus according to an embodiment of the present invention described above has a function of displaying a first image and a second image having different parallaxes. More specifically, it further has a function of displaying an operation pad area for operating those display angles. For example, one or a combination of gesture input, voice input, motion input, and the like can be used to change the display of these stereoscopic images. In addition, this specification also discloses the invention of the method and program corresponding to the said content.

(5) Flow of image creation As a method of creating a medical image, for example, the following method may be employed. In this method, first, a fluoroscopic image of a patient is stored in a predetermined data server (for example, a DICOM server: one that stores data received from a modality in a predetermined format).

Then, in the predetermined data server (or other computer), individual parts of the subject are automatically recognized and managed as separate independent objects. As shown in the table illustrated in FIG. 14, management may be performed by dividing into sections such as arteries, veins, bones, organs, and the like (further subdivided sections). Information for selection by the voice recognition function (for example, voice input of “aorta” for an aorta) may be set for each object.

Also, automatic discrimination keys (symbols, alphabets, numbers, and combinations thereof) may be set so as to be convenient when selectively reading only necessary objects among individual objects.

Also, an offset value may be set for each individual object. In this example, the offset value (see “CombineOfs” in the table) is set to, for example, “+50” for the aorta, “+100” for the abdominal artery, “+400” for the vein, and the like. .

The offset value is used as follows as an example. For example, it is assumed that the central CT value of the artery is 350 HU and the central CT value of the vein is 200 HU. In this way, blood vessels have different degrees of contrast in arteries and veins (and portal veins). Therefore, in the above example in which the offset value is used to align them, the vein offset value is set to +150 HU, and the contrast effect is virtually increased so that the artery and vein are handled with the same threshold setting. It becomes possible. Of course, the offset value may be set not only in the artery and vein but also in other blood vessels such as the portal vein.

Also, the following usage may be performed with respect to the offset value of the real system instead of the vascular system. Here, as an example, it is assumed that the offset value of a real organ (for example, liver) is set to a large value such as +700 HU. By setting the real organ to such a large value, there is an advantage that the organ shape is easily kept as follows.

That is, for example, when it is not necessary to display the deletion of the artery, an operation of increasing the threshold value so that the deletion is not displayed is assumed. In this case, however, the display of the real organs such as the liver changes accordingly. The shape will collapse. In order to prevent this (that is, the real organ remains in its original form while changing the display mode of blood vessels), the real organ may be displayed with a large offset value such as +700 HU.

(Data conversion)
When blood vessels, bones, organs, and the like are managed as separate objects in a predetermined data server (for example, DICOM server) (set in a table as an example), there are the following advantages. That is, when composing a medical image on a tablet terminal or the like, it is not necessary to manually input individual object information (text for speech recognition, offset value for image synthesis, etc.), and the medical image can be easily used. An image can be created.

In this embodiment, it is basically necessary to create a medical image dedicated to the medical image processing apparatus. At that time, the presence of the setting table as described above is preferable in that the labor of production can be saved as much as possible. However, depending on the surgical procedure (what kind of surgery is performed, etc.), it may be preferable that several tables are preferably prepared. This is because it is preferable that the A blood vessel (or A organ) can be clearly seen in one surgical procedure, but the B blood vessel (or B organ) can be clearly seen rather than the A blood vessel (or A organ) in another surgical procedure. This is because it may be preferable. That is, it is preferable in one embodiment that several tables each having an offset value corresponding to the technique are registered.

In this case, the system or apparatus may be configured as follows. That is, (i) a plurality of techniques are displayed, (ii) an operator selects one of them, and (iii) a corresponding table is called (displayed as necessary). May be.

The display of the surgical method may be performed in accordance with a mode (part selection) for selecting which part of the body to perform the treatment in, for example, a user interface for setting injection conditions. For example, it may be configured to display a technique corresponding to a selected predetermined part (head, chest, abdomen, etc.) (see (i) above).

(5) Combined use with physical switch etc. The embodiment described above uses voice input or motion input (input corresponding to human movement). In one embodiment of the present invention, a predetermined physical switch may be used in combination with these inputs.

Specifically, a switch (for example, a foot switch) that detects physical contact can be used. The “foot switch” is an example that is used by being placed on the floor, and includes a switch housing on which a sensor and a substrate are placed, and a pressing unit that can be stepped on with a foot or the like. The pressing portion is not limited, but may be a portion configured to be movable so as to be pushed down when stepped on. The detection signal from the foot switch may be supplied to the outside via a cable (wired), or may be configured to be supplied to the outside wirelessly.

The foot switch may be electrically connected to the medical image processing apparatus of the present invention (for example, the control unit 350, see FIG. 2), but is not limited thereto. A configuration in which the foot switch is connected to another device (provided as part of the chemical injection device in one example) may be used.

FIG. 15 shows an example of the arrangement of foot switches. In this example, the chemical injection device includes an injection head 475 disposed near the imaging device 470, a first control unit (power supply unit) 478 connected thereto, and a console (second control unit) connected thereto. 476). The foot switch 477d is connected to the power supply unit as an example.

(Example of using physical switches)
In the apparatus of one embodiment of the present invention, a plurality of reception conditions may be set for voice operation. A foot switch may be used as one of the reception conditions.

First, as a plurality of reception conditions for voice operation, the following settings may be made (one or more).
1) Accept voice input only when there is motion input 2) Accept voice input only when the foot switch is ON 3) Only when there is motion input and the foot switch is ON Accept voice input 4) Always accept voice input

Note that the input conditions as described above may be registered in the voice operation command table of the device. In other words, the input 1) means that no voice input is accepted in the absence of motion input.

It is also preferable that the following input method is used for voice input. That is, for example, when a combination of direction and angle is entered such as “45 ° left, 30 ° above”, a command input is accepted. This is because if the information is not combination information, the probability of misrecognition may increase. This will be described in detail below.

If voice input is ON and only words such as “up” and “previous” can be recognized, in some cases, the word in the conversation being performed is recognized, and the intention is It is also assumed that no input is made. Therefore, a configuration may be adopted in which a command is accepted only when a combination of a plurality of words is recognized.

Such a voice input method for preventing erroneous recognition is not necessarily limited to the combination of direction and angle as described above. As a specific example, consider the case of moving an image forward. In this case, when the word “direction” (one example) for identifying the operation mode to be moved is recognized and the word “front” (one example) that is the direction to be moved are recognized, the command is Accept. From the viewpoint of preventing misrecognition, an input such as “direction” + “front” may be accepted, but an input such as the opposite “front” + “direction” may not be accepted (that is, the order of combination is determined). ing). In addition, regarding a command for enlarging an image, the command may be received only when a combination such as “3D” + “enlarge” is recognized, for example, instead of simply recognizing the word “enlarge”. . With such a configuration, an unintended voice input can be prevented, and it becomes more convenient.

As an aspect of the voice input, in addition to the above aspect or together with the above aspect, a structure may be adopted in which the voice input is accepted only for a certain period of time (a configuration in which the voice input is not accepted other than the certain period).

Regarding the input of the foot switch, the foot switch may be turned ON only while the pressing part is stepped on. As a process for the input of the foot switch, the subsequent process may be performed only when the foot switch is stepped for a certain period of time (so-called long press). According to this, it is possible to prevent unnecessary processing from being performed by unintentionally pressing the foot switch. The input as described above may be performed for a process in which such processing is a concern. In addition, the above operation may be performed when other switches (physical switches) are ON instead of the foot switch.

∙ Regarding voice input, the following functions may be provided. Here, in a typical image processing apparatus according to an aspect of the present invention, input is performed by voice input only in a predetermined mode suitable for voice input, and input is performed by another input method other than that. It shall be. In such a case, when a predetermined input is made (for example, “speech all ON” is entered as an example), voice input that is not defaulted by default (at least a part of them) is also voiced. A function that enables input may be provided. However, such expansion of voice input is just an example, and may return to the original state after a certain time has elapsed (timeout function). Although not limited, a time-out period of about 1 minute, 3 minutes, or 5 minutes may be set in advance.

Although not limited, regarding the input for designating the angle, it may be configured to always react only to voice. In addition, a table for each word of voice input is prepared, and under what circumstances the input is permitted for each word (for example, the foot switch is ON for the word input “aaa”) Otherwise, the word input “bbb” is always accepted) may be set.

(Home position setting function)
By the way, the image processing apparatus of the present invention may basically be configured such that a medical image viewed from a certain direction is always displayed by default regardless of the part. In one example, the configuration is such that an image viewed from the front of the body is always displayed by default regardless of the site and / or technique.

However, it is also preferable in one embodiment that the configuration is such that an image viewed from a preset angle is displayed by default, which is preferable depending on the type of the region and / or technique. For example, in thoracoscopic surgery, the lateral position is fundamental. Therefore, in the case of thoracoscopic surgery, the orientation of the lateral position may be set as the home position and the default display may be performed.

In other words, according to one aspect of the present invention, not only the front view but also different display angles suitable for each are preset according to the part and / or the type of surgical procedure, and the display is defaulted. It may be.

The selection of the home position may be set manually or automatically according to at least one of the surgical method, the site, and the position of the tumor. In a specific example, whether the lesion is in the left lung or the right lung determines whether to be in the left lateral position or the right lateral position. A configuration is also useful in which the position of a lesion (tumor) is automatically recognized and one of them is automatically determined accordingly.

(Example of screen display)
The screen displayed when the voice input is ON may be as shown in FIG. 16, for example. This screen is displayed by adding several images to the display example shown in FIG. 6, but the images (see reference numerals 391 and 393) in FIG. 6 may be omitted. It will be easily understood by contractors.

As shown in FIG. 16, there may be a voice recognition (voice input) ON display 397b on the screen so that it can be understood that voice recognition (voice input) is possible. Similarly, for motion input, there may be a motion input ON display 397a. It is also preferable to have a display unit 398 that displays the recognized voice as characters. This makes it possible to visually confirm what kind of voice input has been made.

Further, it may have a display unit 399 indicating whether or not the recognized voice is accepted as a command. If it is not accepted, a display such as “REJECTED” may be displayed, and this allows the operator to visually confirm that it has not been accepted.

The embodiments of the present invention have been described above with reference to the drawings. However, the present invention can be variously modified without departing from the spirit of the present invention and is not limited to the above examples.

Although a plurality of technical features have been described above, each of the technical features disclosed above can be used in appropriate combinations, except for cases where they are not in conflict with each other, and the present specification includes such contents. Is also disclosed. Also, for convenience of explanation, one or more of these features may be omitted for portions where some technical features are described as one embodiment. It will be understood by those skilled in the art that any invention can be grasped even by the technical contents described without distinguishing between the invention of the product, the invention of the method, or the invention of the computer program.

(Appendix)
This specification discloses the following invention (note that the reference numerals in parentheses do not limit the present invention):
1. A display (361);
A control unit (350) connected to the display;
An audio input device (370);
A motion sensor (380),
A medical image processing apparatus (301) comprising:
The control unit (350)
a: an image display unit for displaying a three-dimensional medical image on the display;
b: a mode selection unit for recognizing a voice input using the voice recognition device (170) and switching a mode related to the display of the three-dimensional medical image accordingly;
c: a display processing unit for recognizing an operator's motion input via the motion sensor and changing the display of the three-dimensional medical image accordingly;
Having
Medical image processing apparatus. Alternatively, an apparatus or system having at least a control unit having the above-described characteristics.

2. As the mode relating to the display of a three-dimensional medical image,
Zoom mode, to enlarge and reduce the image
Pan mode to translate the image,
Rotation mode to rotate the image,
The medical image processing apparatus according to the above, having at least one of the above.

3. In the zoom mode, when the operator's hand is moved in the first direction, the three-dimensional medical image is enlarged, and when the operator's hand is moved in the opposite second direction, the three-dimensional medical image is reduced.
The medical image processing apparatus as described above.

4). The medical image processing apparatus as described above, wherein in the zoom mode, enlargement or reduction is performed without rotating the three-dimensional image.

5). In the rotation mode, the three-dimensional medical image rotates in response to the movement of the operator's hand.
The medical image processing apparatus as described above.

6). The medical image processing apparatus as described above, wherein in the rotation mode, enlargement or reduction is performed without enlarging or reducing the three-dimensional image.

7). A housing (301a),
The portable medical image processing apparatus according to the above, wherein at least the display and the control unit are integrally incorporated in the housing.

8). The medical image processing apparatus according to the above, wherein the motion sensor includes one or more cameras.

9. The medical image processing apparatus according to the above, wherein the three-dimensional medical image includes at least liver and blood vessel image data.

10. The medical image processing apparatus according to the above, wherein the voice input device is a microphone.

11. On the computer,
a: processing for displaying a three-dimensional medical image on a display;
b: processing for recognizing a voice input using the voice recognition device (170) and switching a mode relating to the display of the three-dimensional medical image accordingly;
c: processing for recognizing an operator's motion input via a motion sensor and changing the display of the three-dimensional medical image accordingly;
A medical image processing program.

12. As the mode related to the display of the three-dimensional medical image,
Zoom mode, to enlarge and reduce the image
Pan mode to translate the image,
Rotation mode to rotate the image,
The program as described above, having at least one of the following.

13. In the zoom mode, when the operator's hand is moved in the first direction, the three-dimensional medical image is enlarged, and when the operator's hand is moved in the opposite second direction, the three-dimensional medical image is reduced. The program as described above, which causes processing to be performed.

14 The program according to the above, wherein in the zoom mode, the computer performs processing so that enlargement or reduction is performed without rotating the three-dimensional image.

15. The program according to the above, wherein in the rotation mode, the computer performs processing so that the three-dimensional medical image is rotated in accordance with the movement of the hand of the operator.

16. The program according to the above, wherein in the rotation mode, the computer performs processing so that the three-dimensional image is enlarged or reduced without being enlarged or reduced.

17. A computer displaying a three-dimensional medical image on a display;
A computer recognizing an input voice using a voice recognition device (170) and switching a mode relating to display of the three-dimensional medical image accordingly;
A computer recognizing an operator's motion input via a motion sensor and changing the display of the three-dimensional medical image accordingly;
A method for operating a medical image processing apparatus, comprising:

18. As the mode relating to the display of a three-dimensional medical image,
Zoom mode, to enlarge and reduce the image
Pan mode to translate the image,
Rotation mode to rotate the image,
The method as described above, comprising at least one of

19. In the zoom mode, when the operator's hand is moved in the first direction, the three-dimensional medical image is enlarged, and when the operator's hand is moved in the opposite second direction, the three-dimensional medical image is reduced. The operating method described above.

20. The operation method described above, wherein in the zoom mode, enlargement or reduction is performed without rotating the three-dimensional image.

21. The operation method according to the above, wherein in the rotation mode, the three-dimensional medical image is rotated in response to the movement of the hand of the operator.

DESCRIPTION OF SYMBOLS 1 Imaging device 10 Chemical solution injection apparatus 71,371 Liver 73,75,375 Blood vessel 301 Medical image processing apparatus 301a Case 350 Control part 351 Speech recognition part 353 Gesture recognition part 355a Image display part 355b Operation determination part 355c Display processing part 359 Storage Part 360 touch panel display 361 display 363 touch panel 365 input device 367 communication part 368 interface 369 slot 370 microphone 380 motion sensor

Claims (12)

1. Display,
   A control unit connected to the display;
   An audio input device;
   A motion sensor,
   A medical image processing apparatus comprising:
   The controller is
   a: an image display unit for displaying a three-dimensional medical image on the display;
   b: a mode selection unit for recognizing a voice input using the voice recognition device and switching a mode related to display of the three-dimensional medical image in accordance with the voice;
   c: a display processing unit for recognizing an operator's motion input via the motion sensor and changing the display of the three-dimensional medical image accordingly;
   Having
   Medical image processing apparatus.
2. As the mode relating to the display of a three-dimensional medical image,
   Zoom mode, to enlarge and reduce the image
   Pan mode to translate the image,
   Rotation mode to rotate the image,
   The medical image processing apparatus according to claim 1, further comprising at least one of the following:
3. In the zoom mode, when the operator's hand is moved in the first direction, the three-dimensional medical image is enlarged, and when the operator's hand is moved in the opposite second direction, the three-dimensional medical image is reduced.
   The medical image processing apparatus according to claim 2.
4. The medical image processing apparatus according to claim 2 or 3, wherein in the zoom mode, enlargement or reduction is performed without rotating the three-dimensional image.
5. In the rotation mode, the three-dimensional medical image rotates in response to the movement of the operator's hand.
   The medical image processing apparatus according to claim 2.
6. The medical image processing apparatus according to claim 5, wherein in the rotation mode, enlargement or reduction is performed without enlarging or reducing the three-dimensional image.
7. With a housing,
   The medical image processing apparatus according to any one of claims 1 to 6, wherein at least the display and the control unit are integrally incorporated in the housing.
8. The medical image processing apparatus according to any one of claims 1 to 7, wherein the motion sensor includes one or more cameras.
9. The medical image processing apparatus according to any one of claims 1 to 8, wherein the three-dimensional medical image includes at least liver and blood vessel image data.
10. 10. The medical image processing apparatus according to claim 1, wherein the voice input device is a microphone.
11. On the computer,
    a: processing for displaying a three-dimensional medical image on a display;
    b: processing for recognizing a voice input using a voice recognition device and switching a mode relating to display of the three-dimensional medical image accordingly;
    c: processing for recognizing an operator's motion input via a motion sensor and changing the display of the three-dimensional medical image accordingly;
    A medical image processing program.
12. A computer displaying a three-dimensional medical image on a display;
    A computer recognizing speech input using a speech recognition device and switching a mode relating to display of the three-dimensional medical image accordingly;
    A computer recognizing an operator's motion input via a motion sensor and changing the display of the three-dimensional medical image accordingly;
    A method for operating a medical image processing apparatus, comprising:

Images