JP2013123227A - Image processing system, device, method, and medical image diagnostic device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system, device, method and a medical image diagnostic device for enabling an observer to perform various types of operations on a stereoscopic image by sensory operation.SOLUTION: An image processing system according to an embodiment includes a stereoscopic display device, a determining unit, and a rendering processing unit. The stereoscopic display device displays a stereoscopic image that is capable of being viewed stereoscopically using parallax images generated from volume data as three-dimensional medical image data. The determining unit identifies positional variation of a predetermined moving substance in a stereoscopic image space as a space in which the stereoscopic image is displayed by the stereoscopic display device from positional variation of the moving substance in a real space in which a coordinate system of the stereoscopic image space is present, and determines an operation content on the stereoscopic image on the basis of the identified positional variation. The rendering processing unit performs rendering processing on the volume data in accordance with the operation content determined by the determining unit to generate parallax images newly.

Description

  Embodiments described herein relate generally to an image processing system, apparatus, method, and medical image diagnostic apparatus.

  2. Description of the Related Art Conventionally, a technique for displaying a stereoscopically visible image for a user using a dedicated device such as stereoscopic glasses by displaying two images taken from two viewpoints on a monitor is known. In recent years, an image captured from a plurality of viewpoints (for example, nine images) is displayed on a monitor using a light controller such as a lenticular lens, so that an image that can be stereoscopically viewed by a naked-eye user can be obtained. A technique for displaying is known. Note that a plurality of images displayed on a stereoscopically visible monitor may be generated by estimating depth information of an image taken from one viewpoint and performing image processing using the estimated information.

  On the other hand, a medical image diagnostic apparatus such as an X-ray CT (Computed Tomography) apparatus, an MRI (Magnetic Resonance Imaging) apparatus, or an ultrasonic diagnostic apparatus is practically capable of generating three-dimensional medical image data (hereinafter referred to as volume data). It has become. Such a medical image diagnostic apparatus generates a planar image for display by executing various image processing on the volume data, and displays it on a general-purpose monitor. For example, the medical image diagnostic apparatus generates a two-dimensional rendering image in which three-dimensional information about the subject is reflected by performing volume rendering processing on the volume data, and the generated rendering image is displayed on the general-purpose monitor. Display above.

JP 2005-84414 A

  The problem to be solved by the present invention is to provide an image processing system, an apparatus, a method, and a medical image diagnostic apparatus that allow an observer to perform various operations with a sensible operation on a stereoscopically viewable image. is there.

  The image processing system according to the embodiment includes a stereoscopic display device, a determination unit, and a rendering processing unit. The stereoscopic display device displays a stereoscopically viewable stereoscopic image using a parallax image group generated from volume data that is three-dimensional medical image data. The determination unit is configured to change a position of the moving object in the three-dimensional image space from a position change of the predetermined moving object in a real space where a coordinate system of the three-dimensional image space, which is a space in which the three-dimensional image is displayed by the three-dimensional display device. And the operation content for the stereoscopic image is determined based on the identified position variation. The rendering processing unit newly generates a parallax image group by performing a rendering process on the volume data in accordance with the operation content determined by the determination unit.

FIG. 1 is a diagram for explaining a configuration example of an image processing system according to the first embodiment. FIG. 2 is a diagram for explaining an example of a stereoscopic display monitor that performs stereoscopic display using two parallax images. FIG. 3 is a diagram for explaining an example of a stereoscopic display monitor that performs stereoscopic display with nine parallax images. FIG. 4 is a diagram for explaining a configuration example of the workstation according to the first embodiment. FIG. 5 is a diagram for explaining a configuration example of the rendering processing unit shown in FIG. FIG. 6 is a diagram for explaining an example of volume rendering processing according to the first embodiment. FIG. 7 is a diagram for explaining an example of processing performed by the image processing system according to the first embodiment. FIG. 8 is a diagram for explaining a configuration example of a control unit in the first embodiment. FIG. 9 is a diagram illustrating an example of a correspondence relationship between the stereoscopic image space and the volume data space. FIG. 10 is a diagram for explaining an example of processing by the control unit in the first embodiment. FIG. 11 is a diagram for explaining an example of processing by the control unit in the first embodiment. FIG. 12 is a flowchart illustrating an example of the flow of processing by the workstation in the first embodiment. FIG. 13 is a diagram for explaining a modification of the first embodiment. FIG. 14 is a diagram for explaining a modification of the first embodiment. FIG. 15 is a diagram for explaining a modification of the first embodiment.

  Hereinafter, embodiments of an image processing system, an apparatus, a method, and a medical image diagnostic apparatus will be described in detail with reference to the accompanying drawings. In the following, an image processing system including a workstation having a function as an image processing apparatus will be described as an embodiment. Here, the terms used in the following embodiments will be described. The “parallax image group” is an image generated by performing volume rendering processing by moving the viewpoint position by a predetermined parallax angle with respect to volume data. It is a group. That is, the “parallax image group” includes a plurality of “parallax images” having different “viewpoint positions”. The “parallax angle” is a predetermined position in the space represented by the volume data and an adjacent viewpoint position among the viewpoint positions set to generate the “parallax image group” (for example, the center of the space) It is an angle determined by. The “parallax number” is the number of “parallax images” necessary for stereoscopic viewing on the stereoscopic display monitor. The “9 parallax images” described below is a “parallax image group” composed of nine “parallax images”. The “two-parallax image” described below is a “parallax image group” composed of two “parallax images”.

(First embodiment)
First, a configuration example of the image processing system according to the first embodiment will be described. FIG. 1 is a diagram for explaining a configuration example of an image processing system according to the first embodiment.

  As shown in FIG. 1, the image processing system 1 according to the first embodiment includes a medical image diagnostic apparatus 110, an image storage apparatus 120, a workstation 130, and a terminal apparatus 140. Each apparatus illustrated in FIG. 1 is in a state where it can communicate with each other directly or indirectly by, for example, an in-hospital LAN (Local Area Network) 2 installed in a hospital. For example, when a PACS (Picture Archiving and Communication System) is introduced into the image processing system 1, each apparatus transmits and receives medical images and the like according to DICOM (Digital Imaging and Communications in Medicine) standards.

  The image processing system 1 generates a parallax image group from volume data that is three-dimensional medical image data generated by the medical image diagnostic apparatus 110, and displays the parallax image group on a stereoscopically viewable monitor. A stereoscopic image, which is an image that can be visually recognized by the observer, is provided to an observer such as a doctor or a laboratory technician working in the hospital. Specifically, in the first embodiment, the workstation 130 performs various image processes on the volume data to generate a parallax image group. In addition, the workstation 130 and the terminal device 140 have a stereoscopically visible monitor, and display a stereoscopic image to the user by displaying a parallax image group generated by the workstation 130 on the monitor. Further, the image storage device 120 stores the volume data generated by the medical image diagnostic device 110 and the parallax image group generated by the workstation 130. For example, the workstation 130 or the terminal device 140 acquires volume data or a parallax image group from the image storage device 120, executes arbitrary image processing on the acquired volume data or parallax image group, or selects a parallax image group. Or display on a monitor. Hereinafter, each device will be described in order.

  The medical image diagnostic apparatus 110 includes an X-ray diagnostic apparatus, an X-ray CT (Computed Tomography) apparatus, an MRI (Magnetic Resonance Imaging) apparatus, an ultrasonic diagnostic apparatus, a SPECT (Single Photon Emission Computed Tomography) apparatus, and a PET (Positron Emission computed Tomography). ) Apparatus, a SPECT-CT apparatus in which a SPECT apparatus and an X-ray CT apparatus are integrated, a PET-CT apparatus in which a PET apparatus and an X-ray CT apparatus are integrated, or a group of these apparatuses. Further, the medical image diagnostic apparatus 110 according to the first embodiment can generate three-dimensional medical image data (volume data).

  Specifically, the medical image diagnostic apparatus 110 according to the first embodiment generates volume data by imaging a subject. For example, the medical image diagnostic apparatus 110 collects data such as projection data and MR signals by imaging the subject, and medical image data of a plurality of axial surfaces along the body axis direction of the subject from the collected data. By reconfiguring, volume data is generated. For example, when the medical image diagnostic apparatus 110 reconstructs 500 pieces of medical image data on the axial plane, the 500 pieces of medical image data groups on the axial plane become volume data. In addition, the projection data of the subject imaged by the medical image diagnostic apparatus 110, the MR signal, or the like may be used as volume data.

  Further, the medical image diagnostic apparatus 110 according to the first embodiment transmits the generated volume data to the image storage apparatus 120. The medical image diagnostic apparatus 110 identifies, for example, a patient ID for identifying a patient, an examination ID for identifying an examination, and the medical image diagnostic apparatus 110 as supplementary information when transmitting volume data to the image storage apparatus 120. A device ID, a series ID for identifying one shot by the medical image diagnostic device 110, and the like are transmitted.

  The image storage device 120 is a database that stores medical images. Specifically, the image storage device 120 according to the first embodiment receives volume data from the medical image diagnostic device 110 and stores the received volume data in a predetermined storage unit. In the first embodiment, the workstation 130 generates a parallax image group from the volume data, and transmits the generated parallax image group to the image storage device 120. For this reason, the image storage device 120 stores the parallax image group transmitted from the workstation 130 in a predetermined storage unit. In the present embodiment, the workstation 130 illustrated in FIG. 1 and the image storage device 120 may be integrated by using the workstation 130 that can store a large-capacity image. That is, this embodiment may be a case where volume data or a parallax image group is stored in the workstation 130 itself.

  In the first embodiment, the volume data and the parallax image group stored in the image storage device 120 are stored in association with the patient ID, examination ID, device ID, series ID, and the like. For this reason, the workstation 130 and the terminal device 140 acquire necessary volume data and a parallax image group from the image storage device 120 by performing a search using a patient ID, an examination ID, a device ID, a series ID, and the like.

  The workstation 130 is an image processing apparatus that performs image processing on medical images. Specifically, the workstation 130 according to the first embodiment generates a parallax image group by performing various rendering processes on the volume data acquired from the image storage device 120.

  Further, the workstation 130 according to the first embodiment includes a monitor (also referred to as a stereoscopic display monitor or a stereoscopic image display device) capable of displaying a stereoscopic image as a display unit. The workstation 130 generates a parallax image group and displays the generated parallax image group on the stereoscopic display monitor. As a result, the operator of the workstation 130 can perform an operation for generating a parallax image group while confirming the stereoscopically viewable stereoscopic image displayed on the stereoscopic display monitor.

  Further, the workstation 130 transmits the generated parallax image group to the image storage device 120 and the terminal device 140. In addition, when transmitting the parallax image group to the image storage device 120 or the terminal device 140, the workstation 130 transmits, for example, a patient ID, an examination ID, a device ID, a series ID, and the like as supplementary information. The incidental information transmitted when transmitting the parallax image group to the image storage device 120 includes incidental information regarding the parallax image group. The incidental information regarding the parallax image group includes the number of parallax images (for example, “9”), the resolution of the parallax images (for example, “466 × 350 pixels”), and the like.

  The terminal device 140 is a device that allows a doctor or laboratory technician working in a hospital to view a medical image. For example, the terminal device 140 is a PC (Personal Computer), a tablet PC, a PDA (Personal Digital Assistant), a mobile phone, or the like operated by a doctor or laboratory technician working in a hospital. Specifically, the terminal device 140 according to the first embodiment includes a stereoscopic display monitor as a display unit. In addition, the terminal device 140 acquires a parallax image group from the image storage device 120 and displays the acquired parallax image group on the stereoscopic display monitor. As a result, a doctor or laboratory technician who is an observer can view a medical image that can be viewed stereoscopically. Note that the terminal device 140 may be any information processing terminal connected to a stereoscopic display monitor as an external device.

  Here, the stereoscopic display monitor included in the workstation 130 and the terminal device 140 will be described. A general-purpose monitor that is most popular at present displays a two-dimensional image in two dimensions, and cannot display a two-dimensional image in three dimensions. If an observer requests stereoscopic viewing on a general-purpose monitor, an apparatus that outputs an image to the general-purpose monitor needs to display two parallax images that can be viewed stereoscopically by the observer in parallel by the parallel method or the intersection method. is there. Alternatively, an apparatus that outputs an image to a general-purpose monitor, for example, uses an after-color method with an eyeglass that has a red cellophane attached to the left eye portion and a blue cellophane attached to the right eye portion. It is necessary to display a stereoscopically viewable image.

  On the other hand, as a stereoscopic display monitor, there is a stereoscopic display monitor that enables a stereoscopic view of a two-parallax image (also referred to as a binocular parallax image) by using dedicated equipment such as stereoscopic glasses.

  FIG. 2 is a diagram for explaining an example of a stereoscopic display monitor that performs stereoscopic display using two parallax images. An example shown in FIG. 2 is a stereoscopic display monitor that performs stereoscopic display by a shutter method, and shutter glasses are used as stereoscopic glasses worn by an observer who observes the monitor. Such a stereoscopic display monitor emits two parallax images alternately on the monitor. For example, the monitor shown in FIG. 2A alternately emits a left-eye image and a right-eye image at 120 Hz. Here, as shown in FIG. 2A, the monitor is provided with an infrared emitting unit, and the infrared emitting unit controls the emission of infrared rays in accordance with the timing at which the image is switched.

  Moreover, the infrared rays emitted from the infrared ray emitting portion are received by the infrared ray receiving portion of the shutter glasses shown in FIG. A shutter is attached to each of the left and right frames of the shutter glasses, and the shutter glasses alternately switch the transmission state and the light shielding state of the left and right shutters according to the timing when the infrared light receiving unit receives the infrared rays. Hereinafter, the switching process between the transmission state and the light shielding state in the shutter will be described.

  As shown in FIG. 2B, each shutter has an incident-side polarizing plate and an output-side polarizing plate, and a liquid crystal layer between the incident-side polarizing plate and the output-side polarizing plate. Have Further, as shown in FIG. 2B, the incident-side polarizing plate and the outgoing-side polarizing plate are orthogonal to each other. Here, as shown in FIG. 2B, in the “OFF” state where no voltage is applied, the light that has passed through the polarizing plate on the incident side is rotated by 90 ° by the action of the liquid crystal layer, and is emitted on the outgoing side. Is transmitted through the polarizing plate. That is, a shutter to which no voltage is applied is in a transmissive state.

  On the other hand, as shown in FIG. 2B, in the “ON” state where a voltage is applied, the polarization rotation action caused by the liquid crystal molecules in the liquid crystal layer disappears. It will be blocked by the polarizing plate on the exit side. That is, the shutter to which the voltage is applied is in a light shielding state.

  Therefore, for example, the infrared emitting unit emits infrared rays during a period in which an image for the left eye is displayed on the monitor. The infrared light receiving unit applies a voltage to the right-eye shutter without applying a voltage to the left-eye shutter during a period of receiving the infrared light. Accordingly, as shown in FIG. 2A, the right-eye shutter is in a light-shielding state and the left-eye shutter is in a transmissive state, so that an image for the left eye is incident on the left eye of the observer. On the other hand, the infrared ray emitting unit stops emitting infrared rays while the right-eye image is displayed on the monitor. The infrared light receiving unit applies a voltage to the left-eye shutter without applying a voltage to the right-eye shutter during a period in which no infrared light is received. Accordingly, the left-eye shutter is in a light-shielding state and the right-eye shutter is in a transmissive state, so that an image for the right eye is incident on the right eye of the observer. As described above, the stereoscopic display monitor illustrated in FIG. 2 displays an image that can be viewed stereoscopically by the observer by switching the image displayed on the monitor and the state of the shutter in conjunction with each other. As a stereoscopic display monitor capable of stereoscopically viewing a two-parallax image, a monitor adopting a polarized glasses method is also known in addition to the shutter method described above.

  Furthermore, as a stereoscopic display monitor that has been put into practical use in recent years, there is a stereoscopic display monitor that allows a viewer to stereoscopically view a multi-parallax image such as a 9-parallax image with the naked eye by using a light controller such as a lenticular lens. . Such a stereoscopic display monitor enables stereoscopic viewing based on binocular parallax, and also enables stereoscopic viewing based on motion parallax that also changes the image observed in accordance with the viewpoint movement of the observer.

  FIG. 3 is a diagram for explaining an example of a stereoscopic display monitor that performs stereoscopic display with nine parallax images. In the stereoscopic display monitor shown in FIG. 3, a light beam controller is arranged on the front surface of a flat display surface 200 such as a liquid crystal panel. For example, in the stereoscopic display monitor shown in FIG. 3, a vertical lenticular sheet 201 whose optical aperture extends in the vertical direction is attached to the front surface of the display surface 200 as a light beam controller. In the example shown in FIG. 3, the vertical lenticular sheet 201 is pasted so that the convex portion of the vertical lenticular sheet 201 becomes the front surface, but the convex portion of the vertical lenticular sheet 201 is pasted so as to face the display surface 200. There may be.

  As shown in FIG. 3, the display surface 200 has an aspect ratio of 3: 1 and pixels in which three sub-pixels, red (R), green (G), and blue (B), are arranged in the vertical direction. 202 are arranged in a matrix. The stereoscopic display monitor shown in FIG. 3 converts a nine-parallax image composed of nine images into an intermediate image arranged in a predetermined format (for example, a lattice shape), and then outputs it to the display surface 200. That is, the stereoscopic display monitor shown in FIG. 3 assigns and outputs nine pixels at the same position in nine parallax images to nine columns of pixels 202. The nine columns of pixels 202 constitute a unit pixel group 203 that simultaneously displays nine images with different viewpoint positions.

  The nine parallax images simultaneously output as the unit pixel group 203 on the display surface 200 are emitted as parallel light by, for example, an LED (Light Emitting Diode) backlight, and further emitted in multiple directions by the vertical lenticular sheet 201. As the light of each pixel of the nine-parallax image is emitted in multiple directions, the light incident on the right eye and the left eye of the observer changes in conjunction with the position of the observer (viewpoint position). That is, the parallax angle between the parallax image incident on the right eye and the parallax image incident on the left eye differs depending on the viewing angle of the observer. Thereby, the observer can visually recognize the photographing object in three dimensions at each of the nine positions shown in FIG. 3, for example. In addition, for example, the observer can view the image three-dimensionally in a state of facing the object to be imaged at the position “5” shown in FIG. 3, and at each position other than “5” shown in FIG. It can be visually recognized in a three-dimensional manner with the direction of the object changed. Note that the stereoscopic display monitor shown in FIG. 3 is merely an example. As shown in FIG. 3, the stereoscopic display monitor that displays the nine-parallax image may be a horizontal stripe liquid crystal of “RRR..., GGG..., BBB. .. ”” May be used. Further, the stereoscopic display monitor shown in FIG. 3 may be a vertical lens system in which the lenticular sheet is vertical as shown in FIG. 3 or a diagonal lens system in which the lenticular sheet is oblique. There may be.

  So far, the configuration example of the image processing system 1 according to the first embodiment has been briefly described. Note that the application of the image processing system 1 described above is not limited when PACS is introduced. For example, the image processing system 1 is similarly applied when an electronic medical chart system that manages an electronic medical chart to which a medical image is attached is introduced. In this case, the image storage device 120 is a database that stores electronic medical records. Further, for example, the image processing system 1 is similarly applied when a HIS (Hospital Information System) and a RIS (Radiology Information System) are introduced. The image processing system 1 is not limited to the configuration example described above. The functions and sharing of each device may be appropriately changed according to the operation mode.

  Next, a configuration example of the workstation according to the first embodiment will be described with reference to FIG. FIG. 4 is a diagram for explaining a configuration example of the workstation according to the first embodiment. In the following description, the “parallax image group” refers to a group of stereoscopic images generated by performing volume rendering processing on volume data. Further, the “parallax image” is an individual image constituting the “parallax image group”. That is, the “parallax image group” includes a plurality of “parallax images” having different viewpoint positions.

  The workstation 130 according to the first embodiment is a high-performance computer suitable for image processing and the like, and as illustrated in FIG. 4, an input unit 131, a display unit 132, a communication unit 133, and a storage unit 134. And a control unit 135 and a rendering processing unit 136. In the following description, the workstation 130 is a high-performance computer suitable for image processing or the like. However, the present invention is not limited to this, and may be any information processing apparatus. For example, any personal computer may be used.

  The input unit 131 is a mouse, a keyboard, a trackball, or the like, and receives input of various operations on the workstation 130 from an operator. Specifically, the input unit 131 according to the first embodiment receives input of information for acquiring volume data to be subjected to rendering processing from the image storage device 120. For example, the input unit 131 receives input of a patient ID, an examination ID, a device ID, a series ID, and the like. Further, the input unit 131 according to the first embodiment receives an input of a condition regarding rendering processing (hereinafter, rendering condition).

  The display unit 132 is a liquid crystal panel or the like as a stereoscopic display monitor, and displays various information. Specifically, the display unit 132 according to the first embodiment displays a GUI (Graphical User Interface) for receiving various operations from the operator, a parallax image group, and the like. The communication unit 133 is a NIC (Network Interface Card) or the like, and performs communication with other devices.

  The storage unit 134 is a hard disk, a semiconductor memory element, or the like, and stores various types of information. Specifically, the storage unit 134 according to the first embodiment stores volume data acquired from the image storage device 120 via the communication unit 133. In addition, the storage unit 134 according to the first embodiment stores volume data during rendering processing, a group of parallax images generated by the rendering processing, and the like.

  The control unit 135 is an electronic circuit such as a CPU (Central Processing Unit), MPU (Micro Processing Unit) or GPU (Graphics Processing Unit), or an integrated circuit such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). Yes, the entire control of the workstation 130 is performed.

  For example, the control unit 135 according to the first embodiment controls the display of the GUI and the display of the parallax image group on the display unit 132. For example, the control unit 135 controls transmission / reception of volume data and a parallax image group performed with the image storage device 120 via the communication unit 133. For example, the control unit 135 controls the rendering process performed by the rendering processing unit 136. For example, the control unit 135 controls reading of volume data from the storage unit 134 and storage of the parallax image group in the storage unit 134.

  The rendering processing unit 136 performs various rendering processes on the volume data acquired from the image storage device 120 under the control of the control unit 135, and generates a parallax image group. Specifically, the rendering processing unit 136 according to the first embodiment reads volume data from the storage unit 134 and first performs preprocessing on the volume data. Next, the rendering processing unit 136 performs a volume rendering process on the pre-processed volume data to generate a parallax image group. Subsequently, the rendering processing unit 136 generates a two-dimensional image in which various kinds of information (scale, patient name, examination item, etc.) are drawn, and superimposes it on each of the parallax image groups, thereby outputting 2 for output. Generate a dimensional image. Then, the rendering processing unit 136 stores the generated parallax image group and the output two-dimensional image in the storage unit 134. In the first embodiment, rendering processing refers to the entire image processing performed on volume data. Volume rendering processing refers to a two-dimensional image reflecting three-dimensional information in the rendering processing. It is a process to generate. For example, a parallax image corresponds to the medical image generated by the rendering process.

  FIG. 5 is a diagram for explaining a configuration example of the rendering processing unit shown in FIG. As illustrated in FIG. 5, the rendering processing unit 136 includes a preprocessing unit 1361, a 3D image processing unit 1362, and a 2D image processing unit 1363. The preprocessing unit 1361 performs preprocessing on the volume data, the 3D image processing unit 1362 generates a parallax image group from the preprocessed volume data, and the 2D image processing unit 1363 stores various information on the parallax image group. A two-dimensional image for output on which is superimposed is generated. Hereinafter, each part is demonstrated in order.

  The preprocessing unit 1361 is a processing unit that performs various types of preprocessing when rendering processing is performed on volume data, and includes an image correction processing unit 1361a, a three-dimensional object fusion unit 1361e, and a three-dimensional object display area setting. Part 1361f.

  The image correction processing unit 1361a is a processing unit that performs image correction processing when processing two types of volume data as one volume data, and as illustrated in FIG. 5, a distortion correction processing unit 1361b, a body motion correction processing unit, 1361c and an inter-image registration processing unit 1361d. For example, the image correction processing unit 1361a performs image correction processing when processing volume data of a PET image generated by a PET-CT apparatus and volume data of an X-ray CT image as one volume data. Alternatively, the image correction processing unit 1361a performs image correction processing when processing the volume data of the T1-weighted image and the volume data of the T2-weighted image generated by the MRI apparatus as one volume data.

  In addition, the distortion correction processing unit 1361b corrects the data distortion caused by the collection conditions at the time of data collection by the medical image diagnostic apparatus 110 in each volume data. Further, the body motion correction processing unit 1361c corrects the movement caused by the body motion of the subject at the time of collecting the data used for generating the individual volume data. Further, the inter-image registration processing unit 1361d performs registration (Registration) using, for example, a cross-correlation method between the two volume data subjected to the correction processing by the distortion correction processing unit 1361b and the body motion correction processing unit 1361c. )I do.

  The three-dimensional object fusion unit 1361e fuses a plurality of volume data that has been aligned by the inter-image registration processing unit 1361d. Note that the processing of the image correction processing unit 1361a and the three-dimensional object fusion unit 1361e is omitted when rendering processing is performed on single volume data.

  The three-dimensional object display region setting unit 1361f is a processing unit that sets a display region corresponding to the display target organ designated by the operator, and includes a segmentation processing unit 1361g. The segmentation processing unit 1361g is a processing unit that extracts organs such as the heart, lungs, and blood vessels designated by the operator by, for example, a region expansion method based on pixel values (voxel values) of volume data.

  Note that the segmentation processing unit 1361g does not perform the segmentation process when the display target organ is not designated by the operator. The segmentation processing unit 1361g extracts a plurality of corresponding organs when a plurality of display target organs are designated by the operator. Further, the processing of the segmentation processing unit 1361g may be executed again in response to an operator fine adjustment request referring to the rendered image.

  The three-dimensional image processing unit 1362 performs a volume rendering process on the pre-processed volume data processed by the pre-processing unit 1361. As a processing unit that performs volume rendering processing, a 3D image processing unit 1362 includes a projection method setting unit 1362a, a 3D geometric transformation processing unit 1362b, a 3D object appearance processing unit 1362f, and a 3D virtual space rendering unit 1362k. Have

  The projection method setting unit 1362a determines a projection method for generating a parallax image group. For example, the projection method setting unit 1362a determines whether to execute the volume rendering process by the parallel projection method or the perspective projection method.

  The three-dimensional geometric transformation processing unit 1362b is a processing unit that determines information for transforming volume data on which volume rendering processing is performed into a three-dimensional geometrical structure. A reduction processing unit 1362e is included. The translation processing unit 1362c is a processing unit that determines the amount of movement to translate the volume data when the viewpoint position when performing the volume rendering processing is translated, and the rotation processing unit 1362d performs the volume rendering processing. This is a processing unit that determines the amount of movement to rotate the volume data when the viewpoint position at the time of rotation is rotated. The enlargement / reduction processing unit 1362e is a processing unit that determines the enlargement rate or reduction rate of the volume data when enlargement or reduction of the parallax image group is requested.

  The three-dimensional object appearance processing unit 1362f includes a three-dimensional object color processing unit 1362g, a three-dimensional object opacity processing unit 1362h, a three-dimensional object material processing unit 1362i, and a three-dimensional virtual space light source processing unit 1362j. The three-dimensional object appearance processing unit 1362f performs a process of determining the display state of the displayed parallax image group in response to an operator's request, for example.

  The three-dimensional object color processing unit 1362g is a processing unit that determines a color to be colored for each region segmented by the volume data. The three-dimensional object opacity processing unit 1362h is a processing unit that determines the opacity (Opacity) of each voxel constituting each region segmented by volume data. It should be noted that the area behind the area having the opacity of “100%” in the volume data is not drawn in the parallax image group. In addition, an area in which the opacity is “0%” in the volume data is not drawn in the parallax image group.

  The three-dimensional object material processing unit 1362i is a processing unit that determines the material of each region segmented by volume data and adjusts the texture when the region is rendered. The three-dimensional virtual space light source processing unit 1362j is a processing unit that determines the position of the virtual light source installed in the three-dimensional virtual space and the type of the virtual light source when performing volume rendering processing on the volume data. Examples of the virtual light source include a light source that emits parallel rays from infinity, a light source that emits radial rays from a viewpoint, and the like.

  The three-dimensional virtual space rendering unit 1362k performs volume rendering processing on the volume data to generate a parallax image group. The three-dimensional virtual space rendering unit 1362k performs various types of information determined by the projection method setting unit 1362a, the three-dimensional geometric transformation processing unit 1362b, and the three-dimensional object appearance processing unit 1362f as necessary when performing the volume rendering process. Is used.

  Here, the volume rendering process by the three-dimensional virtual space rendering unit 1362k is performed according to the rendering conditions. For example, the rendering condition is “parallel projection method” or “perspective projection method”. For example, the rendering condition is “reference viewpoint position, parallax angle, and number of parallaxes”. Further, for example, the rendering conditions are “translation of viewpoint position”, “rotational movement of viewpoint position”, “enlargement of parallax image group”, and “reduction of parallax image group”. Further, for example, the rendering conditions are “color to be colored”, “transparency”, “texture”, “position of virtual light source”, and “type of virtual light source”. Such a rendering condition may be accepted from the operator via the input unit 131 or may be initially set. In any case, the three-dimensional virtual space rendering unit 1362k receives a rendering condition from the control unit 135, and performs volume rendering processing on the volume data according to the rendering condition. At this time, the projection method setting unit 1362a, the three-dimensional geometric transformation processing unit 1362b, and the three-dimensional object appearance processing unit 1362f determine various pieces of necessary information according to the rendering conditions, so the three-dimensional virtual space rendering unit 1362k. Generates a parallax image group using the determined various pieces of information.

  FIG. 6 is a diagram for explaining an example of volume rendering processing according to the first embodiment. For example, as shown in “9 parallax image generation method (1)” in FIG. 6, the three-dimensional virtual space rendering unit 1362k accepts the parallel projection method as a rendering condition, and further, the reference viewpoint position (5) and the parallax. Assume that the angle “1 degree” is received. In such a case, the three-dimensional virtual space rendering unit 1362k translates the position of the viewpoint from (1) to (9) so that the parallax angle is "1 degree", and the parallax angle (line of sight) is obtained by the parallel projection method. Nine parallax images with different degrees between directions are generated. When performing the parallel projection method, the three-dimensional virtual space rendering unit 1362k sets a light source that emits parallel rays from infinity along the line-of-sight direction.

  Alternatively, as shown in “9-parallax image generation method (2)” in FIG. 6, the three-dimensional virtual space rendering unit 1362k accepts a perspective projection method as a rendering condition, and further, the reference viewpoint position (5) and the parallax Assume that the angle “1 degree” is received. In such a case, the three-dimensional virtual space rendering unit 1362k rotates and moves the viewpoint position from (1) to (9) so that the parallax angle is every "1 degree" around the center (center of gravity) of the volume data. Thus, nine parallax images having different parallax angles by 1 degree are generated by the perspective projection method. When performing the perspective projection method, the three-dimensional virtual space rendering unit 1362k sets a point light source or a surface light source that radiates light three-dimensionally radially around the viewing direction at each viewpoint. Further, when the perspective projection method is performed, the viewpoints (1) to (9) may be translated depending on the rendering condition.

  Note that the three-dimensional virtual space rendering unit 1362k radiates light two-dimensionally radially around the line-of-sight direction with respect to the vertical direction of the displayed volume rendered image, and the horizontal direction of the displayed volume rendered image. On the other hand, volume rendering processing using both the parallel projection method and the perspective projection method may be performed by setting a light source that emits parallel light rays from infinity along the line-of-sight direction.

  The nine parallax images generated in this way are a parallax image group. In the first embodiment, the nine parallax images are converted into intermediate images arranged in a predetermined format (for example, a lattice shape) by the control unit 135, for example, and are output to the display unit 132 as a stereoscopic display monitor. Then, the operator of the workstation 130 can perform an operation for generating a parallax image group while confirming a stereoscopically viewable medical image displayed on the stereoscopic display monitor.

  In the example of FIG. 6, the case where the projection method, the reference viewpoint position, and the parallax angle are received as the rendering conditions has been described. However, the three-dimensional virtual space is similarly applied when other conditions are received as the rendering conditions. The rendering unit 1362k generates a parallax image group while reflecting each rendering condition.

  The three-dimensional virtual space rendering unit 1362k has not only volume rendering but also a function of reconstructing an MPR image from volume data by performing a cross-section reconstruction method (MPR: Multi Planer Reconstruction). The three-dimensional virtual space rendering unit 1362k also has a function of performing “Curved MPR” and a function of performing “Intensity Projection”.

  Subsequently, the parallax image group generated from the volume data by the three-dimensional image processing unit 1362 is set as an underlay. Then, an overlay (Overlay) on which various types of information (scale, patient name, examination item, etc.) are drawn is superimposed on the underlay, thereby obtaining a two-dimensional image for output. The two-dimensional image processing unit 1363 is a processing unit that generates an output two-dimensional image by performing image processing on the overlay and the underlay. As illustrated in FIG. 5, the two-dimensional object drawing unit 1363a, A two-dimensional geometric transformation processing unit 1363b and a luminance adjustment unit 1363c are included. For example, the two-dimensional image processing unit 1363 superimposes one overlay on each of nine parallax images (underlays) in order to reduce the load required to generate a two-dimensional image for output. Thus, nine output two-dimensional images are generated. In the following, an underlay with an overlay superimposed may be simply referred to as a “parallax image”.

  The two-dimensional object drawing unit 1363a is a processing unit that draws various information drawn on the overlay, and the two-dimensional geometric transformation processing unit 1363b performs parallel movement processing or rotational movement processing on the position of the various information drawn on the overlay. Or a processing unit that performs an enlargement process or a reduction process of various types of information drawn on the overlay.

  The luminance adjustment unit 1363c is a processing unit that performs luminance conversion processing. For example, the gradation of an output destination stereoscopic display monitor, an image such as a window width (WW: Window Width), a window level (WL: Window Level), or the like. This is a processing unit that adjusts the brightness of the overlay and the underlay according to the processing parameters.

  For example, the control unit 135 temporarily stores the output two-dimensional image generated in this manner in the storage unit 134 and then transmits the two-dimensional image to the image storage device 120 via the communication unit 133. Then, for example, the terminal device 140 acquires the two-dimensional image for output from the image storage device 120, converts it into an intermediate image arranged in a predetermined format (for example, a lattice shape), and displays it on the stereoscopic display monitor. For example, the control unit 135 temporarily stores the output two-dimensional image in the storage unit 134, and then transmits the output two-dimensional image to the image storage device 120 and the terminal device 140 via the communication unit 133. Then, the terminal device 140 converts the output two-dimensional image received from the workstation 130 into an intermediate image arranged in a predetermined format (for example, a lattice shape) and displays the converted image on the stereoscopic display monitor. Accordingly, a doctor or a laboratory technician who uses the terminal device 140 can browse a stereoscopically viewable medical image in a state where various information (scale, patient name, examination item, etc.) is drawn.

  As described above, the stereoscopic display monitor described above provides a stereoscopic image that can be viewed stereoscopically by the observer by displaying a parallax image group. At this time, the observer may perform various operations on the stereoscopic image using a pointing device such as a mouse. For example, the observer operates the pointing device to move the cursor, set a region of interest (ROI) in the stereoscopic image, and perform an operation to display a cross-sectional image of the set ROI. Or However, a cursor that can be operated with a mouse or the like moves in a three-dimensional space in which a stereoscopic image is displayed (hereinafter sometimes referred to as “stereoscopic image space”). It is difficult to grasp the cursor position in. That is, when a mouse or the like is used, it may be difficult for an observer to perform various operations such as setting a region of interest in a stereoscopic image.

  Therefore, in the first embodiment, it is possible to perform various operations on a stereoscopic image as if an observer directly touched the stereoscopic image with a hand. This point will be briefly described with reference to FIG. FIG. 7 is a diagram for explaining an example of processing by the image processing system 1 in the first embodiment. In the following description, a case where direct operation on a stereoscopic image is realized by the workstation 130 will be described as an example.

  In the example illustrated in FIG. 7, the display unit 132 is a stereoscopic display monitor included in the workstation 130 as described above. As shown in FIG. 7, the display unit 132 in the first embodiment displays a stereoscopic image I11 that shows an organ or the like of a subject, and an observer can perform various operations on the stereoscopic image I11. A frame line indicating the operable area SP10 which is an area is displayed. Thus, the observer can recognize that various operations can be performed on the stereoscopic image I11 in the operable region SP10. Further, as shown in FIG. 7, the display unit 132 is provided with a camera 137. Such a camera 137 is a three-dimensional camera (3D camera) that can three-dimensionally recognize a three-dimensional space, and recognizes a positional variation of the observer's hand U11 within the operable region SP10.

  Under such a configuration, the workstation 130 according to the first embodiment detects a change in the position of the hand U11 by the camera 137 when the hand U11 is located in the operable region SP10. Then, the workstation 130 determines the operation content for the stereoscopic image I11 based on the positional fluctuation of the hand U11 detected by the camera 137. Then, the workstation 130 performs a rendering process on the volume data according to the determined operation content to newly generate a parallax image group, and displays the generated parallax image group on the display unit 132. In this way, the workstation 130 can specify an operation desired by the observer from the position change of the hand in the operable area SP10, and can display a stereoscopic image corresponding to the operation. That is, according to the first embodiment, the observer can perform various operations on the stereoscopic image with a touch feeling directly without using an input unit such as a mouse.

  Hereinafter, the workstation 130 in the first embodiment will be described in detail. First, the control unit 135 included in the workstation 130 according to the first embodiment will be described with reference to FIG. FIG. 8 is a diagram for explaining a configuration example of the control unit 135 in the first embodiment. As illustrated in FIG. 8, the control unit 135 of the workstation 130 includes a determination unit 1351, a rendering control unit 1352, and a display control unit 1353. Hereinafter, after briefly explaining these processing units, a specific example of processing will be described.

  The determination unit 1351 determines the operation content for the stereoscopic image based on the position variation of a predetermined moving object located in the stereoscopic image space in which the stereoscopic image is displayed by the display unit 132. Specifically, when the position variation of the observer's hand U11 in the operable region SP10 is detected by the camera 137, the determination unit 1351 in the first embodiment is based on the position variation of the hand U11. The operation content for the stereoscopic image is determined.

  The processing by the determination unit 1351 will be described more specifically. First, the storage unit 134 of the workstation 130 according to the first embodiment stores the operation (position variation) of the observer's hand U11 in the operable region SP10 and the operation content in association with each other. For example, the storage unit 134 correlates with a predetermined operation (position fluctuation) of the hand U11, the operation content for rotating the stereoscopic image, the operation content for enlarging or reducing the stereoscopic image, and the opacity (Opacity) of the stereoscopic image. ), An operation content for cutting a stereoscopic image, an operation content for deleting a part of the stereoscopic image, and the like are stored. Then, the determination unit 1351 specifies the operation content by acquiring the operation content corresponding to the motion (position variation) of the hand U11 detected by the camera 137 from the storage unit 134.

  It supplements about the process by the camera 137. FIG. The camera 137 in the first embodiment includes a predetermined control circuit, and monitors whether or not there is a moving object in the operable area SP10. Then, when there is a moving object, the camera 137 determines, for example, whether the moving object substantially matches a predetermined shape (for example, a human hand). At this time, when the moving object has a predetermined shape (for example, a human hand), the camera 137 detects the time variation of the position of the moving object in the operable region SP10 to detect the moving object (for example, Detects position fluctuations in human hands).

  Here, the workstation 130 in the first embodiment holds correspondence information that associates the positions of the stereoscopic image space in which the stereoscopic image is displayed and the real space that is the operable area SP10. Specifically, the workstation 130 holds correspondence information indicating in which position in the real space the coordinate system of the stereoscopic image space exists in front of the display unit 132. Based on the correspondence information, the determination unit 1351 identifies the position in the stereoscopic image corresponding to the position of the moving object (for example, a human hand) detected in the real space. The above-described operation content specifying process is performed. The workstation 130 holds different correspondence information according to the display magnification of the display unit 132, the parallax angle that is a rendering condition, and the like.

  Note that the processing by the camera 137 is not limited to this example. For example, the observer wears a member (glove or the like) having a shape that becomes a predetermined mark on his / her hand, and the camera 137 detects the position change of the member that becomes the mark, thereby changing the position of the observer's hand. May be detected.

  The rendering control unit 1352 generates a parallax image group from the volume data in cooperation with the rendering processing unit 136. Specifically, the rendering control unit 1352 according to the first embodiment performs an image of a tool for performing various operations on a stereoscopic image on a parallax image group generated from volume data (an “icon stereoscopic image Ic11 described later). ˜Ic14 ”and the like) is controlled to be superimposed. Furthermore, the rendering control unit 1352 controls the rendering processing unit 136 so that an image such as a frame line indicating the operable region is superimposed on the parallax image group.

  In addition, the rendering control unit 1352 according to the first embodiment performs a rendering process on the volume data from which the stereoscopic image displayed on the display unit 132 is generated in accordance with the operation content determined by the determination unit 1351. The rendering processing unit 136 is controlled as described above. At this time, the rendering control unit 1352 controls the rendering processing unit 136 to perform the rendering process based on the position variation of the hand U11 detected by the camera 137. For example, when the operation content is “3D image cutting”, the rendering control unit 1352 acquires the volume data cutting position from the position change of the hand U11 detected by the camera 137, and the rendering control unit 1352 receives the volume data cutting position based on the acquired cutting position. The rendering processing unit 136 is controlled so as to generate a parallax image group in which a specimen organ or the like is cut.

  That is, the rendering control unit 1352 may describe the space in which the volume data is arranged (hereinafter referred to as “volume data space”) from the coordinates where the hand U11 is located in the stereoscopic image space, such as the cutting position in the above example. Get the coordinates in a). Here, since the coordinate system differs between the stereoscopic image space and the volume data space, the rendering control unit 1352 acquires the coordinates of the volume data space corresponding to the stereoscopic image space using a predetermined coordinate conversion formula.

  Hereinafter, the correspondence between the stereoscopic image space and the volume data space will be described with reference to FIG. FIG. 9 is a diagram illustrating an example of a correspondence relationship between the stereoscopic image space and the volume data space. FIG. 9A shows volume data, and FIG. 9B shows a stereoscopic image displayed by the display unit 132. Further, the coordinates 301, 302, and distance 303 in FIG. 9A correspond to the coordinates 304, 305, and distance 306 in FIG. 9B, respectively.

  As shown in FIG. 9, the volume data space in which the volume data is arranged and the stereoscopic image space in which the stereoscopic image is displayed have different coordinate systems. Specifically, the stereoscopic image shown in FIG. 9B is narrower in the depth direction (z direction) than the volume data shown in FIG. In other words, in the stereoscopic image shown in FIG. 9B, the component in the depth direction of the volume data shown in FIG. 9A is displayed after being compressed. In this case, as shown in FIG. 9B, the distance 306 between the coordinates 304 and the coordinates 305 is compressed as compared with the distance 303 between the coordinates 301 and the coordinates 302 in FIG. It gets shorter.

  The correspondence between the stereoscopic image space coordinates and the volume data space coordinates is uniquely determined by the scale, viewing angle, and viewing direction of the stereoscopic image (the viewing direction at the time of rendering or the viewing direction at the time of viewing the stereoscopic image). For example, it can be expressed in the following form (Equation 1).

  (Expression 1) = (x1, y1, z1) = F (x2, y2, z2)

  In (Equation 1), “x2”, “y2”, and “z2” respectively indicate stereoscopic image space coordinates. “X1”, “y1”, and “z1” indicate volume data space coordinates, respectively. The function “F” is a function that is uniquely determined by the scale, viewing angle, line-of-sight direction, and the like of the stereoscopic image. That is, the rendering control unit 1352 can acquire the correspondence between the stereoscopic image space coordinates and the volume data space coordinates by using (Equation 1). Note that the function “F” is generated by the rendering control unit 1352 every time the stereoscopic image scale, viewing angle, line-of-sight direction (line-of-sight direction during rendering, line-of-sight direction during stereoscopic image observation), or the like is changed. . For example, the affine transformation shown in (Expression 2) is used as the function “F” for transforming rotation, translation, enlargement, and reduction.

(Equation 2)
x1 = a * x2 + b * y2 + c * z3 + d
y1 = e * x2 + f * y2 + g * z3 + h
z1 = i * x2 + j * y2 + k * z3 + l
(A to l are conversion coefficients)

  In the above description, the example in which the rendering control unit 1352 acquires the coordinates of the volume data space based on the function “F” has been described, but the present invention is not limited to this. For example, the workstation 130 has a coordinate table that is a table in which stereoscopic image space coordinates and volume data space coordinates are associated, and the rendering control unit 1352 searches the coordinate table using the stereoscopic image space coordinates as search keys. Thus, the volume data space coordinates corresponding to the stereoscopic image space coordinates may be acquired.

  Returning to the description of FIG. 8, the display control unit 1353 causes the display unit 132 to display the parallax image group generated by the rendering processing unit 136. That is, the display control unit 1353 according to the first embodiment displays a stereoscopic image on the display unit 132, and also displays a stereoscopic image indicating an operable region where the observer can perform various operations on the stereoscopic image. Then, an image of a tool for performing various operations is displayed on the stereoscopic image. The display control unit 1353 displays the parallax image group on the display unit 132 when the rendering processing unit 136 newly generates the parallax image group.

  Next, a specific example of processing performed by the control unit 135 will be described with reference to FIGS. 10 and 11. 10 and 11 are diagrams for explaining an example of processing performed by the control unit 135 in the first embodiment. In FIGS. 10 and 11, a case where the operation contents are “stereoscopic image cutting” and “stereoscopic image deletion” will be described as an example.

  In the example illustrated in FIG. 10, first, the display unit 132 displays the parallax image group generated by the rendering processing unit 136, so that the stereoscopic image I <b> 11 that shows the organ of the subject and the like, and various operations that can be performed. A stereoscopic image Ia12 indicating the possible area SP10 and icon stereoscopic images Ic11 to Ic15 indicating tools for the observer to perform various operations on the stereoscopic image I11 are displayed. Here, the display unit 132 displays a substantially rectangular parallelepiped stereoscopic image Ia12 indicated by a dotted line as an image indicating the operable region SP10.

  In other words, the rendering control unit 1352 renders a parallax image group in which the stereoscopic image I11 of the subject illustrated in FIG. 10, the stereoscopic image Ia12 of the operable area SP10, and the stereoscopic images Ic11 to Ic15 of icons are displayed. Generated by the unit 136. The rendering control unit 1352 superimposes images corresponding to the stereoscopic images Ic11 to Ic15 on the parallax image group so that the icon stereoscopic images Ic11 to Ic15 are arranged at specific positions in the stereoscopic image space. The rendering processing unit 136 is controlled.

  Here, the icon Ic11 illustrated in FIG. 10 is an image indicating a cutting member such as a cutter knife, and serves as a tool for cutting the stereoscopic image I11. The icon Ic12 is an image indicating an erasing member such as an eraser and serves as a tool for partially erasing the stereoscopic image I11. The icon Ic13 is an image indicating a coloring member such as a palette, and serves as a tool for coloring the stereoscopic image I11. The icon Ic14 is an image indicating a deletion member such as a trash can and serves as a tool for deleting a part of the stereoscopic image I11. The icon Ic15 is an image showing a region-of-interest setting member for setting a region of interest.

  In the first embodiment, in a state where such various stereoscopic images are displayed on the display unit 132, various operations are performed on the stereoscopic image I11 by the observer's hand U11. For example, when the camera 137 detects that the observer's hand U11 has moved to the display position of the icon Ic11, the display unit 132 displays a stereoscopic image in which the display position of the icon Ic11 moves with the subsequent movement of the hand U11. Is displayed. Specifically, the rendering control unit 1352 makes the display position of the icon Ic11 substantially coincide with the position of the hand U11 with respect to the rendering processing unit 136 each time the position of the hand U11 detected by the camera 137 moves. In addition, the image of the icon Ic11 is superimposed on the parallax image group. Then, the display control unit 1353 causes the display unit 132 to display the parallax image group (superimposed image group) newly generated by the rendering processing unit 136 so that the position of the hand U11 substantially matches the display position of the icon Ic11. A stereoscopic image is provided to the viewer.

  Here, in the example shown in FIG. 10, it is assumed that the icon Ic11 passes through the plane A11 in the stereoscopic image I11 by moving the hand U11 by the observer. In such a case, the determination unit 1351 determines that an operation of cutting the stereoscopic image I11 with the plane A11 has been performed based on the icon Ic11 being a cutting member. At this time, the determination unit 1351 acquires the position information of the hand U11 in the operable area SP10 from the camera 137, thereby identifying the position in the three-dimensional image I11 of the surface A11 through which the icon Ic11 has passed, and the three-dimensional image of the identified surface A11. The rendering control unit 1352 is notified of the position in the image I11. Then, the rendering control unit 1352 obtains an area in the volume data space corresponding to the plane A11 by using the function “F”. Then, for example, the rendering control unit 1352 changes the voxel value of the voxel corresponding to the surface A11 in the voxel group constituting the volume data to a voxel value indicating air or the like, and then performs the rendering process on the volume data. The rendering processing unit 136 is controlled to perform the above. Thereby, the rendering processing unit 136 can generate a parallax image group for displaying the stereoscopic image I11 cut by the plane A11.

  Furthermore, it is assumed that an operation of moving the left part (left side of the plane A11) of the cut stereoscopic image I11 to the icon Ic14 by moving the hand U11 by the observer. Here, the left part of the stereoscopic image I11 may move with the hand U11 or may not move with the hand U11. However, when the left part of the stereoscopic image I11 moves together with the hand U11, the rendering control unit 1352, for example, in the volume data so that the position of the hand U11 and the position of the left part of the stereoscopic image I11 substantially match. After the voxel corresponding to the left part of the stereoscopic image I11 is placed at the position of the hand U11, the rendering processing unit 136 is controlled to perform the rendering process.

  Then, the determination unit 1351 determines that an operation for deleting the left part of the stereoscopic image I11 has been performed based on the fact that the icon Ic14 is a deletion member. The rendering control unit 1352 obtains a region in the volume data space corresponding to the left part of the stereoscopic image I11 by using the function “F”, and corresponds to the left part of the voxel group constituting the volume data. The rendering processing unit 136 is controlled so as to perform rendering processing with voxels to be excluded as rendering targets. Accordingly, the rendering processing unit 136 can generate a parallax image group for displaying the stereoscopic image I11 from which the left part is deleted.

  Note that the rendering processing unit 136 manages information (in this case, “rendering target flag”) indicating whether or not to render each voxel constituting the volume data, and the left of the stereoscopic image I11. By performing the rendering process after updating the rendering target flag of the voxel corresponding to the part to “non-rendering target”, it is possible to generate a parallax image group for displaying the stereoscopic image I11 from which the left part is deleted. For example, the rendering processing unit 136 sets the parallax image group in which the left part is deleted by setting the opacity of the voxel whose rendering target flag is “non-rendering target” to “0%”, for example. Can be generated.

  The display control unit 1353 causes the display unit 132 to display the parallax image group generated in this way. Thereby, the display part 132 can display the stereo image I11 from which the left part was deleted like the example shown in FIG. Here, the stereoscopic image I11 illustrated in FIG. 11 includes a group of parallax images generated by the rendering processing unit 136 performing the rendering process again. Therefore, the observer can observe the cross-sectional image of the portion cut by the icon Ic11 by wrapping around and observing the stereoscopic image I11 shown in FIG.

  Next, an example of the flow of processing by the workstation 130 in the first embodiment will be shown using FIG. FIG. 12 is a flowchart illustrating an example of a processing flow by the workstation 130 in the first embodiment.

  As illustrated in FIG. 12, the control unit 135 of the workstation 130 determines whether or not a stereoscopic image display request has been received from the terminal device 140 (step S101). If the display request is not accepted (No at step S101), the workstation 130 waits until the display request is accepted.

  On the other hand, when a display request is accepted (Yes in step S101), the rendering control unit 1352 of the workstation 130 controls the rendering processing unit 136 to include parallax including images such as an operable region and an operation icon. An image group is generated (step S102).

  Then, the display control unit 1353 of the workstation 130 causes the display unit 132 to display the parallax image group generated by the rendering processing unit 136 (step S103). Accordingly, as illustrated in FIG. 10, the display unit 132 shows a stereoscopic image indicating the organ of the subject, a stereoscopic image indicating an operable region where various operations can be performed, and a tool for performing various operations. Displays a three-dimensional icon image.

  Subsequently, the determination unit 1351 of the workstation 130 monitors whether or not the position variation of the observer's hand is detected within the operable region by the camera 137 (step S104). If no hand position change is detected (No at step S104), the determination unit 1351 waits until a camera position change is detected by the camera 137.

  On the other hand, when the position variation of the hand is detected by the camera 137 (Yes at Step S104), the determination unit 1351 identifies the operation content corresponding to the position variation (Step S105). Then, the rendering control unit 1352 controls the rendering processing unit 136 to perform the rendering process according to the operation content determined by the determination unit 1351. Accordingly, the rendering processing unit 136 newly generates a parallax image group (step S106). Then, the display control unit 1353 causes the display unit 132 to display the parallax image group newly generated by the rendering processing unit 136 (step S107).

  As described above, according to the first embodiment, it becomes possible for the observer to perform various operations on the stereoscopic image by a sensuous operation.

(Second Embodiment)
Now, the embodiment described above can be modified to other embodiments. Therefore, in the second embodiment, a modified example of the above-described embodiment will be described. In addition, FIGS. 13-15 demonstrated below are the figures for demonstrating the modification of 1st Embodiment.

[Operation Details]
In the above-described embodiment, the case where the operation contents are “cutting a stereoscopic image” and “deleting a stereoscopic image” has been described as an example with reference to FIGS. 10 and 11. However, the operation content is not limited to this. Examples of other operation contents received by the workstation 130 will be described in the following (1) to (6).

(1) Erasing a part of the stereoscopic image For example, in the example shown in FIG. 10, after the observer's hand U11 moves to the display position of the icon Ic12 that is an erasing member, a predetermined position (here, the stereoscopic image I11) , The bone is assumed to be displayed. In such a case, the determination unit 1351 determines that an operation for deleting a bone that is a part of the stereoscopic image I11 has been performed. Then, the rendering control unit 1352 controls the rendering processing unit 136 so as to perform the rendering process after updating the rendering target flag of the voxel indicating the bone in the voxel group constituting the volume data to “non-rendering target”. . Thereby, the workstation 130 can display the stereoscopic image I11 from which the bone has been erased.

(2) Change in Display Method of Stereoscopic Image In the example shown in FIG. 10, after the observer's hand U11 moves to the display position of the icon Ic13 that is a colored member, a predetermined position (here, It is assumed that the blood vessel is moved to the displayed position). In such a case, the determination unit 1351 determines that an operation for coloring a blood vessel that is part of the stereoscopic image I11 has been performed. Then, the rendering control unit 1352 performs rendering processing after updating the pixel value of the voxel indicating the blood vessel in the voxel group constituting the volume data to a value corresponding to the color designated by the observer. 136 is controlled. Thereby, the workstation 130 can display the stereoscopic image I11 including the blood vessel colored in the color designated by the observer. Note that a plurality of colors of paints and the like are displayed on the icon Ic13, and the determination unit 1351 can specify a color to be colored in the stereoscopic image in accordance with the color of the paint that has been touched by the observer. .

  Further, for example, the workstation 130 may display a stereoscopic image of an icon indicating an adjustment member such as a control bar in the operable area. Then, the determination unit 1351 determines that an operation for changing the opacity of the stereoscopic image has been performed when the observer performs an operation to move the knob or the like of the control bar left and right or up and down. To do. At this time, the rendering control unit 1352 controls the rendering processing unit 136 so as to perform the rendering process by changing the opacity according to the amount of movement of the knob moved by the observer, for example. In addition, when a predetermined area of the stereoscopic image is designated by the observer and the operation of moving the control bar knob or the like is performed, the workstation 130 sets the opacity only for the predetermined area. It may be changed.

  Here, an operation example of changing the opacity will be described with reference to FIG. In the example illustrated in FIG. 13, the display unit 132 displays the parallax image group generated by the rendering processing unit 136, thereby reducing the opacity of the stereoscopic image I21 indicating the subject's organ and the like and the stereoscopic image I21. An image I22 to be displayed, icon images Ic21 to Ic24 indicating a control bar and a knob for the observer to change the opacity, and the like are displayed. In other words, the rendering control unit 1352 renders a parallax image group in which the stereoscopic image I21 of the subject illustrated in FIG. 13, the image I22 indicating opacity, and the icon images Ic21 to Ic24 are displayed. To generate. The rendering control unit 1352 renders the images corresponding to the stereoscopic images Ic21 to Ic24 so as to be superimposed on the parallax image group so that the icon images Ic21 to Ic24 are arranged at specific positions in the stereoscopic image space. The processing unit 136 is controlled.

  Here, in the image I22 illustrated in FIG. 13, the horizontal axis indicates the CT value, and the vertical axis indicates the opacity. Specifically, the image I22 indicates that a region having a CT value smaller than the point P1 in the stereoscopic image I21 has an opacity “0%” and is not displayed as a stereoscopic image. Further, in the image I22, the region where the CT value is larger than the point P2 in the stereoscopic image I21 is displayed as a stereoscopic image with an opacity of “100%”, and the region behind the region is not drawn. Show. Further, the image I22 indicates that the region of the stereoscopic image I21 in which the CT value is in the range from the point P1 to the point P2 is a region where the opacity is “0%” to “100%”. That is, the region where the CT value is in the range from the point P1 to the point P2 is displayed semi-transparent, and the region where the CT value is closer to the point P1 is a region with higher transparency. A straight line L1 in the image I22 indicates a CT value that is a midpoint between the points P1 and P2.

  In the example illustrated in FIG. 13, the icon Ic21 indicates a control bar and a knob for changing the opacity of the entire stereoscopic image I21. The icon Ic22 indicates a control bar and a knob for determining a CT value range for setting the opacity. Specifically, the straight line L1 shown in the image I22 can be moved left and right together with the points P1 and P2 by moving the knob indicated by the icon Ic22 left and right. The icon Ic23 indicates a control bar and a knob for determining a CT value range with an opacity of “0%”. Specifically, the point P1 shown in the image I22 can be moved left and right by moving the knob indicated by the icon Ic23 left and right. The icon Ic24 indicates a control bar and a knob for determining a CT value range with an opacity of “100%”. Specifically, the point P2 shown in the image I22 can be moved left and right by moving the knob indicated by the icon Ic24 left and right.

  In a state where the stereoscopic image I21, the image I22, the icons Ic21 to Ic24 and the like are displayed on the display unit 132, for example, after the observer's hand U11 is moved to the display position of the icons Ic21 to Ic24, various knobs are displayed. When the camera 137 detects a movement of moving the image to the left or right, the rendering control unit 1352 changes the opacity of the rendering processing unit 136 according to the movement of the hand U11 detected by the camera 137. Then, the rendering processing unit 136 is controlled to perform the rendering process. Thereby, the rendering processing unit 136 can generate a parallax image group in which the opacity is changed according to the movement of the hand U11.

(3) Rotation of stereoscopic image For example, the determination unit 1351 performs an operation of rotating a stereoscopic image when the camera 137 detects the position variation of the hand illustrated in FIG. Judge that it was broken. In such a case, the rendering control unit 1352 controls the rendering processing unit 136 to perform the rendering process by changing the viewpoint position and the line-of-sight direction, thereby generating a parallax image group for displaying the rotated stereoscopic image. Can be generated.

  The setting that can be adjusted by such a control bar is not limited to the opacity. For example, when the stereoscopic image control bar is operated by the observer, the workstation 130 may adjust the enlargement ratio or reduction ratio of the stereoscopic image, the parallax angle of the parallax image group constituting the stereoscopic image, and the like. .

(4) Region of Interest Setting In addition, the determination unit 1351 determines that an operation for setting the predetermined region as the region of interest has been performed when the observer touches the predetermined region of the stereoscopic image. May be. For example, in the example shown in FIG. 10, after the observer's hand U11 moves to the display position of the icon Ic15 that is the region of interest setting member, a predetermined position (here, the position where the blood vessel is displayed) of the stereoscopic image I11. ). In such a case, the determination unit 1351 determines that an operation for setting a blood vessel that is a part of the stereoscopic image I11 as a region of interest has been performed. Specifically, the determination unit 1351 identifies a position (here, a position where a blood vessel is displayed) in the stereoscopic image I11 touched by the hand U11. Then, the determination unit 1351 performs a segmentation process using a pattern matching method using a shape template, a region growing method, or the like, whereby an organ (here, the position specified by the observer) Blood vessel) is extracted, and the extracted organ is set as a region of interest.

(5) Operation format Moreover, although the example shown in FIG.10 and FIG.11 showed the example where various operation is performed with respect to a stereo image with one hand, an observer performs various operation with respect to a stereo image with both hands. You may go. For example, when the operation of wrapping a predetermined area with the palms of both hands is performed, the determination unit 1351 may determine that an operation for setting the predetermined area as the region of interest has been performed.

[segmentation]
In the above-described embodiment, the determination unit 1351 performs an operation in which a predetermined region of the stereoscopic image is touched by an observer by the observer or the predetermined region as described in “(5) Operation format” above. May be determined that an operation of extracting an organ (blood vessel, bone, heart, liver, etc.) included in the predetermined region has been performed. In such a case, the rendering control unit 1352 performs segmentation processing using a pattern matching method using a shape template, a region growing method, or the like, so that the rendering control unit 1352 is included in a predetermined region designated by the observer. The rendering processing unit 136 is controlled to extract the organs (blood vessels, bones, heart, liver, etc.) to be extracted. The rendering processing unit 136 may generate a parallax image group indicating only the organ by performing rendering processing on the extracted organ volume data, or volume data excluding the extracted organ data. A parallax image group indicating a part excluding such an organ may be generated by performing rendering processing on the image. As a result, even when the observer cannot designate with high accuracy in the operable region, the observer causes the display unit 132 to display a stereoscopic image of only a desired organ, a stereoscopic image of a portion excluding only the desired organ, or the like. be able to.

[Operation device]
Moreover, in the said embodiment, the case where various operation with respect to a stereo image was performed by the observer's hand was mentioned as an example, and was demonstrated. Specifically, the example in which the operation content for the stereoscopic image is determined by detecting the position change of the observer's hand with the camera 137 has been described. However, performing various operations on the stereoscopic image is not limited to the hand of the observer. For example, the observer may perform various operations on the stereoscopic image using the operation device illustrated in FIG. The operation device 150 illustrated in FIG. 15 is provided with various buttons 151 to 154. The buttons 151 and 152 accept, for example, which one of rotation, enlargement, reduction, cutting, deletion, coloring, opacity, etc. is changed. The buttons 153 and 154 accept settings such as the amount of rotation of a stereoscopic image, an enlargement rate, a reduction rate, and an opacity, for example. Such an operation device 150 may include a position sensor that can acquire a position in the operable region, and may transmit position information in the operable region acquired by the position sensor to the workstation 130. In such a case, the display unit 132 may not have the camera 137.

[Processing entity]
Further, in the above embodiment, the case where the workstation 130 receives various operations on the stereoscopic image has been described as an example. However, the embodiment is not limited to this. For example, the terminal device 140 may accept various operations on a stereoscopic image. In such a case, the terminal device 140 has functions equivalent to the determination unit 1351 and the display control unit 1353 illustrated in FIG. When the terminal device 140 displays the parallax image group generated by the workstation 130 on the stereoscopic display monitor of the terminal device 140 and receives various operations on the stereoscopic image displayed on the stereoscopic display monitor, By transmitting the operation content to the workstation 130, a parallax image group corresponding to the operation content is acquired from the workstation 130.

  In the above example, the terminal device 140 may have a function equivalent to the rendering control unit 1352 illustrated in FIG. In such a case, the volume data is acquired from the terminal device 140, and the same processing as each processing unit shown in FIG. 8 is performed on the acquired volume data.

  In the above embodiment, the medical image diagnostic apparatus 110 and the workstation 130 may be integrated. That is, the medical image diagnostic apparatus 110 may have a function equivalent to that of the control unit 135.

[System configuration]
In addition, among the processes described in the above embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part of the distribution / integration may be functionally or physically distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the control unit 135 of the workstation 130 may be connected as an external device of the workstation 130 via a network.

[program]
It is also possible to create a program in which the processing executed by the workstation 130 in the above embodiment is described in a language that can be executed by a computer. In this case, the same effect as the above-described embodiment can be obtained by the computer executing the program. Further, such a program may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer and executed to execute the same processing as in the above embodiment. For example, such a program is recorded on a hard disk, flexible disk (FD), CD-ROM, MO, DVD, Blu-ray or the like. Such a program can also be distributed via a network such as the Internet.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 Image processing system 110 Medical image diagnostic apparatus 120 Image storage apparatus 130 Workstation 132 Display part 135 Control part 136 Rendering process part 137 Camera 140 Terminal apparatus 1351 Judgment part 1352 Rendering control part 1353 Display control part

Claims (4)

A stereoscopic display device that displays a stereoscopically viewable stereoscopic image using a parallax image group generated from volume data that is three-dimensional medical image data;
Identifying the position variation of the moving object in the three-dimensional image space from the position variation of the predetermined moving object in the real space where the coordinate system of the three-dimensional image space, which is the space where the three-dimensional image is displayed by the three-dimensional display device, A determination unit that determines an operation content for the stereoscopic image based on the identified positional variation;
An image processing system comprising: a rendering processing unit that newly generates a parallax image group by performing a rendering process on the volume data in accordance with the operation content determined by the determination unit. A stereoscopic display device that displays a stereoscopically viewable stereoscopic image using a parallax image group generated from volume data that is three-dimensional medical image data;
Identifying the position variation of the moving object in the three-dimensional image space from the position variation of the predetermined moving object in the real space where the coordinate system of the three-dimensional image space, which is the space where the three-dimensional image is displayed by the three-dimensional display device, A determination unit that determines an operation content for the stereoscopic image based on the identified positional variation;
An image processing apparatus comprising: a rendering processing unit that newly generates a parallax image group by performing a rendering process on the volume data in accordance with the operation content determined by the determination unit. An image processing method by an image processing system having a stereoscopic display device that displays a stereoscopically viewable stereoscopic image using a parallax image group generated from volume data that is three-dimensional medical image data,
Identifying the position variation of the moving object in the three-dimensional image space from the position variation of the predetermined moving object in the real space where the coordinate system of the three-dimensional image space, which is the space where the three-dimensional image is displayed by the three-dimensional display device, A determination step of determining an operation content for the stereoscopic image based on the identified position variation;
A rendering process step of newly generating a parallax image group by performing a rendering process on the volume data in accordance with the operation content determined in the determination step. A stereoscopic display device that displays a stereoscopically viewable stereoscopic image using a parallax image group generated from volume data that is three-dimensional medical image data;
Identifying the position variation of the moving object in the three-dimensional image space from the position variation of the predetermined moving object in the real space where the coordinate system of the three-dimensional image space, which is the space where the three-dimensional image is displayed by the three-dimensional display device, A determination unit that determines an operation content for the stereoscopic image based on the identified positional variation;
A medical image diagnostic apparatus comprising: a rendering processing unit that newly generates a parallax image group by performing a rendering process on the volume data in accordance with the operation content determined by the determination unit.

Images