JP2012252650A - Three dimensional display processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To indicate an attention point three-dimensionally on an image which is viewed as a stereoscopic vision by a three dimensional display device 30 in a three dimensional display processing system.SOLUTION: A three dimensional display processing system 1 comprises: a reference image generation part 42 that performs rendering processing for volume data including a specimen image and that generates first three-dimensional image data corresponding to plural lines of sight; an operation device 10 by which an operator sets and operates a position on a plane of projection at right angles to a central line of sight of plural lines of sight and a position of a direction of the center line of sight; an image generation part for plane position settlement 44, which generates second three-dimensional image data that corresponds to the plural lines of sight and in which a cursor image is placed on the basis of the plane of projection position; and a three dimensional display device 30 that performs a three dimensional display on the basis of the first three-dimensional image data corresponding to plural lines of sight or the second three-dimensional image data corresponding to plural lines of sight.

Description

  An embodiment as one aspect of the present invention relates to a 3D display processing system applied when 3D display (three-dimensional display, stereoscopic display) is performed using a multi-parallax image.

  Today, a technology for performing 3D display using a multi-parallax image (for example, 9 parallax images) has been developed. In the medical industry, there is a need technology for a technology for performing 3D display using a plurality of rendering images corresponding to a plurality of lines of sight. It is increasing. As a technique for performing 3D display, a naked eye type (such as a parallax barrier method and a lenticular lens method) or a glasses type (such as an anaglyph method, a frame sequential method, and a polarization method) is employed.

  In the autostereoscopic system, a system capable of stereoscopically viewing multi-parallax images taken from a plurality of viewpoints with the naked eye using a light controller such as a lenticular lens has been put into practical use. In the glasses type, a system capable of stereoscopically viewing a two-parallax image (binocular parallax image) taken from two viewpoints using a dedicated device such as stereoscopic glasses has been put into practical use. Note that a 2-parallax image or a 9-parallax image displayed on a stereoscopically viewable 3D display device is generated by estimating depth information of an image captured from one viewpoint and using image processing using the estimated information. There is also.

  As a conventional technique, there is a technique in which a generated rendering image is displayed in a display area, and a mark is displayed so as to be superimposed on the rendering image at a predetermined position on the display area (see, for example, Patent Document 1).

JP 2011-36474 A

  When 3D display is performed using the naked eye type, a display device for 3D display is required. And even if an observer operates the cursor on a display apparatus, there exists a problem that a cursor will be directly displayed on the display surface of a display apparatus. Since the image displayed by the display device is visible to the viewer as 3D, the viewer cannot match the point on the 3D even if he wants the cursor to be positioned on the visible 3D. This is because the conventional interface cannot express the depth direction (the vertical direction of the display device) and can only express it in 2D.

  Similarly, when 3D display is performed using a glasses type, there is a problem that the cursor is displayed directly on the display surface of the display device.

  In order to solve the above-described problem, the 3D display processing system of the present embodiment performs a rendering process on volume data including a subject image to generate first three-dimensional image data corresponding to a plurality of lines of sight. One image generating means, an operating means for an operator to set and operate a position on the projection plane orthogonal to the central line of sight of the plurality of lines of sight and a position in the direction of the center line of sight, and a cursor based on the position on the projection plane Second image generation means for generating second 3D image data corresponding to a plurality of lines of sight in which an image is arranged, and first 3D image data corresponding to the plurality of lines of sight, or second corresponding to the plurality of lines of sight. Display means for performing 3D display based on the three-dimensional image data.

Schematic which shows the 3D display processing system of 1st Embodiment. Schematic which shows the structure of the operating device with which the 3D display processing system of 1st Embodiment is equipped. (A) is the schematic which shows the structure of the 3D display apparatus with which the 3D display processing system of 1st Embodiment is equipped, (b) is II sectional drawing shown to Fig.3 (a). The block diagram which shows the structure of the 3D display processing system of 1st Embodiment. The block diagram which shows the function of the 3D display processing system of 1st Embodiment. The figure for demonstrating the difference in the to-be-viewed subject image in a reference | standard mode and a solid-position determination mode. The figure for demonstrating arrangement | positioning of a cursor image. The figure which shows typically the image containing the cursor image seen stereoscopically. The block diagram which shows the function of the 3D display processing system of 2nd Embodiment. The figure which shows typically the image containing the cursor image seen stereoscopically. (A), (b) is a figure for demonstrating the 1st example of the display method of the rendering image of multiple eyes | visual_axis. (A), (b) is a figure for demonstrating the 2nd example of the display method of the rendering image of multiple eyes | visual_axis. The figure for demonstrating the 1st example of the display layout of the rendering image of multiple eyes | visual_axis. The figure for demonstrating the 2nd example of the display layout of the rendering image of multiple eyes | visual_axis.

The 3D display processing system of this embodiment will be described with reference to the accompanying drawings.
(First embodiment)

  FIG. 1 is a schematic diagram illustrating a 3D display processing system according to the first embodiment.

  FIG. 1 shows a 3D display processing system 1 of the first embodiment. The 3D display processing system 1 has a configuration in which a viewer can stereoscopically view a multi-parallax image, for example, a 9-parallax image with the naked eye by using a light beam controller. The 3D display processing system 1 enables stereoscopic viewing based on binocular parallax, and also enables stereoscopic viewing based on motion parallax that also changes the image observed in accordance with the viewpoint movement of the observer. The 3D display processing system 1 includes an operation device (remote controller) 10, an image processing device 20, and a 3D display device 30.

  The operation device 10 is configured to be able to communicate with the image processing device 20 (for example, wireless communication typified by Bluetooth). The image processing device 20 is configured to be able to communicate with the 3D display device 30.

  FIG. 2 is a schematic diagram illustrating a structure of the operation device 10 provided in the 3D display processing system 1 of the first embodiment.

  2 includes an operation unit 11 and a data transmission / reception unit 12. The operation unit 11 includes, for example, a button 11a and a cross button 11b. The operation unit 11 may include a wheel or the like instead of the button 11a and the cross button 11b or simultaneously.

  The button 11 a has a structure that can be pressed and released in the direction v with respect to the main body of the operating device 10.

  The cross button 11b has four switches arranged on the top, bottom, left, and right, and a cross-shaped cover covers the switches. There is a fulcrum at the center of the cross, and the cover moves in the direction v like a seesaw so that the up / down or left / right switches cannot be pressed simultaneously. The button 11a and the cross button 11b may have an integral structure.

  When the operation unit 11 is operated by an observer (operator) of the 3D display device 30, the data transmission / reception unit 12 transmits an operation signal to the image processing device 20 wirelessly or by wire.

  FIG. 3A is a schematic diagram illustrating a structure of the 3D display device 30 provided in the 3D display processing system 1 of the first embodiment. FIG. 3B is a cross-sectional view taken along the line II shown in FIG.

A 3D display device 30 shown in FIGS. 3A and 3B is a display capable of detecting the illuminance of external light in addition to displaying an image, for example, a liquid crystal display. The 3D display device 30 includes a backlight 31, a liquid crystal panel (liquid crystal shutter) 32, and a convex lens array 33. Note that the 3D display device 30 is not limited to a liquid crystal display, and may be an organic EL (Electro Luminescence) display, a plasma display, or the like.
The backlight 31 of the 3D display device 30 includes a plurality of white LEDs (light emitting diodes) and is disposed on the back surface of the liquid crystal panel 32. The backlight 31 is turned on when a power supply voltage is applied from a backlight drive circuit 34 (shown in FIG. 4), and illuminates the liquid crystal panel 32. Note that the backlight 31 is not limited to white LEDs, and may include LEDs of other colors. Further, the backlight 31 may include, for example, a cold cathode fluorescent lamp (CCFL) instead of the LED.

  The liquid crystal panel 32 includes a plurality of pixels P provided in a matrix in the horizontal direction (x direction) and the vertical direction (y direction) of the liquid crystal panel 32. The pixel P includes a polarizing plate Pa, a glass substrate Pb, a pixel electrode (transparent conductive film) Pc, an alignment film Pd, a liquid crystal Pe, a counter electrode (transparent conductive film) Pf, a color filter Pg, and an optical sensor Ph.

  In the pixel P, pixels corresponding to a plurality of lines of sight, for example, nine lines of sight, are arranged in the x direction of the liquid crystal panel 32. In the picture element, R (red), G (green), and B (blue) color filters Pg are arranged in the y direction, which is a direction perpendicular to the arrangement direction of the picture elements. Is formed. The arrangement direction of the picture elements is not limited to the x direction of the liquid crystal panel 32. In addition, the RGB surface (xy plane) shape is not limited to a rectangle. For example, the surface shape of RB may be a parallelogram inclined to the right (or left), and the surface shape of G may be a parallelogram inclined to the left (or right).

  The polarizing plate Pa is provided on the glass substrate Pb and has a function of allowing only light in a specific direction out of light from the backlight 31 to pass.

  The pixel electrode Pc is provided on the glass substrate Pb. The pixel electrode Pc is an electrode constituting a pixel provided on the thin film transistor (TFT) side of the electrodes sandwiching the liquid crystal Pe.

  The alignment film Pd is provided on each of the electrodes Pc and Pf and has a function of aligning liquid crystal molecules in a specific direction.

  The liquid crystal Pe has a function of changing the arrangement of molecular arrangement when an external voltage is applied.

  The counter electrode Pf is provided on the glass substrate Pb. The counter electrode Pf is an electrode provided on the side facing the TFT among the electrodes sandwiching the liquid crystal Pe.

  The color filter Pg is provided on the glass substrate Pb on the side facing the TFT, and each RGB is arranged for each pixel.

  A convex lens array 33 of the 3D display device 30 shown in FIGS. 3A and 3B is provided on the front surface of the liquid crystal panel 32. The convex lens array 33 is, for example, a lenticular lens array in which a plurality of lenticular lenses (kamaboko lenses) having an axis in the y direction perpendicular to the pixel arrangement direction are arranged in the x direction, which is the pixel arrangement direction. Although not shown, the convex lens array 33 may be a fly-eye lens array in which fly-eye lenses (eye-eye lenses) assigned to each pixel P are arranged for a plurality of pixels P.

  FIG. 4 is a block diagram illustrating a configuration of the 3D display processing system 1 according to the first embodiment.

  The operation device 10 of the 3D display processing system 1 illustrated in FIG. 4 includes the operation unit 11 (the button 11a and the cross button 11b) and the data transmission / reception unit 12 as described with reference to FIG.

  The data transmission / reception unit 12 generates an operation signal in accordance with the operation of the button 11 a and the cross button 11 b and transmits the operation signal to the image processing device 20.

  The image processing apparatus 20 includes basic hardware such as a processing unit (CPU) 21, a memory 22, an HDD (hard disk drive) 23, an IF (interface) 24, and a data transmission / reception unit 25. The processing unit 21 is interconnected to each hardware component constituting the image processing apparatus 20 via a bus as a common signal transmission path. Note that the image processing apparatus 20 may include a storage medium drive (not shown).

  The processing unit 21 is a control device having a configuration of an integrated circuit (LSI) in which an electronic circuit made of a semiconductor is enclosed in a package having a plurality of terminals. When a command is input by operating the operation device 10 by an observer, the processing unit 21 executes a program stored in the memory 22. Alternatively, the processing unit 21 reads a program stored in the HDD 23, a program transferred from the network N and installed in the HDD 23, or a program read from the storage medium installed in the storage medium drive and installed in the HDD 23. , Loaded into the memory 22 and executed.

  The memory 22 is a storage device having a configuration that combines elements such as a ROM (read only memory) and a RAM (random access memory). The memory 22 stores IPL (initial program loading), BIOS (basic input / output system), and data, and is used for work memory of the processing unit 21 and temporary storage of data.

  The HDD 23 is a storage device having a configuration in which a metal disk coated or vapor-deposited with a magnetic material is not removable and is built in. The HDD 23 is a storage device that stores programs installed in the image processing apparatus 20 (including an OS (operating system) in addition to application programs) and data. In addition, the OS can be provided with a graphical user interface (GUI) capable of performing basic operations using the operation device 10 by frequently using graphics for displaying information to the observer.

  The IF 24 includes a connector that conforms to a parallel connection specification or a serial connection specification, and performs communication control according to each standard. The IF 24 has a function capable of being connected to the network N, whereby the image processing apparatus 20 can be connected from the IF 24 to the network N network.

  The data transmitter / receiver 25 receives an operation signal transmitted from the data transmitter / receiver 12 of the controller device 10.

  As described with reference to FIG. 3, the 3D display device 30 includes the backlight 31, the liquid crystal panel 32, and the convex lens array 33. The 3D display device 30 includes a backlight drive circuit 34, a liquid crystal panel drive circuit 35, and a control unit 36.

  The backlight drive circuit 34 drives the backlight 31 by applying a voltage to the backlight 31 according to control by the control unit 36.

  The liquid crystal panel drive circuit 35 drives the pixel circuit of the pixel P of the liquid crystal panel 32 according to control by the control unit 36.

  The control unit 36 controls the backlight drive circuit 34 and the liquid crystal panel drive circuit 35 in accordance with instructions from the processing unit 21 of the image processing apparatus 20.

  FIG. 5 is a block diagram illustrating functions of the 3D display processing system 1 according to the first embodiment.

  When the processing unit 21 illustrated in FIG. 4 executes the program, as illustrated in FIG. 5, the image processing apparatus 20 of the 3D display processing system 1 includes a volume data acquisition unit 41, a reference image generation unit 42, and a mode determination unit 43. , Function as a plane position determination image generation unit 44, a plane position determination unit 45, a part extraction unit 46, a depth determination image generation unit 47, and a depth determination unit 48. Note that the image processing apparatus 20 may include all or a part of each of the units 41 to 48 as hardware.

  The volume data acquisition unit 41 has a function of acquiring volume data including a subject image stored in a storage device such as the HDD 23. The volume data acquisition unit 41 may acquire volume data received from the outside via the network N. For example, the volume data is generated by an ultrasonic diagnostic apparatus, an X-ray CT (computed tomography) apparatus, an MRI (magnetic resonance imaging) apparatus, and a nuclear medicine diagnostic apparatus.

  When the mode determination unit 43 determines that the reference mode (normal mode) is set, the reference image generation unit 42 performs the parallax rendered in the stereoscopic position determination mode on the volume data acquired by the volume data acquisition unit 41. It has a function of generating nine rendering images by performing rendering processing with a plurality of visual lines, for example, nine visual lines, with the following parallax angles. The nine rendering images generated by the reference image generation unit 42 are sent to the control unit 36 of the 3D display device 30 as data signals. The data signals of the nine rendering images are assigned to the nine picture elements constituting the pixel P of the liquid crystal panel 32 through the liquid crystal panel driving circuit 35 and are displayed in the order of the line of sight. Hereinafter, a case where rendering processing is performed with nine lines of sight will be described. The rendered image is obtained by a surface rendering process or a volume rendering process.

  Note that the configuration in which the number of picture elements constituting the pixel P of the liquid crystal panel 32 is the same as the number of rendering images has been described, but the number of picture elements constituting the pixel P is equal to or greater than the number of rendering images. Any configuration may be used. For example, in order to display three rendering images, when the pixel P is configured by nine picture elements, the rendering image of the leftmost line of sight is assigned to the three pixels of the rightmost line of the pixel P, and the rendering image of the central visual line is displayed. Are assigned to the three pixels at the center of the pixel P, and the rendered image of the rightmost line of sight is assigned to the three pixels at the left end of the pixel P. Further, the number of picture elements assigned to each rendering image of a plurality of line-of-sight rendering images may be different.

  When the nine rendering images generated by the reference image generation unit 42 are displayed by the 3D display device 30, the subject image is displayed in 3D.

  The mode determination unit 43 determines whether the 3D display processing system 1 is in the reference mode or the three-dimensional position determination (identification) mode according to the operation signal transmitted from the data transmission / reception unit 12 of the controller device 10. Have If high-precision alignment is required, such as when determining the three-dimensional position, a three-dimensional position determination mode is set.

  When the mode determination unit 43 determines that the plane position determination image generation unit 44 is in the three-dimensional position determination mode, the plane position determination image generation unit 44 is rendered in the reference mode with respect to the volume data rendered by the reference image generation unit 42 in the reference mode. In addition, it has a function of generating nine rendering images by performing rendering processing with nine lines of sight at a parallax angle exceeding the parallax angle. The nine rendering images generated by the plane position determination image generation unit 44 are sent to the control unit 36 of the 3D display device 30 as data signals. The data signals of the nine rendering images are assigned to the nine picture elements constituting the pixel P of the liquid crystal panel 32 through the liquid crystal panel driving circuit 35 and are displayed in the order of the line of sight.

  FIG. 6 is a diagram for explaining a difference in a subject image stereoscopically viewed in the reference mode and the stereoscopic position determination mode.

  FIG. 6 shows the stereoscopically viewed subject image O, the parallax angle θ1, and the 3D display device 30 in the reference mode, and the stereoscopically viewed subject image O, the parallax angle θ2, and the 3D display in the stereoscopic position determination mode. Device 30 is shown. At a parallax angle θ2 that exceeds the parallax angle θ1 rendered in the reference mode, rendering processing is performed with a plurality of lines of sight to generate and display nine rendering images. An observer observing the 3D display device 30 has generated, by the reference image generation unit 42, a subject image O that is stereoscopically viewed based on a plurality of visual line rendering images generated by the plane position determination image generation unit 44. It appears to exist in front of the subject image O that is stereoscopically viewed based on the rendered images of multiple lines of sight.

  According to the display of a plurality of visual line rendering images generated in the stereoscopic position determination mode by the plane position determination image generation unit 44, the difference in the position of the subject image O in the depth direction (z direction) can be easily determined. However, in the three-dimensional position determination mode, the observer must maintain a state (crossed eye) in which the left and right eyeballs are inward (in the direction of the nose), so that the observer's eyes become very tired. However, in the 3D display processing system 1, the mode determination unit 43 can switch between the reference mode and the three-dimensional position determination mode.

  The plane position determination unit 45 illustrated in FIG. 5 allows the observer to view the subject image O stereoscopically viewed based on the nine line-of-sight rendering images generated by the plane position determination image generation unit 44. Is operated to determine the xy plane position on the liquid crystal panel 32 in accordance with the operation signal transmitted from the data transmitting / receiving unit 12 of the controller device 10. That is, the plane position determination unit 45 determines the position on the projection plane orthogonal to the central line of sight of the nine lines of sight corresponding to the nine rendered images displayed on the 3D display device 30 according to the operation signal in the stereoscopic mode. Determine. In order to determine the xy plane position in the stereoscopic mode, the plane position determination unit 45 displays an axis orthogonal to the xy plane on which the cursor is located with a dotted line or the like, and based on the guide, the xy plane It is preferable to fix the position.

  The plane position determining unit 45 operates from the data transmitting / receiving unit 12 of the operating device 10 by the observer operating the operating device 10 while viewing the subject image O viewed in plan based on a rendering image of one line of sight. The xy plane position on the liquid crystal panel 32 may be determined according to the transmitted operation signal.

  The part extraction unit 46 passes through the plane position determined by the plane position determination unit 45 among the plurality of voxels constituting the volume data rendered by the plane position determination image generation unit 44, and includes nine rendering images. And a function of extracting a desired part based on voxels on the central line of sight of nine lines of sight. For example, the part extraction unit 46 extracts the edge portion by performing threshold processing of the amount of change in the voxel value for the voxel that passes through the confirmed position and is on the center line of sight.

  The depth determination image generation unit 47 has a function of generating nine rendering images in which a cursor image is arranged at the edge portion extracted by the part extraction unit 46. For example, the depth determination image generation unit 47 replaces the voxel value of the voxel corresponding to the edge portion extracted by the part extraction unit 46, generates volume data in which the cursor image is arranged in the voxel, and the volume The rendering process is performed on the data with nine lines of sight to generate nine rendering images.

  FIG. 7 is a diagram for explaining the arrangement of the cursor image.

  FIG. 7 shows volume data V and a projection plane orthogonal to the central line of sight L of the nine lines of sight. The projection surface corresponds to the liquid crystal panel 32. The volume data V shown in FIG. 7 includes the yz section and the xz section including the xy plane position of the liquid crystal panel 32 determined by the plane position determining unit 45, that is, the yz including the projection plane position T. It is shown as a cross section and an xz cross section. The central line of sight L that is orthogonal to the projection plane is equal to the z direction that is orthogonal to the liquid crystal panel 32.

  When the projection plane position T is determined by the plane position determination unit 45, the depth determination image generation unit 47, among the plurality of voxels constituting the volume data V, the edge portion on the central line of sight L passing through the projection plane position T. A cursor image B is placed on a voxel including (or a voxel group including the voxel and its neighboring voxels). In the example shown in FIG. 7, four edge portions are shown.

  The nine rendering images generated by the depth determination image generation unit 47 are sent to the control unit 36 of the 3D display device 30 as data signals. The data signals of the nine rendering images are assigned to the nine picture elements constituting the pixel P of the liquid crystal panel 32 through the liquid crystal panel driving circuit 35 and are displayed in the order of the line of sight.

  FIG. 8 is a diagram schematically illustrating an image including a cursor image that is stereoscopically viewed.

  As shown in FIG. 8, the observer uses the operation device 10 to determine the position in the xy plane by stereoscopic viewing of the nine rendering images displayed on the 3D display device 30 by the depth determination image generation unit 47. If the (projection plane position) T is indicated, the edge portion in the z direction (the direction of the central line of sight) including the position T can be stereoscopically viewed via the cursor image B. Each cursor image B may be color-coded, or preferably with a caption (1, 2, 3,...) Or the like.

The depth determination unit 48 illustrated in FIG. 5 is configured so that the observer operates the operation device while viewing the subject image illustrated in FIG. 8 stereoscopically based on the nine line-of-sight rendering images generated by the depth determination image generation unit 47. 10 has a function of selecting and confirming a desired cursor image from the cursor image B in accordance with an operation signal transmitted from the data transmitting / receiving unit 12 of the controller device 10. When the position of the cursor image is confirmed by the depth confirmation unit 48, the controller device 10 and / or the 3D display device 30 may be configured to output confirmation completion. For example, the controller device 10 is vibrated when the position of the cursor image is determined. For example, the 3D display device 30 changes the color of the displayed cursor image in accordance with an instruction from the depth determination image generation unit 47.
(Second Embodiment)

  In the configuration of the 3D display processing system of the second embodiment, the same members as those in the configuration of the 3D display processing system 1 shown in FIGS.

  FIG. 9 is a block diagram illustrating functions of the 3D display processing system according to the second embodiment.

  When the processing unit 21 illustrated in FIG. 4 executes the program, as illustrated in FIG. 9, the image processing apparatus 20 of the 3D display processing system 1A includes a volume data acquisition unit 41, a reference image generation unit 42, and a mode determination unit 43. , Function as a plane position determination image generation unit 44, a plane position determination unit 45, a part extraction unit 46A, a depth determination image generation unit 47A, a depth determination unit 48, and a depth change unit 49. Note that the image processing apparatus 20 may include all or a part of each of the units 41 to 49 as hardware. In addition, in the function of 3D display processing system 1A of 2nd Embodiment shown in FIG. 9, the same code | symbol is attached | subjected to the same member as the function of 3D display processing system 1 shown in FIG. 5, and description is abbreviate | omitted.

  The part extraction unit 46A passes through the projection plane positions determined by the plane position determination unit 45 among the plurality of voxels constituting the volume data rendered by the plane position determination image generation unit 44, and nine renderings are performed. It has a function of extracting a desired part based on voxels on the central line of sight of nine lines of sight corresponding to an image. For example, the part extraction unit 46 has a function of extracting one edge portion by performing threshold processing of the amount of change in the voxel value for voxels that pass through the projection plane position and on the central line of sight. In addition, for example, the part extracting unit 46 detects a surface portion farthest from the projection plane position among a plurality of surface portions of the subject image O that passes through the projection plane position and exists on the central line of sight.

  The depth determination image generation unit 47A has a function of generating nine rendering images in which a cursor image is arranged on one edge portion extracted by the part extraction unit 46A. For example, the depth determination image generation unit 47A replaces the voxel value of the voxel corresponding to one edge portion extracted by the part extraction unit 46A, generates volume data in which one cursor image is arranged, and The volume data is subjected to rendering processing with nine lines of sight to generate nine rendering images. One cursor image arranged in the volume data generated by the depth determination image generation unit 47A is one of the four cursor images shown in FIG.

  The nine rendering images generated by the depth determination image generation unit 47A are sent to the control unit 36 of the 3D display device 30 as data signals. The data signals of the nine rendering images are assigned to the nine picture elements constituting the pixel P of the liquid crystal panel 32 through the liquid crystal panel driving circuit 35 and are displayed in the order of the line of sight.

  The depth changing unit 49 is operated by the observer operating the operating device 10 while viewing the subject image O stereoscopically viewed based on the nine line-of-sight rendering images generated by the depth determination image generating unit 47A. It has a function of changing the position of the cursor image in the z direction according to the operation signal transmitted from the data transmitting / receiving unit 12 of the apparatus 10. That is, the depth changing unit 49 changes the position of the cursor image on the central line of sight of nine lines of sight corresponding to the nine rendered images displayed on the 3D display device 30 in accordance with the operation signal.

  The depth determination image generation unit 47A has a function of generating nine rendering images in which the cursor image after the change by the depth change unit 49 is arranged. For example, the depth determination image generation unit 47A moves away from the voxel in which the cursor image before the change is arranged according to the change amount, and replaces the voxel value of the voxel on the center line of sight, thereby arranging one cursor image. Volume data is generated, and rendering processing is performed on the volume data with nine lines of sight to generate nine rendering images.

  FIG. 10 is a diagram schematically illustrating an image including a cursor image that is stereoscopically viewed.

  As shown in FIG. 10, the observer uses the operation device 10 to determine the position in the xy plane by stereoscopic viewing of the nine rendering images displayed on the 3D display device 30 by the depth determination image generation unit 47A. If (projection plane position) T is indicated, one edge portion in the z direction (direction of the central line of sight) including the position T can be stereoscopically viewed via one cursor image B1.

  Furthermore, if the observer uses the operation device 10 to indicate a change in the z direction (the direction of the central line of sight) of the cursor image B1, the position of the cursor image B1 is changed to the near side or the far side in the z direction. be able to.

  A display method for rendering images with a plurality of lines of sight in the 3D display processing system 1A of the second embodiment will be described.

  FIGS. 11A and 11B are diagrams for explaining a first example of a display method for rendering images with a plurality of lines of sight.

  FIGS. 11A and 11B show the volume data V and the projection plane orthogonal to the central line of sight L of the nine lines of sight. The projection surface corresponds to the liquid crystal panel 32. The volume data V shown in FIGS. 11A and 11B is a y-z cross section including the xy plane position of the liquid crystal panel 32 determined by the plane position determination unit 45, that is, y including the projection plane position T. Shown as -z cross section. The central line of sight L that is orthogonal to the projection plane is equal to the z direction that is orthogonal to the liquid crystal panel 32.

  When the projection plane position T is determined by the plane position determination unit 45, the depth determination image generation unit 47A arranges one cursor image B1 among the plurality of voxels constituting the volume data V. At that time, as shown in FIG. 11A, the depth determination image generation unit 47A calculates the luminance and color of the voxel on the projection plane side of the cursor image B1 and the luminance and color of the voxel on the opposite side of the projection plane. Volume data is generated so that at least one of them is different.

  Further, when the position of the cursor image is changed on the central line of sight L by the depth changing unit 49, the ratio of the high luminance area and the low luminance area changes as shown in FIG.

  FIGS. 12A and 12B are diagrams for explaining a second example of a display method of rendering images with a plurality of lines of sight.

  12A and 12B show the volume data V and the projection plane orthogonal to the central line of sight L of the nine lines of sight. The projection surface corresponds to the liquid crystal panel 32. Volume data V shown in FIGS. 12A and 12B is a y-z cross section including the xy plane position of the liquid crystal panel 32 determined by the plane position determination unit 45, that is, y including the projection plane position T. Shown as -z cross section. The central line of sight L that is orthogonal to the projection plane is equal to the z direction that is orthogonal to the liquid crystal panel 32.

  When the projection plane position T is determined by the plane position determination unit 45, the depth determination image generation unit 47A arranges one cursor image B1 among the plurality of voxels constituting the volume data V. At that time, as shown in FIG. 12A, the depth determination image generation unit 47A calculates the luminance and color of the voxel on the plane including the cursor image B1 and orthogonal to the z direction, and the luminance and color of the other voxels. Volume data is generated so that at least one of them is different.

  Further, when the position of the cursor image is changed on the center line of sight L by the depth changing unit 49, the position in the direction of the center line of sight L in the high luminance area changes as shown in FIG.

  Needless to say, the 3D display processing system 1 of the first embodiment may be combined with the 3D display processing system 1A of the second embodiment. In that case, in the depth changing unit 49, the observer operates the operation device 10 while viewing the subject image O stereoscopically viewed based on the nine line-of-sight rendering images generated by the depth determination image generating unit 47. Thus, according to the operation signal transmitted from the data transmitting / receiving unit 12 of the controller device 10, the position of the cursor image is switched between the four cursor images B extracted by the part extracting unit 46.

  (Operation method of the operation device 10)

  In the 3D display processing system 1, 1 </ b> A of the present embodiment, the observer performs an operation for determining the xy plane position on the liquid crystal panel 32 using the operation device 10 and the z direction passing through the determined xy plane position. An example of an operation method for changing / determining the position will be described.

  For example, the operating device 10 includes a position sensor. In that case, when the observer operates the displayed cursor image with the operation device 10, the 3D display processing systems 1 and 1A change the xy plane position and the z-direction position of the cursor image. Specifically, when the observer moves the operating device 10 in the horizontal direction of the operating device 10 while pressing the button 11a of the operating device 10, the cursor image is changed in the x direction, and the operating device 10 is operated while pressing the button 11a. When the device 10 is moved in the vertical direction, the cursor image is changed in the y direction, and when the operation device 10 is moved in the axial direction of the operation device 10 while pressing the button 11a, the cursor image is changed in the z direction. Then, the movement amount of the controller device 10 is reflected as the change amount of the cursor image. Note that the above operation method shows a simple example, and of course, complex movement is possible.

  Further, in the 3D display processing system 1A of the second embodiment, for example, the operation device 10 includes an acceleration sensor (a three-dimensional motion sensor, not shown) that detects an operation such as swinging or tilting itself. In that case, when the observer operates the displayed cursor image with the operation device 10, the 3D display processing system 1A changes the z-direction position of the cursor image. Specifically, when the observer moves the operation device 10 in the y direction while dragging the button 11a of the operation device 10, the cursor image B1 is quickly changed in the z direction, and the operation device 10 is moved to the x direction while dragging the button 11a. When moved in the direction, the cursor image B1 is slowly changed in the z direction.

  (Display layout of 3D display device 30)

  In the 3D display processing systems 1 and 1A of the present embodiment, an example of a display method for rendering images with a plurality of lines of sight will be described.

  FIG. 13 is a diagram for describing a first example of a display layout of a rendering image having a plurality of lines of sight.

  The liquid crystal panel 32 of the 3D display device 30 shown in FIG. 13 has a 3D display region R3 and a 2D display region R2. As described in the 3D display processing systems 1 and 1A, the rendering images of the plurality of lines of sight generated by the plane position determination image generation unit 44 and the depth determination image generation units 47 and 47A are displayed in the 3D display region R3. The The image generation units 44, 47, and 47A display the nine line-of-sight rendering images by assigning them to the nine picture elements of each pixel P in the 3D display region R3. That is, the image generation units 44, 47, and 47A give nine pixel values corresponding to nine rendering images to the nine picture elements constituting each pixel P of the 3D display device 30, respectively.

  On the other hand, command information that presents information related to the rendered image (such as imaging conditions) is displayed in the 2D display area R2. The image generation units 44, 47, and 47A provide pixel values (data signals) to be given to the respective pixels of the normal 2D display device to the respective pixels P of the 3D display device 30 in the 2D display region R2. That is, the image generation units 44, 47, 47A give the same pixel value to the nine picture elements constituting each pixel P in the 2D display region R2.

  FIG. 14 is a diagram for explaining a second example of the display layout of a rendering image with a plurality of lines of sight.

  The liquid crystal panel 32 of the 3D display device 30 shown in FIG. 14 has a 3D display region R3 and 2D display regions R21, R22, R23. As described in the 3D display processing systems 1 and 1A, the rendering images of a plurality of lines of sight generated by the image generation units 44, 47, and 47A are displayed in the 3D display region R3. The image generation units 44, 47, and 47A display the nine line-of-sight rendering images by assigning them to the nine picture elements of each pixel P in the 3D display region R3. That is, the image generation units 44, 47, and 47A give nine pixel values corresponding to nine rendering images to the nine picture elements constituting each pixel P of the 3D display device 30, respectively.

  The image generation units 44, 47, and 47A perform processing such as MPR (Multi Planar Reconstruction) and MIP (Maximum Intensity Projection) on the volume data to acquire an image such as an MPR image. The MPR image is displayed in the 2D display area R21. In the 2D display area R21, the image generation units 44, 47, and 47A give the same pixel value to the nine picture elements constituting each pixel P, as in the case of displaying the command information in FIG.

  Command information that presents information related to the rendered image is displayed in the 2D display area R22. In the 2D display area R22, the image generation units 44, 47, and 47A give the same pixel value to the nine picture elements constituting each pixel P in the same manner as the display of command information in FIG.

  Command information that presents information related to the MPR image is displayed in the 2D display area R23. In the 2D display area R23, the image generation units 44, 47, and 47A give the same pixel value to the nine picture elements constituting each pixel P in the same manner as the display of command information in FIG.

  As described above, according to the 3D display processing system 1, 1 </ b> A of the present embodiment, the point of interest can be three-dimensionally shown to the discussion partner or the like on the image stereoscopically viewed by the 3D display device 30.

  The 3D display processing systems 1 and 1A of the present embodiment are described in order to facilitate understanding of the present invention, and are not described in order to limit the present invention. Therefore, each element disclosed in the 3D display processing systems 1 and 1A of the present embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  For example, in the 3D display processing systems 1 and 1A of the present embodiment, the configuration in which the lenticular lens type naked eye type is employed has been described. However, the 3D display processing systems 1 and 1A are not limited to that case, and may be configured to employ a parallax barrier type naked eye type. The technical idea of the 3D display processing systems 1 and 1A of the present embodiment can also be applied to a configuration that employs a spectacle type (an anaglyph method, a frame sequential method, a polarization method, etc.) that includes dedicated glasses.

1,1A 3D display processing system 10 operation device 20 image processing device 21 processing unit 30 3D display device 33 convex lens array 41 volume data acquisition unit 42 reference image generation unit 43 mode determination unit 44 plane position determination image generation unit 45 plane position determination Unit 46, 46A Site extraction unit 47, 47A Depth determination image generation unit 48 Depth determination unit 49 Depth change unit L Center line of sight T Projection plane position

Claims (15)

First image generation means for performing rendering processing on volume data including a subject image to generate first three-dimensional image data corresponding to a plurality of lines of sight;
An operating means for an operator to set and operate the position on the projection plane orthogonal to the central line of sight of the plurality of lines of sight and the position of the direction of the central line of sight
Second image generation means for generating second three-dimensional image data corresponding to a plurality of lines of sight in which a cursor image is arranged based on a position on the projection plane;
Display means for performing 3D display based on the first three-dimensional image data corresponding to the plurality of lines of sight or the second three-dimensional image data corresponding to the plurality of lines of sight;
3D display processing system.   The operation means includes a parallax angle mode of the plurality of visual lines for generating the first three-dimensional image data, and a parallax angle mode of the plurality of visual lines for generating the second three-dimensional image data. The 3D display processing system according to claim 1, wherein the 3D display processing system is configured to be switchable.   The 3D display process according to claim 2, wherein the second image generation unit is configured to generate the second three-dimensional image data in a mode in which a parallax angle of the plurality of lines of sight is large among the two modes. system. The operation means is capable of changing the position on the projection plane and changing the direction of the central line of sight,
The second image generation means generates second 3D image data corresponding to a plurality of lines of sight in which the cursor image after the change operation is arranged,
4. The configuration according to claim 1, wherein the operation unit is configured to determine a position of the cursor image based on second 3D image data corresponding to a plurality of lines of sight in which the cursor image after the change operation is arranged. The 3D display processing system according to claim 1. The operating means has a position sensor and a button,
The 3D display processing system according to claim 4, wherein when the operation unit is moved while the button is pressed, the movement amount is reflected as a change amount of the cursor image.   The second image generation means includes second 3D image data corresponding to the plurality of lines of sight in which a cursor image is arranged at one of a plurality of edge portions on the central line of sight in the subject image. The 3D display processing system according to claim 5, wherein the 3D display processing system is configured to generate an image.   The second image generation unit generates the second three-dimensional image data in which at least one of the luminance and color on the projection plane side of the cursor image is different from the luminance and color on the opposite side of the projection plane. The 3D display processing system according to any one of claims 1 to 6, which is configured.   The second image generation means includes the second three-dimensional image data in which at least one of luminance and color of a surface including the cursor image and orthogonal to the direction of the central line of sight differs from other luminance and color. The 3D display processing system according to any one of claims 1 to 6, wherein the 3D display processing system is configured to generate an image. The second image generation means generates second three-dimensional image data corresponding to the plurality of lines of sight in which a cursor image is arranged at each of a plurality of parts on the central line of sight in the subject image;
The 3D display processing system according to claim 1, wherein the operation unit is configured to determine a required cursor image from the plurality of cursor images.   10. The 3D display processing system according to claim 4, wherein when the position of the cursor image is confirmed, the operation unit outputs a confirmation completion.   The 3D display processing system according to claim 10, wherein the operation unit is configured to vibrate when the position of the cursor image is determined.   The 3D display processing system according to any one of claims 4 to 9, wherein the display unit is configured to change a color of the cursor image when the position of the cursor image is determined.   The display means is configured to perform the 3D display of the first 3D image data corresponding to the plurality of lines of sight or the second 3D image data corresponding to the lines of sight, and the 2D display of command information in parallel. The 3D display processing system according to any one of claims 1 to 12.   The display means includes a 3D display of the first three-dimensional image data corresponding to the plurality of lines of sight or a second three-dimensional image data corresponding to the plurality of lines of sight, and a 2D display of a two-dimensional image based on the volume data. The 3D display processing system according to claim 1, wherein the 2D display of command information is performed in parallel.   The 3D display processing system according to claim 14, wherein the display unit is configured to perform 2D display of an MPR (Multi Planar Reconstruction) image as a two-dimensional image based on the volume data.