JP7013849B2 - Computer program, image processing device and image processing method - Google Patents

Description

The present disclosure relates to computer programs, image processing devices and image processing methods .

In the medical field, it is continuously photographed at predetermined slice intervals by medical diagnostic imaging equipment such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging), and PET (Positron Emission Tomography), and DICOM (Digital Imaging and COmmunications in Medicine). ) A technique has been developed for three-dimensionally visualizing an observation object such as an organ based on a two-dimensional tomographic image stored in the format to be useful for diagnostic imaging support, surgery or treatment support.

A volume rendering method is known as a typical method for displaying a three-dimensional image of the internal structure of an observation object. Ray casting and texture-based rendering are known as volume rendering methods. In Patent Document 1, a plurality of tomographic images are stacked to construct a three-dimensional voxel space, and a ray (virtual ray) is applied to the three-dimensional voxel space from an arbitrary viewpoint direction at a sampling point on the line of sight. Raycasting is disclosed that cumulatively calculates color and brightness values along the line of sight. Specifically, coordinate transformation is performed so that the viewpoint direction is parallel to the Z axis of the three-dimensional coordinate axis, and a ray is applied to the three-dimensional boxel space after the coordinate transformation in the Z-axis direction on the XY plane. The projection result is calculated as a pixel value on the projection surface (screen).

Further, Patent Document 2 discloses texture-based rendering using the polygon display function of graphics hardware by expressing the entire volume data by stacking slice data using a texture mapping function and an alpha blending function. ing.

Japanese Unexamined Patent Publication No. 11-175743 Japanese Patent No. 3589654

The present disclosure has been made in view of such circumstances, and clarifies a computer program, an image processing device, and an image processing method capable of suppressing moire.

The present disclosure has been made in view of such circumstances, and clarifies a computer program, an image processing device, an image processing method, and voxel data capable of suppressing moire.

The present application includes a plurality of means for solving the above problems, and to give an example thereof, a computer program causes a computer to obtain image data of a plurality of xy plane images captured at predetermined intervals along the z-axis direction. The process of acquiring, the process of generating a voxel structure composed of voxels having RGB values and opacity corresponding to each pixel of each of the plurality of xy plane images, and the opacity of the voxel structure are A process of correcting the RGB value of the target voxel based on the RGB value of the neighboring voxel located in the vicinity of the target voxel with respect to the target voxel having a predetermined value, and the voxel of the voxel structure in which the RGB value is corrected. A process of generating a 3D texture image composed of, a process of performing a predetermined conversion on the 3D texture image to generate a converted 3D texture image, and a Z-axis of a quadrangle on the XY coordinate plane in the three-dimensional space. The process of setting stacked voxels arranged in a direction and the RGB value of the voxel of the converted 3D texture image corresponding to the XY coordinates of the voxel on the line of sight parallel to the Z-axis direction from a predetermined viewpoint are the opacity of the voxel. The RGB values obtained by alpha blending in the order of the squares far from the viewpoint based on the above are executed as the pixel values of the pixels whose line of sight intersects the projection plane to generate a rendered image.

According to the present disclosure, moire can be suppressed.

It is a block diagram which shows an example of the structure of the ray casting apparatus as the image processing apparatus of this embodiment. It is a schematic diagram which shows an example of the tomographic image of DICOM format. It is explanatory drawing which shows the 1st example of the color map data of this embodiment. It is explanatory drawing which shows the 2nd example of the color map data of this embodiment. It is explanatory drawing which shows an example of the gradation compression processing by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows an example of the smoothing process by the 3D texture mapping apparatus of this embodiment. It is a schematic diagram which shows the concept of 3D texture mapping by the 3D texture mapping apparatus of this embodiment. It is a flowchart which shows an example of the procedure of the 3D texture mapping processing by the 3D texture mapping apparatus of this embodiment. It is a flowchart which shows an example of the procedure of the 3D texture mapping processing by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows an example of the transparency processing of the voxel outside the ROI clipping area by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows an example of the dummy transparent image addition processing by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows an example of the transparent processing of the voxel of the outer edge part of the tomographic image by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows an example of the projection screen setting by the 3D texture mapping apparatus of this embodiment. It is a flowchart which shows an example of the procedure of the color interpolation processing of a transparent voxel by the 3D texture mapping apparatus of this embodiment. It is a flowchart which shows an example of the procedure of the rendering process by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows an example of the correspondence method with 3D texture mapping. It is a block diagram which shows the other example of the structure of the image processing part of this embodiment. It is a schematic diagram which shows an example of how the curved moire occurs. It is explanatory drawing which shows the 1st example of the moire measures result by the 3D texture mapping apparatus of this embodiment. It is a schematic diagram which shows an example of the appearance of the stripe / grid-like moire. It is explanatory drawing which shows an example of the appearance of the rounding error at the time of coordinate conversion. It is a schematic diagram which shows an example of the state of generation of a periodic pattern by coordinate transformation. It is a schematic diagram which shows an example of the state of the color interpolation by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows the 2nd example of the moire measures result by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows the 3rd example of the moire measures result by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows the 4th example of the moire measures result by the 3D texture mapping apparatus of this embodiment. It is explanatory drawing which shows the 5th example of the moire measures result by the 3D texture mapping apparatus of this embodiment.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of the 3D texture mapping device 100 as the image processing device of the present embodiment. The 3D texture mapping device 100 of the present embodiment includes an input unit 10, a storage unit 11, an output unit 12, an image processing unit 20, and the like. The image processing unit 20 includes a gradation compression processing unit 21, a smoothing processing unit 22, a voxel structure generation unit 23, a conversion processing unit 24, a rendering processing unit 25, a 3D texture image generation unit 26, a laminated square setting unit 27, and a color. It includes an interpolation unit 28, a ROI (Region of Interest) clipping processing unit 29, and the like.

The input unit 10 has a function as an acquisition unit, and acquires image data of a plurality of xy plane images captured at predetermined intervals along the z-axis direction. More specifically, the input unit 10 is a plurality of tomographic images taken at predetermined intervals with respect to an observation object (for example, various parts of the human body / animal including organs, brains, bones, blood vessels, etc.). Acquire image data in DICOM (Digital Imaging and COmmunications in Medicine) format of a two-dimensional tomographic image.

FIG. 2 is a schematic diagram showing an example of a DICOM format tomographic image. When the xyz axis is set as shown in FIG. 2, a plurality of two-dimensional tomographic images, which are xy plane images, are arranged along the z-axis direction. Sx is the number of pixels in the x-axis direction, Sy is the number of pixels in the y-axis direction, and Sz is the number of tomographic images. Note that Sz represents the number of pixels in the z-axis direction. Rxy is the resolution in the x-direction and the y-direction, and indicates the reciprocal of the pixel spacing, that is, the number of pixels per unit distance. The resolutions in the x-axis direction and the y-axis direction are the same, but may be different. Rz is the resolution in the z-axis direction, and represents the reciprocal of the interval between adjacent tomographic images, that is, the number of images per unit distance. The pixel value at the coordinates (x, y, z) of each tomographic image is represented by Do (x, y, z). Do (x, y, z) can take a value of -32768 or more and 32767 or less. For example, in the case of a CT image, the pixel value Do (x, y, z) shows 0 for water, a tissue having a density higher than water (for example, bone) has a positive value, and a tissue having a density lower than water. (For example, adipose tissue) has a negative value. In the coordinates (x, y, z) of the tomographic image, the x coordinate takes a value of 0 or more and Sx-1 or less, the y coordinate takes a value of 0 or more and Sy-1 or less, and the z coordinate takes a value of 0 or more and Sz. Takes a value less than or equal to -1.

Further, the input unit 10 acquires color map data in which RGB values and opacity are defined in association with pixel values.

FIG. 3 is an explanatory diagram showing a first example of the color map data of the present embodiment. The color map data of the first example is shown by Cmap (v, c). The variable v indicates a pixel value of Do (x, y, z), and represents a numerical value within the range of -23768 or more and 32767 or less, similarly to the range of Do (x, y, z). The variable c includes the values of 0, 1, 2, and 3, c = 0 indicates the R value of RGB, c = 1 indicates the G value, and c = 2 indicates the B value. c = 3 indicates the opacity α. Cmap (v, c) can take a numerical value of 0 or more and 255 or less. As shown in FIG. 3, the color map data defines the relationship between the pixel value Do (x, y, z) of the sliced image, the RGB value, and the opacity α. For example, when the pixel value Do (x, y, z) is 32767, the R value is R (32767), the G value is G (32767), the B value is B (32767), and the opacity is α (32767). ). Here, R (32767), G (32767), B (32767), and α (32767) indicate for convenience that they correspond to the pixel value 32767, and are actually appropriate values. Further, the color map data does not need to be defined for all the pixel values from -32768 to 32767, and in the case of a CT image, it is usually adjusted to take the range of -2048 to 2048. The RGB values and opacity in the range from 2048 to 2048 and from 2048 to 32767 may all be set to 0.

FIG. 4 is an explanatory diagram showing a second example of the color map data of the present embodiment. The color map data of the second example is shown by Cmap8 (v, c). As will be described later, when the DICOM image is gradation-compressed from, for example, 16 bits to 8 bits, the pixel value can be in the range of 0 or more and 255 or less from the range of 32768 or more and 32767 or less. The variable v of Cmap8 (v, c) is a numerical value within the range of 0 or more and 255 or less. The variable c is the same as in the first example.

By using the color map data Cmap (v, c) and Cmap8 (v, c), for example, not only different colors can be added to each part of the human body, but also the opacity can be set for each part. It is possible to make it easier to recognize the observation target, such as making the located organ transparent and depicting the organ hidden in the back.

Further, the input unit 10 has a rotation angle centered on the x-axis, a rotation angle centered on the y-axis, a rotation angle centered on the z-axis, an offset value in the xyz-axis direction, an enlargement / reduction magnification in the xyz-axis direction, and a variable magnification in the z-axis direction. , And the parameters of coordinate conversion including the distance from the gazing point to the viewpoint, the value of ROI (Region of Interest) in the xyz direction, the perspective conversion parameters, and the like are acquired. Details of the coordinate conversion parameters, ROI value, and perspective conversion parameters will be described later.

The storage unit 11 can store the data acquired by the input unit 10, the processing result of the image processing unit 20, and the like. The color map data, the coordinate conversion parameter, the ROI value, the perspective conversion parameter, and the like may be stored in the storage unit 11 in advance.

The output unit 12 outputs the processing result of the image processing unit 20, for example, the image data of the rendered image generated by the rendering processing unit 25 to the display device (not shown).

The gradation compression processing unit 21 converts a DICOM format tomographic image into, for example, an 8-bit gradation compression tomographic image.

FIG. 5 is an explanatory diagram showing an example of gradation compression processing by the 3D texture mapping device 100 of the present embodiment. The gradation compression processing unit 21 specifies a minimum value and a maximum value of pixels of a predetermined tomographic image among a plurality of tomographic images. For example, among the tomographic images from 1 to Sz, the minimum value Dmin and the maximum value Dmax of all the pixels in the Sz / second intermediate tomographic image can be specified. As the predetermined slice image, the method of specifying the minimum value and the maximum value of all the pixels in all the slice images is originally accurate. However, in that case, it becomes necessary to temporarily hold all the large-capacity 16-bit DICOM format tomographic images in the memory, and the gradation compression effect is halved. Further, the minimum value Dmin and the maximum value Dmax are only used as parameters for gradation compression, and do not directly affect the accuracy of the 8-bit gradation compression tomographic image. Therefore, a method of roughly specifying is taken by using the minimum value and the maximum value of the pixels of a single slice image. However, since the subject is often not properly reflected in the first slice image, a method of specifying the minimum value Dmin and the maximum value Dmax is adopted only in the intermediate slice image. As a result, it is possible to directly construct only an 8-bit gradation compressed tomographic image in the memory without holding a large-capacity 16-bit DICOM format tomographic image in the memory.

The 8-bit gradation compressed tomographic image D8 (x, y, z) is based on the formula D8 (x, y, z) = (Do (x, y, z) -Lmin) · 255 / (Lmax-Lmin). Can be generated. However, when D8 (x, y, z)> 255, D8 (x, y, z) = 255, and when D8 (x, y, z) <0, D8 (x, y, z). = 0. Here, Lmin = (Dmax-Dmin) · γ + Dmin, Lmax = (Dmax−Dmin) · (1-γ) + Dmin. γ is the contrast adjustment range of the gradation compressed image, and the closer it is to 0, the higher the contrast but the lower the brightness. Normally, it is set to γ = 0.1. Since the luminance contrast of the rendered image can be adjusted by various settings such as color map data, γ may be a fixed value. Further, 0 ≦ D8 (x, y, z) ≦ 255, 0 ≦ x ≦ Sx-1, 0 ≦ y ≦ Sy-1, and 0 ≦ z ≦ Sz-1. The resolution in the xy direction is Rxy, and the resolution in the z direction is Rz.

The color map Cmap8 (v, c) shown in FIG. 4 can be applied to the 8-bit gradation compressed tomographic image D8 (x, y, z).

As described above, the gradation compression processing unit 21 specifies the minimum value Dmin and the maximum value Dmax of the pixels of the predetermined xy plane image among the plurality of xy plane images, and the upper limit value Lmax smaller than the specified maximum value. And the lower limit Lmin larger than the specified minimum value is calculated. The gradation compression processing unit 21 compresses within the range of the upper limit value Lmax and the lower limit value Lmin of the pixel value of each pixel of the plurality of xy plane images. That is, the gradation compression processing unit 21 compresses the range of the upper limit value Lmax and the lower limit value Lmin into, for example, 256 steps, sets the upper limit value Lmax or more to 255, and sets the lower limit value Lmin or less to 0.

The smoothing processing unit 22 smoothes the one xy plane image based on the pixel values of an arbitrary pixel of interest on one xy plane image of each of the plurality of xy plane images and pixels in the vicinity of the pixel of interest in the xyz direction. To become. The xy plane image may be a DICOM image Do (x, y, z) or a gradation compression tomographic image D8 (x, y, z) that is gradation-compressed by the gradation compression processing unit 21.

FIG. 6 is an explanatory diagram showing an example of smoothing processing by the 3D texture mapping device 100 of the present embodiment. In the example of FIG. 6, the gradation compressed tomographic image D8 (x, y, z) is shown, but it may be a DICOM image Do (x, y, z). FIG. 6 shows a pixel of interest D8 (x, y, z) of one tomographic image and 26 nearby pixels D8 (x + i, y + j, z + k) existing in the vicinity of the pixel of interest. Here, i = -1, 0, 1, j = -1, 0, 1, and k = -1, 0, 1. For convenience, three tomographic images (z-1, z, z + 1) are shown in a plane. Assuming that the smoothed image of the xyz coordinates is D (x, y, z), D (x, y, z) is the average value of the pixel values of the pixel of interest and the pixels in the vicinity of 26 of the pixel of interest.

There are patterns in tomographic images that do not have perceptible regularity that are naturally present. By performing coordinate transformation (rotation around the xyz axis) of a plurality of tomographic images, the above-mentioned patterns existing in the tomographic images are deviated and synthesized, so that these patterns are linearly connected, and as a result, the patterns are linearly connected. A curved striped pattern is formed on the volume rendered image, resulting in moire.

The average value of the pixel values of the 26 neighborhood pixels in the xyz direction of the gradation compressed tomographic image D8 (x, y, z) or the DICOM image Do (x, y, z) and the pixel of interest is isotropic (meaning in the xyz direction). ) By performing the smoothing process, it is possible to suppress the occurrence of moire due to the above-mentioned curved striped pattern. In particular, it is possible to suppress the occurrence of moire on CT images with coarse slice intervals.

The voxel structure generation unit 23 generates a voxel structure composed of voxels in which RGB values and opacity α are set for each voxel and corresponding to each pixel of each of a plurality of smoothed xy plane images. More specifically, the voxel structure generation unit 23 associates the RGB values and opacity of the color map data corresponding to the pixel values of each pixel of each of the smoothed plurality of xy plane images with the RGB values and opacity. Set α to generate a voxel structure. That is, the voxel structure has a structure (also referred to as a volume) in which voxels are three-dimensionally packed. Each voxel has a voxel value (RGB value and opacity α).

The voxel structure is represented by V (x, y, z, c). Here, 0 ≦ V (x, y, z, c) ≦ 255, c = 0 (R), 1 (G), 2 (B), 3 (α), 0 ≦ x ≦ Sx-1, 0 ≦ y ≦ Sy-1 and 0 ≦ z ≦ Sz-1.

Further, the voxel structure generation unit 23 generates a voxel structure including voxels having RGB values and opacity as predetermined values. As predetermined values, the RGB values are R = G = B = 0 (that is, colorless or black) and the opacity α = 0 (that is, transparent), but these do not contribute to the calculation of the volume rendering image of the voxel. It does not have to be limited to R = G = B = 0 and opacity α = 0 as long as it means that it is a value and does not contribute to the calculation of the volume rendering image.

The color interpolation unit 28 has a function as a color correction unit, and is based on the RGB value of the voxel structure and the RGB value of the neighboring voxel located in the vicinity of the target voxel with respect to the target voxel whose opacity is a predetermined value. Then, color interpolation is performed to correct the RGB value of the target voxel. Note that color interpolation is also referred to as color correction. The details of color interpolation will be described later.

The 3D texture image generation unit 26 generates a 3D texture image composed of voxels of a voxel structure in which RGB values are color-interpolated. That is, the 3D texture image generation unit 26 generates a 3D texture image based on the voxels generated by the voxel structure generation unit 23 (including voxels having RGB values and opacity as predetermined values). The data of the 3D texture image is also referred to as a 3D texture map, and the substance of the data is the same as the voxel structure. The 3D texture image generation unit 26 registers the volume of the voxel structure as a 3D texture map of a graphic API (for example, OpenGL). More specifically, the volume of the boxel structure is transferred from the main memory of the CPU to the video memory of the GPU via the graphic API, converted into the texture format of the graphic API, and held in the video memory. Hereinafter, for convenience of explanation, the content of the process will be described for OpenGL as a graphic API, but the present application is not limited thereto.

The 3D texture map is represented by Tex (xt, yt, zt, c). Here, 0≤Tex (xt, yt, zt, c) ≤255, c = 0 (R), 1 (G), 2 (B), 3 (α), 0≤xt≤Sx-1, 0≤ yt ≦ Sy-1, 0 ≦ zt ≦ Sz-1.

The coordinate system of the boxel structure is defined by the OpenGL world coordinate system, and is represented by XYZ coordinates (-1≤x≤1, -1≤y≤1, -1≤z≤1), and the coordinates of the 3D texture map. The system is defined by the texture coordinate system, and in OpenGL, it is customary to express the texture coordinate system by STR coordinates (0 ≦ s ≦ 1, 0 ≦ t ≦ 1, 0 ≦ r ≦ 1). The coordinate s corresponds to the coordinate x, the coordinate t corresponds to the coordinate y, and the coordinate r corresponds to the coordinate z. In the present embodiment, for convenience, the XYZ of the world coordinate system is represented by the XYZ coordinates, and the STR coordinates of the texture coordinate system are represented by the xyz coordinates.

The laminated quadrangle setting unit 27 sets a laminated quadrangle in which a plurality of quadrangles defined in the XY coordinate plane are arranged by shifting them by a predetermined interval in the Z-axis direction in the world coordinate system. A laminated quadrangle is a quadrangle, each defined by four vertices in three-dimensional space.

The conversion processing unit 24 has a function as a 3D texture image generation unit after conversion, and performs predetermined conversion in the texture coordinate system for a 3D texture image composed of a plurality of slice images defined in the texture coordinate system. After conversion, a 3D texture image is generated. The predetermined transformation includes a perspective transformation process, a coordinate transformation process, and the like.

FIG. 7 is a schematic diagram showing the concept of 3D texture mapping by the 3D texture mapping device 100 of the present embodiment. FIG. 7A shows a state in which the 3D texture map before coordinate conversion is mapped to the laminated quadrangle, and FIG. 7B shows a state in which the 3D texture map after coordinate conversion is mapped to the laminated quadrangle. In world coordinates (X, Y, Z), a plurality of quadrilaterals on the XY coordinate plane are arranged along the Z-axis direction. The viewpoint is located above the projection plane (the plane on which the rendered image is generated) (that is, in the positive direction of the Z axis) in FIG. 7. Since the stacked quadrilaterals defined in the world coordinate system are fixed, each quadrangle is always perpendicular to the line of sight. In FIG. 7, the elliptical shape shown inside the laminated quadrangle indicates a mapped 3D texture map (voxel data). The 3D texture map is appropriately converted in the texture coordinate system, but since the correspondence (mapping) between the world coordinate system and the texture coordinate system is fixed, it is mapped to the laminated square after the conversion of the 3D texture map. The contents of the existing 3D texture map are automatically changed.

As shown in FIG. 7A, in 3D texture mapping, voxel data is defined as a 3D texture map in the texture coordinate system, and the 3D texture map is pasted on each of the laminated squares defined in the world coordinate system. That is, by mapping each cross section (slice) of the 3D texture map to the quadrangle, the 3D texture map is reconstructed as a collection of cross sections perpendicular to the line of sight.

As shown in FIG. 7B, the viewpoint is changed by pasting the laminated quadrangle defined in the world coordinates (X, Y, Z) and the correspondence between the world coordinate system and the texture coordinate system on the quadrangle without moving the initial state. Rotate the attached 3D texture map in the texture coordinate system. As a result, the four vertices of each quadrangle and the coordinate positions of the 3D texture map (represented by xyz coordinates in FIG. 7) are changed by rotation, and an image with a changed viewpoint can be obtained.

The rendering processing unit 25 bases the RGB values of the voxels of the converted 3D texture image corresponding to the XY coordinates of the voxels on the line of sight parallel to the Z-axis direction from the predetermined viewpoint in the order of the voxels far from the viewpoint. The RGB value obtained by alpha blending is used as the pixel value of the pixel whose line of sight intersects the projection plane to generate a rendered image. In this case, the rendering processing unit 25 calculates the intersection coordinates (XYZ coordinates) of the rectangle that intersects the XY coordinates of the rendered image in the line-of-sight direction, and calculates the xyz coordinates of the converted 3D texture image corresponding to the calculated intersection coordinates. , Alpha blending based on the RGB value and opacity of the boxel corresponding to the calculated xyz coordinates. That is, volume rendering is performed by alpha blending the quadrangle to which the 3D texture map is pasted in the order of distance from the viewpoint. By instructing these procedures using a graphic API such as OpenGL, high-speed processing using the GPU function becomes possible. By using the GPU function, it is possible to suppress the waiting time and generate a rendered image at high speed. In the present embodiment, even a GPU mounted on a video card mounted on a general-purpose computer can perform rendering processing within 0.1 seconds, and can be rendered interactively (or ensure interactivity). ) The rendered image can be displayed in real time.

As described above, the voxel data of the voxel structure of the present embodiment corresponds to each pixel of a plurality of xy plane images captured at predetermined intervals along the z-axis direction, and has RGB values and opacity. It has a structure including voxel data of a voxel structure composed of defined voxels, and the voxel data is a target voxel for a target voxel whose RGB value and opacity are predetermined values among the voxels of the voxel structure. A predetermined 3D texture image composed of voxels of a voxel structure, which includes data in which the RGB values of the target voxels are color-interpolated (color-corrected) based on the RGB values of neighboring voxels located in the vicinity of. Processing to generate a 3D texture image after conversion by performing conversion, processing to set a laminated quadrangle in which quadrangles on the XY coordinate plane of the three-dimensional space are arranged in the Z-axis direction, and processing parallel to the Z-axis direction from a predetermined viewpoint The line of sight projects the RGB values obtained by alpha blending the RGB values of the voxels of the converted 3D texture image corresponding to the XY coordinates of the squares on the line of sight based on the alpha values of the voxels in the order of the voxels far from the viewpoint. It is used to execute a process of generating a rendered image as a pixel value of a pixel intersecting with.

In this embodiment, the 3D texture mapping method is used, but in the case of a general 2D texture mapping method, there are the following problems. That is, in the 2D texture mapping method, the contour shape of each slice constituting the voxel data is expressed by a square in the world coordinate system, which is the same as in the present application, but each slice constituting the voxel data in the two-dimensional texture coordinate system. The difference is that the image of the above is defined as a plurality of 2D texture maps (in the present embodiment, the images of all slices are defined as a single 3D texture map in the three-dimensional texture coordinate system). Then, one of the 2D texture maps corresponding to each quadrangle is pasted to each quadrangle (in the present embodiment, the four coordinates in the 3D texture map corresponding to the four vertices of each quadrangle are associated and pasted. ). Since the 2D texture map is defined in the two-dimensional texture coordinate system, rotation can only be done two-dimensionally. Therefore, in the world coordinate system, there is no choice but to take a method of rotating the stacked quadrangles three-dimensionally. Then, the depth relationship is broken depending on the rotation angle, or the 2D texture map is inverted, and when the stacked quadrangles are rotated 90 degrees around the center of the Y axis, the quadrangle (thickness is 0) is not drawn and the rendered image becomes black.

In addition, as the angle between the slice and the line of sight decreases, the projected area decreases, the number of data sampled decreases, the significance of volume rendering is lost, and a practical rendered image may not be obtained. .. That is, an appropriate rendered image cannot be obtained unless the correspondence between the stacked quadrangle of the world coordinate system and the 2D texture map of the texture coordinate system is corrected by the rotation angle.

According to this embodiment, it is possible to solve the problem in the 2D texture mapping method or the problem that occurs when the world coordinates are rotated.

Next, the operation of the 3D texture mapping device 100 of the present embodiment will be described.

8 and 9 are flowcharts showing an example of the procedure of the 3D texture mapping process by the 3D texture mapping device 100 of the present embodiment. Hereinafter, for convenience, the subject of processing will be described as the image processing unit 20. The image processing unit 20 acquires the image data of the DICOM image (S11) and acquires the color map data (S12). The color map data is illustrated in FIGS. 3 and 4.

The image processing unit 20 acquires the conversion parameter (S13) and acquires the ROI value in the xyz direction (S14). The conversion parameters include coordinate conversion parameters, perspective conversion parameters, and the like. Details of the conversion parameters will be described later. The image processing unit 20 performs gradation compression processing of the DICOM image (S15), and defines a color map for the gradation compression tomographic image D8 (S16). The color map definition defines the RGB value and the opacity α with respect to the pixel value of the gradation compressed tomographic image D8. The process of step S15 is not essential, and if the process of step S15 is not performed, the color map may be defined for the DICOM image. However, in the case of a CT image, the color map can be generally defined for each part to be viewed and is not defined depending on the given DICOM image. Therefore, the process of step S16 is omitted. A predefined color map can also be used.

The image processing unit 20 performs smoothing processing of the gradation compressed tomographic image D8 (S17) to generate a voxel structure V (x, y, z, c) (S18). The image processing unit 20 performs voxel transparency processing outside the ROI clipping region (S19).

FIG. 10 is an explanatory diagram showing an example of voxel transparency processing outside the ROI clipping region by the 3D texture mapping device 100 of the present embodiment. FIG. 10 illustrates an example of an xy plane image located at the z-th position. In clipping by ROI, Xs-Xe is set as the ROI in the x-axis direction, Ys-Ye is set as the ROI in the y-axis direction, and although not shown in FIG. 10, it is set as the ROI in the z-axis direction. Set Zs-Ze. Here, 0 ≦ Xs, Xe ≦ Sx-1, 0 ≦ Ys, Ye ≦ Sy-1, 0 ≦ Zs, and Ze ≦ Sz-1. That is, the ROI clipping processing unit 29 sets the xyz coordinates that define the target area of the volume rendering processing. The voxel structure generation unit 23 performs voxel transparency processing outside the ROI clipping region.

In the voxel transparency process, in the case of x <Xs or x> Xe or y <Ys or y> Ye or z <Zs or z> Ze, V (x, y, z, c) for 0 ≦ c ≦ 3 ) = 0, that is, R = G = B = α = 0 (that is, the color of the pixel is colorless or black and transparent). In the example of FIG. 10, the color of the pixel at the coordinates (x, y) satisfying x <Xs, x> Xe, y <Ys, and y> Ye is made colorless (black), and the opacity is set to α = 0 (transparent). ).

In step S24 described later, the color interpolation unit 28 is a voxel (opaque voxel) located in the vicinity of the target voxel with respect to a colorless / transparent voxel including a voxel that has been made transparent such as outside the ROI clipping region. Color-interpolates the RGB values of the target voxel based on the RGB values. However, even if it is a transparent voxel, if it is not transparently processed and a value of 1 or more is set for any of the RGB values (air in the lung field, etc.), color interpolation is not performed.

As described above, in the voxel data of the voxel structure of the present embodiment, the voxel structure is outside the target area of the volume rendering process, and the RGB value and the opacity are predetermined values for the target voxel. Contains data in which the RGB values of the target voxels are color-rendered based on the RGB values of the neighboring voxels located in the vicinity of the target voxels.

Not limited to the ROI clipping cut surface, the opaque area (the area where the opacity has a value of 1 or more) which is the target area of the volume rendering process and the transparent area (the opacity is 0) which is outside the target area of the volume rendering process. The interface with the region that has the value of) is the source of moire. (Details will be described later, but this is because a step-like pattern is generated at the boundary surface by coordinate transformation.) In the first place, the tomographic image has a background part other than the subject such as a human body and a photographing jig (sleeper, etc.). Generally, since the background part is colorless and transparent with air, the boundary surface with the opaque subject originally exists and can be a source of moire. It is necessary to perform color interpolation together.

Not limited to the ROI clipping cut surface, the first slice and / or the last slice of the tomographic image may be the boundary surface between the opaque region and the transparent region as it is, and the boundary surface may be the source of moire. There is no image data in front of the first slice of the tomographic image CT-scanned for the chest and abdomen, which are part of the human body, and behind the last slice, and volume rendering is performed as a colorless and transparent space. Therefore, it has the same characteristics as the ROI clipping cut surface. That is, the first slice and the last slice of the tomographic image inevitably become the boundary surface between the opaque region and the transparent region, and become a moire generation source. Therefore, by adding voxels to the regions before the first slice and after the last slice to make them colorless and transparent, and by performing color interpolation which is a subsequent process, it is possible to suppress the occurrence of moire. The details of color interpolation will be described later.

The image processing unit 20 newly adds a dummy transparent image (here, colorless and transparent) having predetermined RGB values and opacity to both ends in the z direction (S20), and the dummy transparent image is added to the voxel structure. Add a voxel corresponding to (S21). A voxel structure to which a voxel corresponding to a dummy transparent image is added is represented as V (x, y, z, c). Here, 0 ≦ x ≦ Sx-1, 0 ≦ y ≦ Sy-1, and 0 ≦ z ≦ Sz + 1.

FIG. 11 is an explanatory diagram showing an example of a dummy transparent image addition process by the 3D texture mapping device 100 of the present embodiment. A dummy transparent image in which the RGB values and opacity of all pixels are set to predetermined values (here, colorless and transparent) at both ends of the original tomographic image arranged at slice intervals in the z-axis direction. To add. That is, the voxel structure generation unit 23 corresponds to each pixel of the dummy xy plane image arranged at both ends in the z-axis direction with the plurality of xy plane images in between, and the RGB value and the opacity are set to predetermined values. Generate a voxel structure with a dummy voxel added. Specifically, the voxel structure generation unit 23 adds voxels corresponding to the dummy transparent image at both ends of the tomographic image in the z-axis direction. As predetermined values, the RGB value can be set to R = G = B = 0 and the opacity can be set to α = 0 (that is, colorless / transparent) so that the boxel does not contribute to the generation of the volume rendering image. The RGB value may not be limited to R = G = B = 0, and the opacity may not be limited to α = 0 as long as the value does not contribute to the generation of the volume rendering image.

In step S24 described later, the color interpolation unit 28 has the RGB value of the voxel of the generated voxel structure and the RGB value of the neighboring voxel located in the vicinity of the target voxel with respect to the target voxel whose opacity is a predetermined value. The RGB values of the target voxels are color-interpolated based on. However, even for transparent voxels, if any of the RGB values is set to a value other than a predetermined value (here, a value of 1 or more) (for example, air in the lung field), color interpolation is performed. Not performed.

As described above, in the voxel data of the voxel structure of the present embodiment, the voxel structure corresponds to each pixel of the dummy xy plane image arranged at both ends in the z-axis direction with a plurality of xy plane images in between. However, the voxel data includes a dummy voxel which is a voxel whose RGB value and opacity are predetermined values, and the voxel data is for a target voxel whose RGB value and opacity are predetermined values of the voxel structure to which the dummy voxel is added. , Contains data in which the RGB values of the target voxels are color-interpolated based on the RGB values of the neighboring voxels located in the vicinity of the target voxels.

Not limited to the ROI clipping cut surface, the peripheral portion of the tomographic image becomes the boundary surface between the opaque region and the transparent region as it is, and may become a source of moire. For example, depending on the magnification setting of the CT scan, the image may be taken in a state where the peripheral portion of the tomographic surface of the human body is cut off. In that case, x = 0, y in FIG. 10 without instructing ROI clipping. The surface satisfying any of = 0, x = Sx-1, y = Sy-1 is the outer region of the tomographic image of any one of x <0, y <0, x> Sx-1, y> Sy-1. Since there is no image data in the image and volume rendering is performed as a colorless and transparent space, it has the same characteristics as the ROI clipping cut surface. That is, a plane satisfying any one of x = 0, y = 0, x = Sx-1, and y = Sy-1 inevitably becomes a boundary surface between an opaque region and a transparent region, and becomes a moire generation source. Therefore, regardless of whether or not ROI clipping is instructed, for all voxels at the outer edge portion of one voxel width satisfying any of x = 0, y = 0, x = Sx-1, and y = Sy-1. Colorless and transparent as in the shaded area of FIG. 10, and color interpolation, which is a subsequent process, is also performed. That is, the image processing unit 20 performs voxel transparency processing corresponding to the pixels of the outer edge portion (also referred to as the border portion) of the gradation compressed tomographic image (S22). The process of step S22 is performed regardless of whether or not the process of step S23 described later is performed. The outer edge portion means a region in the image, not the outside of the image, which corresponds to the edge of the image.

FIG. 12 is an explanatory diagram showing an example of voxel transparency processing at the outer edge of the tomographic image by the 3D texture mapping device 100 of the present embodiment. The color of the voxels corresponding to the pixels at the outer edge of each tomographic image arranged at slice intervals in the z-axis direction is set to R = G = B = 0 (colorless), and the opacity α is set to 0 (transparent). That is, the voxel structure generation unit 23 makes the voxels corresponding to the pixels at the outer edge of the tomographic image colorless and transparent. Here, the outer edge portion is, for example, a region in the tomographic image and can be one pixel from the edge. The voxel structure V (x, y, z, c) has V (x, y, z, c) = 0 (x = 0 or x = Sx-1 or y = 0 or y = Sy-1 or z = 0 or z = Sz + 1, 0 ≦ c ≦ 3). That is, the voxel structure generation unit 23 generates a voxel structure including voxels having RGB values and opacity as predetermined values corresponding to each pixel of the outer edge portion of the plurality of xy plane images. Specifically, the voxel structure generation unit 23 makes the voxels corresponding to the pixels at the outer edge of the tomographic image colorless and transparent. The outer edge portion does not have to be limited to one pixel. As a predetermined value, the RGB value can be R = G = B = 0 and the opacity α = 0 (that is, the pixel color is colorless, that is, black and transparent), but the voxel does not contribute to volume rendering. If it is a value, the RGB value may not be limited to R = G = B = 0, and the opacity may not be limited to α = 0.

The color interpolation unit 28 is based on the RGB values of the neighboring voxels located in the vicinity of the target voxels with respect to the target voxels of the voxels corresponding to the colorless / transparent pixels at the outer edge of the tomographic image in step S24 described later. Then, the RGB values of the target voxels are color-interpolated. However, even for transparent voxels, if any of the RGB values is set to a value other than a predetermined value (here, a value of 1 or more) (for example, air in the lung field), color interpolation is performed. Not performed.

As described above, in the voxel data of the voxel structure of the present embodiment, the voxel structure corresponds to each pixel of the outer edge portion of a plurality of xy plane images, and the RGB value and the opacity are set to predetermined values. The voxel data includes, and the RGB value of the target voxel is based on the RGB value of the neighboring voxel located in the vicinity of the target voxel with respect to the target voxel whose opacity is a predetermined value and the RGB value of the voxel structure containing the voxel. Contains data whose values are color interpolated.

The image processing unit 20 calculates the voxel shadow value (S23). However, the process of step S23 is not indispensable, and if the process of step S23 is not performed, the process of step S24 may proceed.

The voxel shadow value can be calculated as follows. That is, the voxel structure generation unit 23 calculates the gradient vector regarding the opacity of the voxel of the voxel structure, and calculates the voxel shadow value based on the calculated gradient vector and the predetermined light source vector.

More specifically, the voxel structure generation unit 23 calculates the difference between the opacity of the voxel of the voxel structure and the opacity of each of the plurality of neighboring voxels located in the vicinity of the voxel. The voxel structure generation unit 23 calculates a voxel gradient vector based on the distance between the voxel and the neighboring voxel and the difference in opacity between the voxel and the neighboring voxel. In this case, when using the distance to the neighboring voxels, considering that the resolutions in the XY directions and the Z directions are different, the voxel structure generation unit 23 uses the z-axis direction component of the gradient vector as the variable magnification Rxy in the z-axis direction. It is necessary to correct with / Rz.

The light source vector (Lx, Ly, Lz) is designated as a unit vector by a parallel light source, for example, (Lx, Ly, Lz) = (0.57735, 0.57735, 0.57735). Further, the ambient light component is Ab. Here, 0 ≦ Ab ≦ 1. For example, Ab = 0.2.

The gradient vector (Gx, Gy, Gz) of the voxel (x, y, z) in the range of 0 ≦ x ≦ Sx-2, 0 ≦ y ≦ Sy-2, 0 ≦ z ≦ Sz is expressed in the equations (1) and (1). It can be calculated by 2) and (3). In the present embodiment, the gradient vector can be calculated with high accuracy by the difference between the voxel value in the vicinity of 26 and the voxel value in the center voxel. When calculating the gradient vector, keep in mind that the voxels are not scaled in the z direction before the coordinate conversion, so the resolution differs between the xy direction and the z direction, and the change in the z direction component Gz of the gradient vector. Double correction (multiplication of Rxy / Rz) is performed.

G = {Gx ² + Gy ² + Gz ² } ^1/2 (G is the square root of the sum of the square of Gx, the square of Gy, and the square of Gz). When G ≧ 1, the shadow value S (x, y, z) is given by the equation (4) by calculating only the diffuse reflection component. As a result, when the line-of-sight vector is changed and the voxel is rotated, a three-dimensional effect can be expressed. Even if the line of sight is changed, the specular reflection component is not added because the gradient vector does not change before the coordinate rotation processing is performed. The voxel structure generation unit 23 corrects the RGB value of the voxel based on the calculated luminance value. When the voxel after the shading calculation is represented by V'(x, y, z, c), the RGB value of the voxel can be corrected by applying the equation (5) in the range of 0 ≦ c ≦ 2. ..

In the 3D texture mapping method using OpenGL, since the processing related to the voxel gradient vector and the shadow calculation is not supported in the hardware rendering, the α value is always given as the shadow value. However, in the present embodiment, the shadow calculation can be performed in a pseudo manner. That is, in order to perform the shadow calculation accurately, it is necessary to perform it after the voxel coordinate conversion, but since the coordinate conversion is performed together with the hardware rendering, the voxel data to be input to the hardware rendering before the coordinate conversion is performed in advance. By incorporating the shading calculation result, it is possible to obtain a rendered image with pseudo shading.

In the voxels at the outer edge and background of the tomographic image, G <1 may be set. In that case, the color and transparency (R = G = B = α = 0) are uniformly set without performing the shadow calculation. That is, when the voxel after the shadow calculation is represented by V'(x, y, z, c), V'(x, y, z, c) = 0 (c = 0, 1, 2, 3). This is the same process as in step S22 described above.

The image processing unit 20 performs color interpolation processing for transparent voxels (S24). The details of the color interpolation processing will be described later.

The image processing unit 20 registers the color-interpolated voxel structure as a 3D texture map (S25), and sets the projection screen (S26).

FIG. 13 is an explanatory diagram showing an example of projection screen setting by the 3D texture mapping device 100 of the present embodiment. The conversion processing unit 24 sets the screen size (number of vertical / horizontal pixels, vertical / horizontal aspect ratio) of the rendered image. The conversion processing unit 24 sets either parallel projection (normal appearance rendering) or perspective projection (endoscope mode). Then, when the perspective projection is set, the conversion processing unit 24 sets the perspective projection parameter. The perspective projection parameters are, for example, the viewing angle (focal length) of the camera and the viewpoint position (viewpoint and gaze point, the former is the position of the eye and the latter is the position on the object being viewed, both on the Z axis. In general, the gazing point is fixed to the origin of the world coordinate system), clipping position (distance on the Z axis from the viewpoint, two points near and far), and the like. The clipping position may be only near. As shown in FIG. 13, in the case of parallel projection, all the lines of sight from the viewpoint are parallel to the Z axis, and the viewpoint is assumed to be virtually at a position at infinity to the left. All stacked quadrangles defined in the Z-axis direction are rendered targets. On the other hand, in the case of fluoroscopic projection, the line of sight is widened according to the viewing angle from the viewpoint, and the rendering target is also limited by near clipping.

The image processing unit 20 performs a rendering process (S27), generates a rendered image on the frame memory (S28), and ends the process. The details of the rendering process will be described later.

FIG. 14 is a flowchart showing an example of a procedure for color interpolation processing of transparent voxels by the 3D texture mapping device 100 of the present embodiment. The image processing unit 20 extracts colorless and transparent voxels from all the voxels of the voxel structure of 0 ≦ x ≦ Sx-1, 0 ≦ y ≦ Sy-1, and 0 ≦ z ≦ Sz + 1 (S101). That is, a voxel satisfying the formula (6) is extracted.

The image processing unit 20 identifies an opaque voxel that is a voxel in the vicinity of the target voxel among the extracted voxels (S102). Assuming that the xyz coordinates of the target voxel are (x, y, z), there are 26 voxels in the vicinity of the target voxel in the range of -1≤i≤1, -1≤j≤1, and -1≤k≤1. An opaque voxel satisfying V (x + i, y + j, z + k, 3)> 0 is specified, and the number is C according to the definition of the equation (7). That is, the voxel structure generation unit 23 identifies a neighboring voxel whose opacity is larger than a predetermined value among the neighboring voxels related to the target voxel. The predetermined value can be opacity α = 0 (that is, transparent) that does not contribute to volume rendering, but the opacity may not be limited to α = 0 if it does not contribute to volume rendering.

The image processing unit 20 determines whether C> 0, that is, the presence or absence of opaque voxels (S103), and when C> 0, that is, when there is opaque voxels (YES in S103), the average of the RGB values of the opaque voxels. The RGB value of the target voxel is updated with the value (S104), and the process of step S106 described later is performed. Assuming that the average value of the RGB values is Ra, Ga, and Ba, the average value is the R value of each opaque box cell satisfying V (x + i, y + j, z + k, 3)> 0 according to the equations (8), (9), and (10). It can be calculated by the sum of V (x + i, y + j, z + k, 0), G value V (x + i, y + j, z + k, 1), and B value V (x + i, y + j, z + k, 2). Further, the RGB values of the colorless / transparent target voxels can be updated by the equations (11), (12) and (13).

When C = 0, that is, when there is no opaque voxel (NO in S103), the image processing unit 20 sets the RGB value of the target voxel to 0 (that is, Ra = Ga = Ba = 0) and extracts (S105). It is determined whether or not the color interpolation of all the target voxels of the colorless / transparent voxels has been completed (S106). When the color interpolation of all target voxels is not completed (NO in S106), the image processing unit 20 continues the processing after step S102, and when the color interpolation of all target voxels is completed (YES in S106). End the process.

The color interpolation processing is performed by the color interpolation unit 28 of the image processing unit 20. More specifically, the color interpolation unit 28 color-interpolates the RGB values of the target voxels based on the statistical values of the RGB values of the specified neighboring voxels. Although the average value can be used as the statistical value, the statistical value is not limited to the average value, and for example, the median value may be used.

As described above, the voxel data of the voxel structure of the present embodiment is RGB of the neighboring voxels having opacity greater than a predetermined value (that is, voxels having opacity α of 1 or more) among the neighboring voxels related to the target voxels. Contains data in which the RGB values of the target voxel are color interpolated based on the statistical values of the values.

FIG. 15 is a flowchart showing an example of the procedure of the rendering process by the 3D texture mapping device 100 of the present embodiment. The image processing unit 20 performs scaling and z-direction scaling processing on the 3D texture map (S111), and rotation processing on the 3D texture map (S112).

The image processing unit 20 performs offset processing on the 3D texture map (S113), and sets a laminated quadrangle composed of a plurality of quadrangles (S114). The image processing unit 20 associates each quadrangle constituting the laminated quadrangle with the 3D texture map (S115).

FIG. 16 is an explanatory diagram showing an example of a method of associating with 3D texture mapping. As shown in FIG. 16A, the DICOM tomographic image is converted into RGBα format and registered as a 3D texture map via the graphic API. Then, as shown in FIG. 16B, a laminated quadrangle is defined and 3D texture mapping is performed. In this case, the 3D world coordinates of the 4 vertices of each square in the world coordinate system are associated with the 3D texture coordinates of the 3D texture map in the texture coordinate system. As shown in FIG. 16, a quadrangle composed of four vertices (-1, -1, z), (-1,1, z), (1, -1, z), and (1,1, z). On the other hand, the world coordinates (-1, -1, z) are associated with the texture coordinates (0, 0, r).

When setting 3D texture mapping, the laminated quadrangle setting unit 27 sets a laminated quadrangle in which the same number of quadrangles as the total value of the number of a plurality of xy plane images and the number of dummy xy plane images are arranged in the Z-axis direction. ..

The number of quadrangles in the laminated quadrangle can be set arbitrarily, but if the number of quadrangles and the number of tomographic images do not match, the interpolation process works in the Z-axis direction, and the rounding error of the coordinate values due to the coordinate conversion accumulates. Striped / quadrangular moire occurs.

Therefore, the number of quadrangles in the laminated quadrangle and the number of 2D tomographic images registered in the 3D texture map are set to the number of original DICOM format tomographic images + 2 (dummy transparent images added in the z-axis direction). When constructing a 3D voxel structure, it is easier to post-process if the XYZ 3D resolution is unified, so the number of tomographic images in the original DICOM format is increased by interpolating between slices of the 2D tomographic image. Although the treatment is generally performed, the present embodiment is more preferable from the viewpoint of suppressing moire.

The image processing unit 20 performs scan conversion while pasting a 3D texture map on the laminated quadrangle, performs alpha blending processing in the order of the quadrangles far from the viewpoint (S116), and ends the processing.

More specifically, all the RGB values of the rendered image are initialized with the background color (for example, R = G = B = 0) in advance. The rendering processing unit 25 attaches a 3D texture map to the quadrangle farthest from the viewpoint, performs parallel projection or perspective projection on the rendered image defined in FIG. 13, and (-1) of the XYZ coordinates in the world coordinate system of the projected quadrangle. , -1, -1) to the X-axis direction and the Y-axis direction, the interval u corresponding to the pixel interval in the rendered image (for example, u is the interval of the world coordinates corresponding to one pixel of the rendered image. Perform scan conversion at = 0.002). At each scan-converted world coordinate (-1 + iu, -1 + ju, -1) (i, j are the coordinate values of the rendered image, for example, an integer of 0≤i, j≤511), each corresponding texture coordinate (2iu). , 2ju, 0) is calculated, the RGB value and opacity are acquired by referring to the 3D texture map based on the calculated texture coordinates, and the RGB already recorded in the corresponding rendered image coordinates (i, j). Performs alpha blending processing with the value, and updates the RGB values of the coordinates (i, j) of the corresponding rendered image. In this way, when the alpha blending process is completed for the single quadrangle farthest from the viewpoint, the Z coordinate is v (the interval between the world coordinates where the quadrangle is arranged in the Z axis direction is v. For example, v = 2. / Sz) is increased, and the same processing is repeated for the quadrangle (-1 + iu, -1 + ju, -1 + v) next to the viewpoint direction to update the rendered image. Scan-convert all quadrilaterals and render the image when the alpha blending process for world coordinates (-1 + iu, -1 + ju, -1 + kv) (k is the number of the quadrangle and 0≤k≤Sz-1) is completed. Is completed.

Equations (14), (15) and (16) can be used for the alpha blending process.

Here, R', G', and B'are RGB values of the rendered image updated on the projection plane. R, G, and B are RGB values in the voxels (2iu, 2ju, 2kv) of the 3D texture map corresponding to the world coordinates (-1 + iu, -1 + ju, -1 + kv) of the quadrangle, and correspond to the values of the overlapping colors. α is also an α value in the voxels (2iu, 2ju, 2kv) of the 3D texture map corresponding to the world coordinates (-1 + iu, -1 + ju, -1 + kv) of the quadrangle, and controls the overlapping ratio. Further, Rb, Gb, and Bb are RGB values already recorded in the coordinates (i, j) of the rendered image corresponding to the world coordinates (-1 + iu, -1 + ju, -1 + kv), and are viewpoints with respect to the quadrangle. It matches the values of R', G', and B'calculated based on the equations (14), (15), and (16) for the quadrangle located one before on the opposite side, and is the farthest from the viewpoint. When calculating for a quadrangle, a background color (for example, Rb = Gb = Bb = 0) is given.

FIG. 17 is a block diagram showing another example of the configuration of the image processing unit 20 of the present embodiment. As shown in FIG. 17, the image processing unit 20 can be composed of a CPU 201, a ROM 202, a RAM 203, a GPU 204, a video memory 205, a recording medium reading unit 206, and the like. The computer program recorded on the recording medium 1 can be read by the recording medium reading unit 206 and stored in the RAM 203. By executing the computer program stored in the RAM 203 by the CPU 201, the gradation compression processing unit 21, the smoothing processing unit 22, the voxel structure generation unit 23, the conversion processing unit 24, the rendering processing unit 25, and the 3D texture image generation unit 26, the processing performed by the laminated square setting unit 27, the color interpolation unit 28, and the ROI clipping processing unit 29 can be executed. The CPU 201 is a central processing unit, which is usually composed of a plurality of cores (processors) and operates in cooperation with the RAM 203, whereas the GPU 204 is a graphics processing unit, which is used for many graphics processing. It is composed of a specialized processor and operates in cooperation with the video memory 205. Further, the computer program can be downloaded via a network such as the Internet instead of the configuration of reading by the recording medium reading unit 206.

Further, the CPU 201 outputs an instruction such as a rendering process to the GPU 204. The GPU 204 interprets the instruction and executes necessary processing such as rendering processing. Graphics API (eg, OpenGL, etc.) can generally be used for programming the GPU 204. For example, the computer program can cause the CPU 201 to execute a process of outputting a command for executing a conversion process and a rendering process to the GPU 204.

Next, the pattern of moire occurrence will be described. The reason why moire occurs in a volume rendered image is that tomographic images are laminated and combined. As a type of moire, there is a curved pattern as the first type. Further, as the second pattern, there is a stripe / grid pattern. It should be noted that the curved pattern and the striped / grid-like pattern are not strictly separated, and in the volume rendering image, both are present in a complex mixture. Hereinafter, they will be described separately for convenience of explanation. First, the first curved pattern will be described.

FIG. 18 is a schematic diagram showing an example of how curved moire is generated. The cause of occurrence is a pattern that does not have perceptible regularity that originally exists naturally in a tomographic image (slice image). By performing coordinate transformation (rotation around the xyz axis) of a plurality of tomographic images having such a pattern, the above-mentioned non-regular patterns existing in each tomographic image are deviated and synthesized. The patterns are linearly connected, resulting in a curved striped pattern in the volume rendered image, which is perceived as a perceptible moire.

FIG. 19 is an explanatory diagram showing a first example of a moire countermeasure result by the 3D texture mapping device 100 of the present embodiment. FIG. 19A is a comparative example and shows a case where the moire countermeasure of the present embodiment is not implemented. In FIG. 19B, the smoothing processing unit 22 of the 3D texture mapping device 100 of the present embodiment smoothes each tomographic image in the 3D space (isotropic smoothing in the xyz direction) as described in FIG. Is shown. In FIG. 19A, a curved striped pattern is generated, but in FIG. 19B, it can be seen that the striped pattern disappears. In particular, in a CT image with a relatively coarse slice interval, the effect of suppressing moire is large.

Next, a stripe / grid pattern, which is the second pattern of moiré, will be described.

FIG. 20 is a schematic diagram showing an example of how stripe-lattice moire is generated. The characteristic of the striped / grid pattern is that unlike the first pattern, the moire source is not in the tomographic image itself, but in the artificial pattern generated by calculation in the process of 3D texture mapping processing. By performing coordinate conversion (rotation around the xyz axis) of each tomographic image of the 3D texture map, the fraction is rounded down to an integer in the process of real number operation called coordinate conversion, and the boxel value before conversion is referred to by the xyz coordinate of the integer value. In the process, a periodic pattern is generated due to the accumulation of rounding errors, and it is added to the tomographic image of the 3D texture map after the coordinate conversion. The periodic pattern added to each tomographic image is weak at an imperceptible level, but if multiple tomographic images overlap due to alpha blending processing and the shift width at the time of overlap satisfies the phase condition of each periodic pattern, By strengthening each other, so-called interference fringes are generated, and the level becomes perceptible as stripes and grid-like moire.

FIG. 21 is an explanatory diagram showing an example of the occurrence of rounding error during coordinate conversion. In FIG. 21, voxels are represented in two dimensions for convenience. The voxels before coordinate conversion are represented by V11, V12, ..., V49 with the horizontal direction as the x-axis direction and the vertical direction as the z-axis direction. Due to the coordinate transformation that is rotated around the z-axis, the rounding error that accompanies the integerization of the x-coordinate of the real value is accumulated in the x-coordinate value to be referenced. For example, the calculated x-coordinate value of the real number is 1.3, 2.6, 3.9, 5.2, 6.5, 7.8 from left to right.

For example, when the calculated x-coordinate value of the real number is 1.3, the voxel after the coordinate conversion refers to V11 and V12 before the coordinate conversion by V11 × 0.7 + V12 × 0.3, and 0.3. It can be obtained by interpolating while multiplying the weights based on the fraction. Further, when the calculated x-coordinate value of the real number is 2.6, the voxel after the coordinate conversion refers to V12 and V13 before the coordinate conversion by V12 × 0.4 + V13 × 0.6, and is 0.6. It can be obtained by interpolating while multiplying the weights based on the fraction. Further, when the calculated x-coordinate value of the real number is 3.9, the voxel after the coordinate conversion refers to V13 and V14 by V13 × 0.1 + V14 × 0.9 before the coordinate conversion, and 0.9. It can be obtained by interpolating while multiplying the weights based on the fraction. Further, when the calculated x-coordinate value of the real number is 5.2, the voxel after the coordinate conversion refers to V15 and V16 before the coordinate conversion by V15 × 0.8 + V16 × 0.2, and 0.2. It can be obtained by interpolating while multiplying the weights based on the fraction.

In this way, the fourth voxel from the left after the coordinate conversion is not interpolated by the voxels V14 and V15 before the coordinate conversion, but is interpolated by V15 and V16, and the third voxel and 4 from the left after the coordinate conversion. There is a step in the voxel value with the third voxel. However, this step is not a perceptible level in a single layer (tomographic image) formed by V11 to V19. However, as shown in FIG. 21, when a similar step occurs at the same location in the three layers formed by V21 to V49, as a result, it is perceived as a striped / grid-like moire along the z-axis direction. It will be possible. In particular, when such a step occurs at the boundary surface between the colored opaque region (R + G + B> 0, opacity α> 0) and the colorless transparent region (R = G = B = α = 0), it is easily noticeable.

FIG. 22 is a schematic diagram showing an example of how a periodic pattern is generated by coordinate transformation. The source of the periodic pattern that causes moiré is the boundary surface between the opaque region (R + G + B> 0, opacity α> 0) and the transparent region (R = G = B = α = 0). Is a flat surface, and moire is easily noticeable when the color (RGB value) or opacity (α value) of the opaque region is relatively large. When the pixels of the boundary surface (that is, voxels) are arranged in a stepped manner by the coordinate conversion, one or more opacity is set for the voxels in the nearby transparent area by the interpolation process, and the voxels become translucent and the color is also changed. Colorless, that is, from black to gray. The opacity and color of the interpolated transparent voxels change periodically with the stepped arrangement, and stripes and grid-like stripes are generated, which becomes a source of moire.

FIG. 23 is a schematic diagram showing an example of the state of color interpolation by the 3D texture mapping device 100 of the present embodiment. In the present embodiment, the color interpolation unit 28 performs color interpolation (color correction) in advance on the color of the voxels in the transparent area in contact with the opaque area before the coordinate conversion by the conversion processing unit 24 is performed. That is, the opacity of the transparent voxels in contact with the opaque region is left as 0, and the color of the transparent voxels is given the average value of the RGB values of the opaque voxels in the vicinity of 26. Then, due to the coordinate transformation, the opacity of the transparent voxels changes periodically with the stepwise arrangement, but the color of the transparent voxels becomes almost uniform and the periodic pattern cannot be perceived. That is, even if the pixels in the transparent area at the boundary (edge part) between the transparent area and the opaque area become semi-transparent, a color similar to that of the opaque area in the vicinity is added, so that the periodic change in color does not occur. It becomes difficult to be recognized. As a result, according to the color interpolation processing according to the present embodiment, only the color of the transparent voxel in contact with the opaque region changes, and the opacity remains transparent, so that the action of the voxel that contributes to volume rendering does not change. It is possible to solve the problem of moire generation due to coordinate conversion without theoretically causing image quality deterioration.

FIG. 24 is an explanatory diagram showing a second example of the moire countermeasure result by the 3D texture mapping device 100 of the present embodiment. FIG. 24A is a comparative example and shows a case where the moire countermeasure of the present embodiment is not implemented. In FIG. 24B, the color interpolation unit 28 of the 3D texture mapping device 100 of the present embodiment makes the voxels outside the ROI clipping region colorless and transparent based on the processes of steps S19 and S24 of FIG. 9, and is colorless and transparent. The case where the color of the voxel is color-interpolated based on the RGB value of the opaque voxel in the vicinity is shown. It can be seen that moiré is generated in FIG. 24A, but disappears in FIG. 24B.

FIG. 25 is an explanatory diagram showing a third example of the moire countermeasure result by the 3D texture mapping device 100 of the present embodiment. FIG. 25 is an example in which the shadow value calculation process of step S23 of FIG. 9 is added to the example of FIG. 24B. FIG. 25A is the same comparative example as in FIG. 24A, and shows a case where no shadow is added and the moire countermeasure of the present embodiment is not implemented. FIG. 25B shows ROI clipping based on the processes of steps S19 and S24 of FIG. 9 by the color interpolation unit 28 of the 3D texture mapping device 100 of the present embodiment after adding the shadow value calculation process of step S23. The case where the voxels outside the region are made transparent and the color of the transparent voxels is color-interpolated based on the RGB values of the neighboring opaque voxels is shown. In FIG. 25A, moiré is generated, but in FIG. 25B, it can be seen that the moiré disappears even if a shadow is added.

FIG. 26 is an explanatory diagram showing a fourth example of the moire countermeasure result by the 3D texture mapping device 100 of the present embodiment. FIG. 26A is a comparative example and shows a case where the moire countermeasure of the present embodiment is not implemented. FIG. 26B shows the voxel structure generation unit 23 and the color interpolation unit 28 of the 3D texture mapping device 100 of the present embodiment at both ends of the tomographic image in the z-axis direction based on the processes of steps S20 and S24 of FIG. A case where a dummy transparent image is added, a dummy voxel is added, and the color of the transparent voxel is color-interpolated based on the RGB value of a nearby opaque voxel is shown. It can be seen that the diagonal stripe pattern is generated in FIG. 26A, but the stripe pattern disappears in FIG. 26B.

FIG. 27 is an explanatory diagram showing a fifth example of the moire countermeasure result by the 3D texture mapping device 100 of the present embodiment. FIG. 27 is an example when the shadow value calculation process of step S23 of FIG. 9 is added to both cases of FIG. 26. FIG. 27A is a comparative example, and shows a case where the shadow value calculation process of step S23 of FIG. 9 is added, but the moire countermeasure of the present embodiment is not implemented. 27B shows step S22 and step S24 of FIG. 9 by the voxel structure generation unit 23 and the color interpolation unit 28 of the 3D texture mapping device 100 of the present embodiment after adding the shading value calculation process of step S23. The case where the voxels corresponding to the pixels at the outer edge of the tomographic image are made transparent based on the processing of, and the color of the transparent voxels is color-interpolated based on the RGB values of the neighboring opaque voxels is shown. It can be seen that the diagonal stripe pattern is generated in FIG. 27A, but the stripe pattern disappears in FIG. 27B.

As described above, according to the present embodiment, it is possible to suppress the generation of various moire (curved moire, striped / grid-shaped moire) while suppressing the deterioration of the image quality of the rendered image.

The computer program of the present embodiment corresponds to a process of acquiring image data of a plurality of xy plane images captured at predetermined intervals along the z-axis direction and a pixel of each of the plurality of xy plane images. Then, the process of generating a voxel structure composed of voxels having a defined RGB value and opacity, and the position in the vicinity of the target voxel with respect to the target voxel in which the opacity of the voxel structure is a predetermined value. A process of correcting the RGB value of the target voxel based on the RGB value of the neighboring voxel, a process of generating a 3D texture image composed of the voxel of the voxel structure whose RGB value has been corrected, and the process of generating the 3D texture image. To generate a 3D texture image after conversion by performing a predetermined conversion on Obtained by alpha blending the RGB values of the voxels of the converted 3D texture image corresponding to the XY coordinates of the squares on the line of sight parallel to the axial direction in the order of the voxels far from the viewpoint based on the opacity of the voxels. The processing of generating a rendered image is executed by using the RGB value as the pixel value of the pixel whose line of sight intersects the projection surface.

In the computer program of the present embodiment, the computer corresponds to each pixel of the dummy xy plane image arranged at both ends in the z-axis direction with the plurality of xy plane images in between, and the opacity is set to the predetermined value. The process of generating a voxel structure with a dummy voxel added, and the RGB of the neighboring voxels located in the vicinity of the target voxel with respect to the target voxel whose opacity is the predetermined value among the generated voxel structures. Based on the value, the process of correcting the RGB value of the target voxel is executed.

In the computer program of the present embodiment, a process of setting a laminated quadrangle in which the same number of the quadrangles as the total value of the number of the plurality of xy plane images and the number of the dummy xy plane images is arranged in the Z-axis direction is set in the computer. Let it run.

The computer program of the present embodiment processes and generates a voxel structure including a voxel having an opacity set to the predetermined value corresponding to each pixel of the outer edge portion of the plurality of xy plane images on the computer. A process of correcting the RGB value of the target voxel based on the RGB value of the neighboring voxel located in the vicinity of the target voxel for the target voxel whose opacity is the predetermined value among the voxels of the voxel structure. To execute.

The computer program of the present embodiment sets the xyz coordinates that define the target area of the volume rendering process in the computer, and the voxels of the voxel structure whose voxels are outside the target area of the voxels. The process of setting the predetermined value for the opacity of the voxel and the position in the vicinity of the target voxel with respect to the target voxel whose opacity is the predetermined value among the voxels of the voxel structure in which the predetermined value is set. The process of correcting the RGB value of the target voxel is executed based on the RGB value of the neighboring voxel.

The computer program of the present embodiment is based on a process of specifying to a computer a nearby voxel whose opacity is larger than the predetermined value among the neighboring voxels related to the target voxel, and a statistical value of the RGB value of the specified neighboring voxel. Then, the process of correcting the RGB value of the target voxel is executed.

The computer program of the present embodiment tells a computer based on the pixel values of an arbitrary pixel of interest on one xy plane image of each of the plurality of xy plane images and pixels in the vicinity of the pixel of interest in the xyz direction. The process of smoothing the xy plane image and the process of generating a voxel structure composed of voxels having RGB values and opacity corresponding to each pixel of each of the smoothed xy plane images. To execute.

The computer program of the present embodiment has a process of specifying to a computer a minimum value and a maximum value of pixels of a predetermined xy plane image among the plurality of xy plane images, and an upper limit value smaller than the specified maximum value and a maximum value. A process of calculating a lower limit value larger than the specified minimum value, a process of compressing the pixel value of each pixel of the plurality of xy plane images within the range of the upper limit value and the lower limit value, and a process of compressing the plurality of xy planes. The process of smoothing the plurality of xy plane images is executed based on the pixel values of any pixel of interest and the neighboring pixels of each image.

The computer program of the present embodiment has a process of acquiring color map data in which RGB values and opacity are defined in association with pixel values, and each pixel of each of the smoothed plurality of xy plane images. A process of associating the RGB value and opacity of the color map data with the RGB value and opacity of the corresponding voxel to generate a voxel structure is executed.

The computer program of the present embodiment has a process of calculating the xyz coordinates of the 3D texture image corresponding to the XY coordinates of the square on the line of sight, and the converted 3D texture image corresponding to the calculated xyz coordinates. The process of alpha blending based on the RGB value and opacity of the boxel corresponding to the xyz coordinate of is executed.

The computer program of the present embodiment causes a computer to calculate a gradient vector relating to voxel opacity of the voxel structure, and calculates a voxel shading value based on the calculated gradient vector and a predetermined light source vector. The process and the process of correcting the RGB value of the voxel of the voxel structure based on the calculated shadow value are executed.

The computer program of the present embodiment causes a computer to calculate a difference between the opacity of a voxel of the voxel structure and the opacity of each of the neighboring voxels located in the vicinity of the voxel, and the voxel and the vicinity thereof. The process of calculating the gradient vector of the voxel based on the distance to the voxel and the difference in opacity between the voxel and the neighboring voxel is executed.

The computer program of the present embodiment causes a computer to execute a process of correcting a component of the gradient vector in the z-axis direction at a predetermined variable magnification.

The computer program of the present embodiment has a process of setting the predetermined value for the opacity of the voxel for the voxel in which the computer cannot calculate the gradient vector regarding the opacity of the voxel of the voxel structure. Among the voxels of the voxel structure in which the predetermined value is set, for the target voxel whose opacity is the predetermined value, the RGB of the target voxel is based on the RGB values of the neighboring voxels located in the vicinity of the target voxel. The process of correcting the value is executed.

The computer program of the present embodiment tells the computer the rotation angle of the x-axis center, the y-axis center and the z-axis center, the viewing angle, the viewpoint position, the clipping position, the offset value in the xyz axis direction, and the enlargement or reduction in the xyz axis direction. A process of acquiring the parameters of the predetermined conversion including the magnification and the variable magnification in the z-axis direction and a predetermined conversion of the 3D texture image using the acquired computer are performed to generate a converted 3D texture image. Process and execute.

The computer program of the present embodiment causes the computer to execute a process of generating the converted 3D texture image and a process of outputting a command for executing the process of generating the rendered image to the GPU.

The image processing device of the present embodiment is an image processing device that performs volume rendering processing using a 3D texture mapping method, and obtains image data of a plurality of xy plane images captured at predetermined intervals along the z-axis direction. A box cell structure generation unit that generates a box cell structure composed of a box cell to be acquired, a box cell corresponding to each pixel of each of the plurality of xy plane images and a defined RGB value and opacity, and the box cell structure. For a target boxel whose body opacity is a predetermined value, a color correction unit that corrects the RGB value of the target boxel and a RGB value correction based on the RGB value of the neighboring boxel located in the vicinity of the target boxel. A 3D texture image generation unit that generates a 3D texture image composed of the box cells of the boxel structure, and a converted 3D texture image that generates a converted 3D texture image by performing predetermined conversion on the 3D texture image. Corresponds to the generation unit, the setting unit that sets a laminated quadrangle in which quadrangles on the XY coordinate plane of the three-dimensional space are arranged in the Z-axis direction, and the XY coordinates of the quadrangle on the line of sight parallel to the Z-axis direction from a predetermined viewpoint. The RGB values of the box cells of the converted 3D texture image are alpha-blended in the order of the squares far from the viewpoint based on the opacity of the box cells, and the RGB values obtained by alpha blending the pixels of the pixels whose line of sight intersects the projection plane. It includes a rendered image generator that generates a rendered image as a value.

The image processing method of the present embodiment is an image processing method using an image processing device that performs volume rendering processing using a 3D texture mapping method, and is a plurality of xy plane images captured at predetermined intervals along the z-axis direction. The image data of the above is acquired, and a voxel structure composed of voxels having RGB values and opacity corresponding to each pixel of each of the plurality of xy plane images is generated, and the opacity of the voxel structure is increased. For the target voxel which is a predetermined value, the RGB value of the target voxel is corrected based on the RGB value of the neighboring voxel located in the vicinity of the target voxel, and the voxel is composed of the voxel structure in which the RGB value is corrected. A 3D texture image is generated, a predetermined conversion is performed on the 3D texture image, and a converted 3D texture image is generated. The RGB value of the voxel of the converted 3D texture image corresponding to the XY coordinates of the quadrangle on the line of sight parallel to the Z-axis direction from the predetermined viewpoint is set to the viewpoint based on the opacity of the voxel. A rendered image is generated using the RGB values obtained by alpha blending in the order of the quadrangles far from the to as the pixel values of the pixels whose line of sight intersects the projection plane.

The voxel data of the present embodiment is voxel data of a voxel structure, and corresponds to each pixel of a plurality of xy plane images captured at predetermined intervals along the z-axis direction, and has RGB values and opacity. Has a structure including voxel data of a voxel structure composed of voxels in which voxels are defined, and the voxel data is in the vicinity of the target voxel with respect to the target voxel in which the opacity of the voxel structure is a predetermined value. A 3D texture image composed of voxels of the voxel structure, which contains data in which the RGB values of the target voxels are corrected based on the RGB values of neighboring voxels located in, is converted by performing a predetermined conversion. After that, the process of generating a 3D texture image, the process of setting a laminated quadrangle in which quadrangles on the XY coordinate plane of the three-dimensional space are arranged in the Z-axis direction, and the process of setting the quadrangle on the line of sight parallel to the Z-axis direction from a predetermined viewpoint. The RGB value obtained by alpha blending the RGB value of the voxel of the converted 3D texture image corresponding to the XY coordinates of It is used to execute a process of generating a rendered image as a pixel value of intersecting pixels.

In the voxel data of the present embodiment, the voxel structure corresponds to each pixel of the dummy xy plane image arranged at both ends in the z-axis direction with the plurality of xy plane images in between, and the opacity is the predetermined. The voxel data includes a dummy voxel which is a valued voxel, and the voxel data is located in the vicinity of the target voxel with respect to the target voxel in which the opacity of the voxel structure to which the dummy voxel is added is the predetermined value. Includes data in which the RGB value of the target voxel is corrected based on the RGB value of the neighboring voxel.

In the voxel data of the present embodiment, the voxel structure corresponds to each pixel of the outer edge portion of the plurality of xy plane images, and includes a voxel having an opacity set to the predetermined value, and the voxel data is the voxel data. Among the voxels of the voxel structure containing the voxels, the RGB values of the target voxels are corrected based on the RGB values of the neighboring voxels located in the vicinity of the target voxels for the target voxels whose opacity is the predetermined value. Includes data.

In the voxel data of the present embodiment, the voxel structure is outside the target area of the volume rendering process and includes a voxel in which the predetermined value is set for the opacity, and the voxel data is set to the predetermined value. Among the voxels of the voxel structure containing the voxels, the RGB value of the target voxel is based on the RGB value of the neighboring voxel located in the vicinity of the target voxel with respect to the target voxel whose opacity is the predetermined value. Includes corrected data.

In the voxel data of the present embodiment, the voxel data is the RGB value of the target voxel based on the statistical value of the RGB value of the neighboring voxel whose opacity is larger than the predetermined value among the neighboring voxels related to the target voxel. Includes corrected data.

10 Input unit 11 Storage unit 12 Output unit 20 Image processing unit 21 Gradation compression processing unit 22 Smoothing processing unit 23 Voxel structure generation unit 24 Conversion processing unit 25 Rendering processing unit 26 3D texture image generation unit 27 Laminated square setting unit 28 Color interpolation unit 29 ROI clipping processing unit 100 3D texture mapping device 201 CPU
202 ROM
203 RAM
204 GPU
205 Video memory 206 Recording medium reader

Claims (18)

On the computer
Processing to acquire image data of a plurality of xy plane images taken at predetermined intervals along the z-axis direction, and
A process of generating a voxel structure composed of voxels having RGB values and opacity corresponding to each pixel of each of the plurality of xy plane images.
A process of correcting the RGB value of the target voxel based on the RGB value of the neighboring voxel located in the vicinity of the target voxel for the target voxel whose opacity of the voxel structure is a predetermined value.
A process to generate a 3D texture image composed of voxels of a voxel structure whose RGB values have been corrected, and
A process of performing a predetermined conversion on the 3D texture image to generate a converted 3D texture image, and
The process of setting a laminated quadrangle in which quadrilaterals on the XY coordinate plane of the three-dimensional space are arranged in the Z-axis direction, and
Alpha blending the RGB values of the voxels of the converted 3D texture image corresponding to the XY coordinates of the squares on the line of sight parallel to the Z-axis direction from a predetermined viewpoint in the order of the squares far from the viewpoint based on the opacity of the voxels. A computer program that executes a process of generating a rendered image using the RGB values obtained in the process as pixel values of pixels whose line of sight intersects the projection plane. On the computer
A voxel structure is generated to which a dummy voxel having an opacity of a predetermined value is added corresponding to each pixel of the dummy xy plane image arranged at both ends in the z-axis direction with the plurality of xy plane images in between. Processing and
Processing to correct the RGB value of the target voxel based on the RGB value of the neighboring voxel located in the vicinity of the target voxel for the target voxel whose opacity is the predetermined value among the voxels of the generated voxel structure. The computer program according to claim 1, wherein the computer program is executed. On the computer
The computer program according to claim 2, wherein a process of setting a laminated quadrangle in which the same number of the quadrangles as the total value of the number of the plurality of xy plane images and the number of dummy xy plane images is arranged in the Z-axis direction is executed. On the computer
A process of generating a voxel structure including a voxel having an opacity set to the predetermined value corresponding to each pixel of the outer edge portion of the plurality of xy plane images.
Processing to correct the RGB value of the target voxel based on the RGB value of the neighboring voxel located in the vicinity of the target voxel for the target voxel whose opacity is the predetermined value among the voxels of the generated voxel structure. The computer program according to any one of claims 1 to 3, wherein the computer program is executed. On the computer
The process of setting the xyz coordinates that define the target area of the volume rendering process, and
A process of setting the opacity of the voxel to a predetermined value for a voxel whose xyz coordinate is outside the target area among the voxels of the voxel structure.
Among the voxels of the voxel structure in which the predetermined value is set, for the target voxel whose opacity is the predetermined value, the RGB of the target voxel is based on the RGB values of the neighboring voxels located in the vicinity of the target voxel. The computer program according to any one of claims 1 to 4, wherein the process of correcting the value is executed. On the computer
Among the neighboring voxels related to the target voxel, the process of specifying the neighboring voxels whose opacity is larger than the predetermined value, and
The computer program according to any one of claims 1 to 5, wherein the process of correcting the RGB values of the target voxels is executed based on the statistical values of the RGB values of the specified neighboring voxels. On the computer
A process of smoothing the one xy plane image based on the pixel values of an arbitrary pixel of interest on one xy plane image of each of the plurality of xy plane images and pixels in the vicinity of the pixel of interest in the xyz direction.
Claims 1 to 6 for executing a process of generating a voxel structure composed of voxels having RGB values and opacity defined for each pixel of each of the plurality of smoothed xy plane images. The computer program described in any one of the sections. On the computer
A process of specifying a minimum value and a maximum value of pixels of a predetermined xy plane image among the plurality of xy plane images, and a process of specifying the minimum value and the maximum value.
Processing to calculate the upper limit value smaller than the specified maximum value and the lower limit value larger than the specified minimum value,
A process of compressing the pixel value of each pixel of the plurality of xy plane images within the range of the upper limit value and the lower limit value, and
The computer program according to claim 7, wherein the process of smoothing the plurality of xy plane images is executed based on the pixel values of the arbitrary attention pixels and the neighboring pixels of the plurality of compressed xy plane images. On the computer
The process of acquiring color map data in which RGB values and opacity are defined in association with pixel values, and
A request to execute a process of generating a voxel structure by associating the RGB values and opacity of the color map data with the RGB values and opacity of the voxels corresponding to each pixel of each of the smoothed plurality of xy plane images. 7. The computer program according to claim 8. On the computer
The process of calculating the xyz coordinates of the 3D texture image corresponding to the XY coordinates of the quadrangle on the line of sight, and
According to any one of claims 1 to 9, the process of performing alpha blending based on the RGB value and opacity of the voxel corresponding to the xyz coordinates of the converted 3D texture image corresponding to the calculated xyz coordinates is executed. Described computer program. On the computer
The process of calculating the gradient vector regarding the voxel opacity of the voxel structure and
Processing to calculate the shadow value of the voxel based on the calculated gradient vector and a predetermined light source vector, and
The computer program according to any one of claims 1 to 10, wherein the process of correcting the RGB value of the voxel of the voxel structure is executed based on the calculated shadow value. On the computer
The process of calculating the difference between the opacity of the voxels of the voxel structure and the opacity of each of the neighboring voxels located in the vicinity of the voxels.
The computer program according to claim 11, wherein a process of calculating a gradient vector of the voxel based on the distance between the voxel and the neighboring voxel and the difference in opacity between the voxel and the neighboring voxel is executed. On the computer
The computer program according to claim 11 or 12, wherein a process of correcting a component of the gradient vector in the z-axis direction at a predetermined variable magnification is executed. On the computer
A process of setting the predetermined value for the opacity of the voxel for a voxel for which the gradient vector regarding the opacity of the voxel of the voxel structure cannot be calculated.
Among the voxels of the voxel structure in which the predetermined value is set, for the target voxel whose opacity is the predetermined value, the RGB of the target voxel is based on the RGB values of the neighboring voxels located in the vicinity of the target voxel. The computer program according to any one of claims 11 to 13, wherein the process of correcting the value is executed. On the computer
The predetermined value including the x-axis center, the y-axis center and the z-axis center rotation angle, the viewing angle, the viewpoint position, the clipping position, the offset value in the xyz-axis direction, the enlargement or reduction magnification in the xyz-axis direction, and the variable magnification in the z-axis direction. And the process of getting the conversion parameters of
The invention according to any one of claims 1 to 14, wherein the 3D texture image is subjected to a predetermined conversion using the acquired parameters to generate a converted 3D texture image. Computer program. On the computer
The computer program according to any one of claims 1 to 15, wherein the process of generating the converted 3D texture image and the process of outputting a command for executing the process of generating the rendered image to the GPU are executed. .. An image processing device that performs volume rendering processing using the 3D texture mapping method.
An acquisition unit that acquires image data of a plurality of xy plane images captured at predetermined intervals along the z-axis direction, and an acquisition unit.
A voxel structure generation unit that generates a voxel structure composed of voxels having RGB values and opacity defined for each pixel of each of the plurality of xy plane images.
A color correction unit that corrects the RGB value of the target voxel based on the RGB value of the neighboring voxel located in the vicinity of the target voxel with respect to the target voxel whose opacity of the voxel structure is a predetermined value.
A 3D texture image generation unit that generates a 3D texture image composed of voxels of a voxel structure whose RGB values have been corrected, and a 3D texture image generation unit.
A converted 3D texture image generation unit that generates a converted 3D texture image by performing a predetermined conversion on the 3D texture image.
A setting unit for setting a laminated quadrangle in which quadrilaterals on the XY coordinate plane of the three-dimensional space are arranged in the Z-axis direction,
Alpha blending the RGB values of the voxels of the converted 3D texture image corresponding to the XY coordinates of the squares on the line of sight parallel to the Z-axis direction from a predetermined viewpoint in the order of the squares far from the viewpoint based on the opacity of the voxels. An image processing device including a rendered image generation unit that generates a rendered image using the RGB values obtained in the process as pixel values of pixels whose line of sight intersects the projection surface. It is an image processing method by an image processing device that performs volume rendering processing using a 3D texture mapping method.
Image data of a plurality of xy plane images taken at predetermined intervals along the z-axis direction are acquired, and the image data is acquired.
A voxel structure composed of voxels having RGB values and opacity corresponding to each pixel of each of the plurality of xy plane images is generated.
For the target voxel whose opacity of the voxel structure is a predetermined value, the RGB value of the target voxel is corrected based on the RGB value of the neighboring voxel located in the vicinity of the target voxel.
Generates a 3D texture image composed of voxels of a voxel structure with RGB corrected values.
A predetermined conversion is performed on the 3D texture image to generate a converted 3D texture image.
Set a laminated quadrangle in which quadrilaterals on the XY coordinate plane of the three-dimensional space are arranged in the Z-axis direction.
Alpha blending the RGB values of the voxels of the converted 3D texture image corresponding to the XY coordinates of the squares on the line of sight parallel to the Z-axis direction from a predetermined viewpoint in the order of the squares far from the viewpoint based on the opacity of the voxels. An image processing method for generating a rendered image using the RGB values thus obtained as pixel values of pixels whose line of sight intersects the projection plane.

Images