JP2016105291A - Image processing device, image processing system and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily visualize layered image information as a three-dimensional object.SOLUTION: A sensor output data acquisition unit 42 of an image data generation unit 30 of an image processor 20 acquires data of layered image information from a sensor group 12. A slice image generation unit 44 generates data of a two-dimensional image for each slice plane where a distribution is acquired. An image axis conversion unit 50 generates the same data of a two-dimensional image with respect to a plurality of planes vertical to an axis different from a sensor axis. A slice image management unit 52 of a display processing unit 32 manages a slice image used in drawing in accordance with a view point position or the like. A drawing memory 56 sequentially stores data of an image necessary for drawing. An additional object image storage unit 58 stores image data of an object additionally displayed such as a cursor. An image drawing unit 54 draws a three-dimensional object using data of the slice image.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image processing apparatus, an image processing system, and an image processing method used in the apparatus for visualizing information on an object as an image.

  Techniques for acquiring information on the cross-sectional shape and contents of objects using electron beams, X-rays, and magnetic fields have been put to practical use in transmission electron microscopes, computed tomography (CT), magnetic resonance image (MRI), etc. Yes. In this technique, information for each cross section of the object is acquired in an axial direction perpendicular to the cross section, and can be held as laminated image information of the entire object. The laminated image information is not limited to information obtained by a relatively large-scale apparatus as described above, but can also be obtained by a cross-sectional shape measuring apparatus using visible light, laser, or the like.

  Information obtained in the above-mentioned technology becomes information in a three-dimensional space consisting of a two-dimensional plane composing a cross section and an axis perpendicular to it, so that it can be easily understood as a general two-dimensional image. It is difficult to visualize as much as possible. The simplest way is to display the two-dimensional information of the cross section while changing the position in the axial direction. In this case, it is difficult to grasp the size and position of the object in the axial direction. In addition, a method of acquiring data for each voxel and performing volume rendering has been put into practical use, but the resources and processing load required for data acquisition, storage, and image rendering increase, and the degree of freedom for data processing and manipulation There is a problem that is low.

  The present invention has been made in view of such problems, and an object thereof is to provide an image processing technique for easily visualizing layered image information.

  One embodiment of the present invention relates to an image processing apparatus. This image processing apparatus generates a slice image information by using a pixel value including an alpha value determined based on a physical quantity acquired at each position on a plurality of slice planes defined in a space including an observation object. And an image drawing unit for alpha blending drawing of slice image information in accordance with the input viewpoint position in order to display a three-dimensional object including an observation target, and the image drawing unit includes a preset additional object Is drawn inside a three-dimensional object.

  Here, the “physical quantity” may be a value actually measured by a sensor or the like, a value obtained by performing some calculation on the measurement value, or a calculated value calculated by CAD (Computer Aided Design) or a game. Therefore, the “slice plane” may be either a plane or curved surface in real space, or a plane or curved line in virtual space.

  Another aspect of the present invention relates to an image processing system. The image processing system includes slice image information based on at least one sensor that acquires a physical quantity at each position on a plurality of slice planes defined in a space including an observation target, and a pixel value that includes an alpha value determined based on the physical quantity. An image processing apparatus including an image drawing unit that generates a slice image information in accordance with an input viewpoint position, and an image drawing unit that performs alpha blending drawing according to an input viewpoint position; The image drawing unit draws a preset additional object inside the three-dimensional object.

  Yet another embodiment of the present invention relates to an image processing method. The image processing method includes generating slice image information from pixel values including alpha values determined based on physical quantities acquired at positions on a plurality of slice planes defined in a space including an observation object; A step of alpha-blending drawing of slice image information in accordance with an input viewpoint position in order to display a three-dimensional object including the target object. It is characterized by drawing inside the object.

  It should be noted that any combination of the above-described constituent elements and a representation of the present invention converted between a method, an apparatus, a system, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to visualize laminated image information in a form desired by a user easily and at low resource costs.

It is a figure which shows the structural example of the image processing system which can apply the image processing technique of this Embodiment. It is a figure which shows typically the form of the information which an image processing apparatus acquires as a result of the sensor group measuring in this Embodiment. It is a figure which shows the example of the image which an image processing apparatus produces | generates in this Embodiment and displayed on a display apparatus. It is a figure which shows the structure of the image processing apparatus in this Embodiment in detail. It is a flowchart which shows the process sequence in which the image data generation part in this Embodiment produces | generates the data of a slice image. It is a figure which shows the example of the method of determining the alpha value of a slice image in S12 of FIG. It is a figure which shows another example of the method of determining the alpha value of a slice image in S12 of FIG. It is a figure for demonstrating the method of suppressing generation | occurrence | production of noise in the setting of the alpha value of this Embodiment. It is a flowchart which shows the process sequence in which a display process part performs the display process of a three-dimensional object in this Embodiment. It is a figure which shows typically a mode that an image drawing part performs alpha blending drawing in S40 of FIG. It is a flowchart which shows the process sequence which calculates | requires the RGB value in the point on a screen in this Embodiment. FIG. 10 is a diagram for describing processing in which a slice image management unit determines necessary slice image data according to a viewpoint in S34 of FIG. 9. It is a figure for demonstrating the example of the method of determining the angle of the boundary line which switches a slice image in this Embodiment. It is a figure for demonstrating another method of drawing a three-dimensional object in this Embodiment. It is a figure which shows typically the process sequence which draws a three-dimensional object after drawing an additional object previously in this Embodiment. It is a figure which shows typically the process sequence of another method which draws an additional object in this Embodiment. It is a figure which shows typically the read-out procedure of the slice image data at the time of displaying a three-dimensional object as a moving image in this Embodiment. It is a figure which shows typically the shape of the slice image in the case of measuring distribution with respect to a partial cylindrical curved surface or a partial spherical surface in this Embodiment.

  FIG. 1 shows a configuration example of an image processing system to which the image processing technique of this embodiment can be applied. The image processing system 10 includes N sensor groups 12 each including predetermined sensors 12_1, 12_2,... 12_N, an image processing device 20 that acquires and visualizes measurement results from each sensor, and a user performs image processing. An input device 15 for inputting instructions to the device 20 and a display device 14 for displaying an image generated by the image processing device 20 are included.

  The image processing device 20, the sensor group 12, and the display device 14 may be connected by a wired cable, or may be wirelessly connected by a wireless LAN (Local Area Network) or the like. Any two or all of the sensor group 12, the image processing device 20, and the display device 14 may be combined and integrally provided.

  As will be described later, the processing performed by the image processing apparatus 20 includes a step of generating basic two-dimensional image data from the measurement result and a step of generating a display image from the two-dimensional image data, which can be performed independently. Therefore, all of the image processing device 20, the sensor group 12, and the display device 14 may not be connected at the same time. The display device 14 may be a device that displays an image alone, such as a liquid crystal display or a plasma display, or may be a combination of a projector that projects an image and a screen.

  Each of the N sensors 12_1, 12_2,..., 12_N acquires a predetermined physical quantity as a distribution in a plurality of planes at a predetermined position and orientation, such as the plane 18 that crosses the object 16. Here, the predetermined physical quantity is information such as color, temperature, moisture content, hardness, etc. that can be obtained by using general sensing technology using visible light, X-rays, magnetism, electricity, ultrasonic waves, etc. It is not limited. In the figure, the arrangement and shape of the sensors 12_1, 12_2,..., 12_N are merely examples, and it is understood by those skilled in the art that various modes are conceivable depending on the sensing technology used.

  The image processing apparatus 20 integrates physical quantity distributions on a plurality of planes measured by the sensor group 12 and visualizes a measurement space in which the planes are stacked as a three-dimensional object. The object can be viewed from an arbitrary direction in response to a viewpoint movement request received from the user via the input device 15. In addition, the image processing apparatus 20 comprehensively controls the entire image processing system 10 such as controlling a plane from which the sensor group 12 obtains a physical quantity and controlling display on the display device 14.

  A sensing device including the sensor group 12 and having a mechanism for controlling the measurement processing by the sensors and the output of the laminated image information may be introduced separately from the image processing device 20. The sensing device may image the physical quantity distribution measured by the sensor, and the image processing device 20 may acquire the image data. In this case, instead of the physical quantity measured by the sensor, a distribution of pixel values such as RGB is obtained. In the following description, such a pixel value is similarly treated as a “physical quantity”.

  The display device 14 displays an image including a three-dimensional object generated as a result of visualizing the physical quantity distribution by the image processing device 20. As described above, the input device 15 includes a viewpoint movement request for the three-dimensional object displayed on the display device 14, a physical quantity measurement start by the sensor group 12, a measurement result acquisition start, an image and data processing, and a display device. A request for starting object display by 14 is received from the user and notified to the image processing apparatus 20. The input device 15 may be any of general input devices such as a keyboard, a controller, and a joystick in addition to the illustrated mouse, and may be a touch panel mounted on the screen of the display device 14.

  FIG. 2 schematically shows the form of information acquired by the image processing apparatus 20 as a result of measurement by the sensor group 12. Each sensor included in the sensor group 12 acquires a distribution of a predetermined physical quantity with respect to a plurality of planes 19a, 19b,..., 19n perpendicular to a predetermined axis (z axis) as shown in FIG. It is assumed that the planes 19a, 19b,..., 19n are at different positions with respect to the z axis and in the same range with respect to the xy plane perpendicular thereto.

  Hereinafter, the direction in which the plane for acquiring the physical quantity distribution by the sensor group 12 is moved as the z-axis in the figure is referred to as the “sensor axis”, and the plane is referred to as the “slice plane”. The sensor group 12 may be provided so that a plurality of sensor axes can be set. In this case, a plurality of slice plane groups are set for each sensor axis, and a physical quantity distribution is obtained for each plane. In FIG. 2, the slice plane is perpendicular to the sensor axis. However, the slice plane is not necessarily perpendicular depending on the physical quantity to be measured and the structure of the sensor.

  Since the “distribution” is obtained as a value for such a position coordinate on each slice plane, if the sensor group 12 includes N sensors 12_1, 12_2,..., 12_N as shown in FIG. N values for a certain position on the plane are obtained. As a result, the information acquired by the image processing apparatus 20 includes position coordinates (x, y) for each of a plurality of positions P1, P2,... Set for each slice plane 19a, 19b,. And N-valued vector values (V_1, V_2,..., V_N) are associated with each other.

  Note that the positions P1, P2,... May actually be set at intervals of about a general pixel, and depending on the resolution of the sensor, etc., the positions P1, P2,. Vector values of the same number of dimensions are obtained for P2,. The sensor group 12 may repeat measurement at predetermined time intervals, and the image processing apparatus 20 may acquire the information shown in FIG. 2 at each time. In this case, the finally displayed three-dimensional object is a moving image that changes over time.

  FIG. 3 shows an example of an image generated by the image processing device 20 and displayed on the display device 14. Image example 4 is a visualization of the measurement information shown in FIG. 2, and is displayed in a state in which the object 16 is present inside the three-dimensional object 6 indicating the measurement space. Since the user can move the virtual viewpoint by rotating the input device 15 and rotate the three-dimensional object 6 to a desired angle, the user can check the back side and the side surface of the object.

  In addition, the transmittance of a desired region such as a region other than the target 16 or the region of the target 16 in the three-dimensional object 6 can be changed by a user operation to be transparent or translucent. Thereby, even when there are a plurality of objects, the positional relationship and size thereof can be observed from a desired direction. When a region other than the target object of the three-dimensional object is displayed so as to be transparent, the three-dimensional object cannot be viewed, and only the target object can be viewed. In the following description, it is referred to as “three-dimensional object” including such a transparent region. When the three-dimensional object 6 is a moving image, the movement of the target object and the generation of a new target object can be observed. In addition, an object different from the measurement result, such as an arrow-shaped cursor 8, can be displayed in a state of entering the inside of the three-dimensional object 6.

  FIG. 4 shows the configuration of the image processing apparatus 20 in detail. In FIG. 4, each element described as a functional block for performing various processes can be configured by a CPU (Central Processing Unit), a GPU (Graphics Porcessing Unit), a memory, and other LSIs in terms of hardware. In terms of software, it is realized by a program for performing image processing.

  Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any one. Further, as described above, the image processing apparatus 20 performs transmission / reception control of data with other apparatuses in the image processing system 10. However, since general techniques can be applied to such processing, illustration is omitted here. ing.

  The image processing apparatus 20 acquires output information from the sensor, generates an image data generation unit 30 that generates 2D image data used for display, and a display processing unit 32 that draws a 3D object using the 2D image data. including. As described above, since the processing performed by the image data generation unit 30 and the processing performed by the display processing unit 32 can be performed independently, both are not provided in the same device, and may be individual devices having respective functions. Good.

  The image data generation unit 30 is different from a sensor output data acquisition unit 42 that acquires output data from the sensor group 12, a slice image generation unit 44 that generates data of a two-dimensional image for each slice plane from which a distribution is acquired, and a sensor axis. An image axis conversion unit 50 that generates similar two-dimensional image data for a plurality of planes having a predetermined angle with the axis, a vector value information storage unit 46 that stores data generated by the sensor output data acquisition unit 42, and a slice image A slice image storage unit 48 that stores data generated by the generation unit 44 and the image axis conversion unit 50 is included.

  The sensor output data acquisition unit 42 acquires physical quantity distribution information for each slice plane from the sensor group 12, and generates vector value information in which position coordinates and vector values are associated as shown in FIG. The vector value information is temporarily stored in the vector value information storage unit 46 in association with the slice plane identification information, and the fact is notified to the slice image generation unit 44 and the image axis conversion unit 50. Here, the identification information of the slice plane is constituted by the orientation of the sensor axis and the position on the sensor axis.

  Based on the vector value information stored in the vector value information storage unit 46, the slice image generation unit 44 generates 2D image data that is the basis of the 3D object for each slice plane. Hereinafter, such a two-dimensional image is referred to as a “slice image”. The slice image holds an alpha value representing the transmittance in addition to the color space information for each pixel. In the image display stage, slice images are arranged in the sensor axis direction, and alpha blending drawing is performed so that the inside of the three-dimensional object can be seen through. The color space information and alpha value setting method will be described in detail later. The generated slice image data is stored in the slice image storage unit 48 in association with the slice plane identification information. There is also a method of calculating the alpha value from the pixel value of the slice image. In this case, if the alpha value is calculated in the drawing unit and used for blending, it is not necessary to store the alpha value in the slice image storage unit.

  The image axis conversion unit 50 has a predetermined angle with respect to each of an x axis and a y axis orthogonal to a predetermined axis different from the sensor axis, for example, when the z axis is a sensor axis as shown in FIG. A plurality of planes are generated, and vector value information similar to that shown in FIG. 2 is generated for each plane. That is, data is reconstructed as a distribution for the plane by picking up values corresponding to positions on planes with different orientations from physical values on the slice plane measured by the sensor group 12.

  Then, two-dimensional image data corresponding to each plane is generated by the same processing as the slice image generation unit 44. Hereinafter, this two-dimensional image is also referred to as a “slice image”, but the axis relative to the plane at this time is referred to as an “image axis” and is distinguished from a “sensor axis”. The slice image data generated by the image axis conversion unit 50 is also stored in the slice image storage unit 48 in association with the orientation of the image axis and the position on the axis.

  The display processing unit 32 additionally displays a slice image management unit 52 that manages slice images used for drawing according to the position of the viewpoint, a drawing memory 56 that sequentially stores image data necessary for drawing, a cursor, and the like. An additional object image storage unit 58 that stores object image data and an image drawing unit 54 that draws a three-dimensional object using slice image data are included.

  The slice image management unit 52 switches the axis of the slice image used for drawing by moving the viewpoint, and reads necessary slice image data from the slice image storage unit 48 to the drawing memory 56. When an object is generated immediately from the slice image generated by the image data generation unit 30, the slice image storage unit 48 may also serve as the drawing memory 56. On the other hand, in a mode in which all slice image data is stored in the slice image storage unit 48 mounted in a secondary storage device or the like and loaded into the drawing memory 56 at another occasion as a display stage, it is necessary for drawing. The drawing process is performed while loading the data gradually.

  When drawing a three-dimensional object, basically, slice images are superimposed by alpha blending processing in the order of distance from the viewpoint. Therefore, the loading of the slice image data to the drawing memory 56 is basically performed in that order. Thereby, even when another slice image is required due to viewpoint movement or the like, the latency until display can be suppressed. This loading process may be performed by a memory controller or the like controlled by the slice image management unit 52.

  The slice image management unit 52 further performs interpolation between slice images in accordance with a decrease in the distance between the viewpoint and the three-dimensional object. Slice images are generated at discrete positions on the sensor axis or the image axis, and when the viewpoint approaches the object, the discontinuity becomes easy to be visually recognized. Therefore, a new slice image having a pixel value for interpolating the pixel values of adjacent slice images with respect to the position on the axis is generated. The generated slice images are inserted between the original slice images, and the discontinuity is made inconspicuous by narrowing the interval between the slice images. The number of slice images to be inserted may be gradually increased so as to be inversely proportional to the distance to the viewpoint, or may be increased stepwise by comparing the distance with a threshold value.

  When the image drawing unit 54 obtains notification from the slice image management unit 52 that necessary slice image data has been loaded into the drawing memory 56, the image drawing unit 54 draws a three-dimensional object using the data. Basically, each slice image is arranged at each position on the axis, and superposition is performed by sequentially projecting the slice image far from the viewpoint onto the screen coordinates. The three-dimensional object in the measurement space represented in this way becomes transparent or semi-transparent, for example, in a space without an object, depending on the setting of the alpha value. Therefore, the image drawing unit 54 further draws an additional object such as a cursor in such a space.

  The additional object is allowed to be moved by the user via the input device 15, or is moved or generated following the object. Therefore, the image drawing unit 54 calculates the display position of the additional object according to these modes, reads out the image data to the drawing memory 56, and draws it. Furthermore, the image drawing unit 54 may realize a stereoscopic view by performing the same three-dimensional object drawing process for a plurality of viewpoints. The relative position of the viewpoint in this case can be determined as appropriate according to the stereoscopic method to be introduced.

  Next, the operation of the image processing apparatus 20 that can be realized by the above configuration will be described. FIG. 5 is a flowchart illustrating a processing procedure in which the image data generation unit 30 generates slice image data. First, the sensor output data acquisition unit 42 acquires, from the sensor group 12, distributions of N physical quantities for a plurality of slice planes of the sensor axis in accordance with a user instruction or the like via the input device 15, and positions on each plane. And vector value information that associates a vector value composed of a set of physical quantities with each other (S10).

  In this processing, the vector value included in the vector value information may not be the N physical quantities transmitted from the sensor group 12 itself. For example, unnecessary physical quantities may be excluded from the elements in view of the information to be displayed in the end, or values obtained as a result of operations performed on different physical quantities may be added as new elements of vector values. Good. Further, masking by threshold determination may be performed, for example, when a certain physical quantity exceeds a threshold value, another physical quantity is set to 0. For example, this method is effective when displaying only an object below a certain temperature.

  In this way, a plurality of patterns of vector value information obtained by operating the measurement values may be generated, and the display target may be switched by an operation at the time of display. Specific processing rules for physical quantities are determined by the user and stored as additional data in a memory (not shown) or the like, and the sensor output data acquisition unit 42 refers to the vector value information when creating it.

  Next, the slice image generation unit 44 determines an alpha value based on the vector value for each pixel of the two-dimensional array set on each slice plane (S12). A specific method for determining the alpha value will be described later. Furthermore, the slice image generation unit 44 determines color information for each pixel based on the vector value, thereby generating slice image data that holds the color information and the alpha value as the pixel value, and stores the data in the slice image storage unit 48. (S14). The color information color system such as RGB and YCbCr is not limited. Further, as described above, when the alpha value is generated from the slice image at the time of drawing in the image drawing unit 54, the slice image generating unit 44 does not need to set the alpha value, so the slice image storage unit 48 does not store the alpha value. .

  The color information is determined according to a rule set in advance according to the type of physical quantity, display purpose, and the like. For example, the values measured by the three sensors 12_1, 12_2, and 12_3 may be expressed as RGB as red luminance, green luminance, and blue luminance, respectively. In this case, in the finally displayed three-dimensional object, the substance whose physical quantity measured by the sensor 12_1 increases is red, the substance whose physical quantity measured by 12_2 increases is green, and the substance whose physical quantity measured by 12_3 increases is blue. Appears strongly and can be color-coded by substance.

  Alternatively, one value measured by a certain sensor 12_1 may be substituted for all of R, G, and B, and the physical quantity may be represented by white luminance. The rules for determining the alpha value and the color information are appropriately determined by the user according to the display contents and display purpose, and stored in a memory (not shown). Alternatively, it may be set on the spot via the input device 15 while confirming the actual display.

  The processes in S12 and S14 are repeated until all slice images for the sensor axis are generated (N in S16, S12, and S14). When all the slice images are generated, the image axis conversion unit 50 prepares a plurality of slice planes with respect to a predetermined image axis at positions where physical quantities are obtained, and a vector composed of a position on each plane and a set of physical quantities Vector value information in which values are associated is generated for each image axis (Y in S16, N in S18, S20, S10). After that, similarly to the above, the process of determining the alpha value and color information for the pixels set on each plane and generating and storing slice images is repeated until all slice images for each axis are generated (S12, S14, N of S16).

  If all slice images are generated for all image axes (Y in S18), the process from S10 to S20 is repeated (N in S22) if the physical quantity can be acquired by another sensor axis. If slice images are generated for all sensor axes, the process is terminated (Y in S22). In order to suitably draw a three-dimensional object to be finally displayed in the present embodiment, it is desirable to prepare slice images for three axes. Most simply, slice images are generated with respect to the x-axis, y-axis, and z-axis that are orthogonal to each other. Any of these three axes may be used as the image axis, or the image axis may not be included.

  That is, the image axis conversion unit 50 does not need to perform processing as long as the three axes can be measured as sensor axes. On the other hand, when the sensor axis is only in one direction, the image axis conversion unit 50 generates slice images for the other image axes, and the branch of S22 is eliminated. When the sensor axis is two directions, the image axis conversion unit 50 generates a slice image using the remaining one direction as an image axis. For example, the setting of the image axis is appropriately changed according to the direction that can be set as the sensor axis and the number thereof. Good.

  FIG. 6 shows an example of a method for determining the alpha value of the slice image in S12 of FIG. The upper side of the drawing schematically represents vector value information in a certain slice plane 19i, and the lower side schematically represents an alpha value set in the corresponding slice image 100a. In the slice plane 19i, for example, at least one of the vector values greatly changes in the region 102 where the object exists and the other region 104. In the figure, the difference is represented by two types of shading.

In order to set an alpha value reflecting this difference, the following conditional branch is performed.
if max (V_1, V_2, ..., V_N) <Th
A = 0;
else A = 1;

  Here, Th is a predetermined threshold value, and A is an alpha value given to each pixel. In other words, the alpha value is 0 if the maximum value of the vector values (V_1, V_2,..., V_N) is smaller than the threshold value Th, and the alpha value is 1 if it is greater than or equal to the threshold value Th. In the slice image 100a of FIG. 6, the region 102 has an alpha value of 1 and the region 104 has an alpha value of 0 in white and black, respectively. The position where the vector value is held in the slice plane of the vector value information may be different from the position where the pixel is defined in the slice image. In this case, the alpha value is determined at the former position and is interpolated accordingly. The value for each pixel is determined.

  When the above conditions are used, when the types of physical quantities constituting the vector value are different, all values are normalized in order to align the scales. Further, conditional branching may be performed after a physical quantity to be subjected to branch determination is selected. When a physical quantity that increases in the range of the object is included in the vector value and an alpha value is given by such conditional branching, the area without the object becomes transparent and the area with the object becomes opaque. As a result, when α blending is drawn, only the area without the object can be viewed from the other side.

FIG. 7 shows another example of a technique for determining the alpha value of a slice image. The way of representing the figure is the same as in FIG. The conditional branch in this case is set as follows.
if max (V_1, V_2, ..., V_N) <Th
A = 0;
else A = max (V_1, V_2, ..., V_N);

  Of the vector values (V_1, V_2,..., V_N), if the maximum value is smaller than the threshold value Th, the alpha value is 0 as in FIG. 6, but the maximum value is greater than or equal to the threshold value Th. When this occurs, this maximum value itself is taken as the alpha value. Here, it is assumed that each physical quantity constituting the vector value is normalized. In the slice image 100b in the figure, the fact that the alpha value of the region 102 changes in the range of Th ≦ A ≦ 1 is represented by gradation.

  According to such conditional branching, for example, a change in physical quantity inside the object can be represented by a color transmittance. Compared with the example shown in FIG. 6, the alpha value near the contour of the object can be changed gently, so that even if jaggy or noise occurs in the contour, it is less noticeable.

There are various other conditions for determining the alpha value from the vector value. For example, contrary to the case of FIG. 7, the following conditional branch may be set.
if max (V_1, V_2, ..., V_N) <Th
A = max (V_1, V_2, ..., V_N);
else A = 1;

  According to this setting, the alpha value can be changed according to the magnitude of the physical quantity constituting the vector value in an area where there is no object. For example, the ambient temperature and ambient humidity of the object can be represented by color transmittance.

Further, by focusing on only one physical quantity V_i among the physical quantities constituting the vector value, the following conditional branch is also conceivable.
if V_i <Th
A = 0;
else A = 1;
Or
if V_i <Th
A = 0;
else A = V_i;

  The above two conditional branches are the same as those shown in FIGS. 6 and 7 except that one physical quantity V_i is subject to branch determination. For example, when the physical quantity that differs depending on the substance becomes high, the substance to be displayed as the object can be switched by switching the target physical quantity V_i.

Further, two threshold values Th1 and Th2 (Th1 <Th2) may be provided, and the following conditional branch may be set.
if max (V_1, V_2, ..., V_N) <Th2 || min (V_1, V_2, ..., V_N)> Th1
A = 0;
else A = 1;

  That is, the vector value (V_1, V_2,..., V_N) has a maximum value smaller than the larger threshold value Th2 or a minimum value larger than the smaller threshold value Th1, that is, the vector value is a predetermined value. When it falls within the intermediate range, it is made transparent with an alpha value of 0. In this example, in a situation where the physical quantity acquired by one sensor is small and the physical quantity acquired by another sensor is large in the area of the target object, if either sensor reacts, they are all displayed as the target object. Applicable when you want to

  As shown in FIGS. 6 and 7, when the alpha value setting is branched depending on the magnitude relationship between the threshold value Th and the maximum value of the vector value, a portion where the maximum value is in the vicinity of the threshold value Th, such as the vicinity of the contour of the object. Since the alpha value setting changes greatly with a slight change in vector value, noise tends to occur during display. FIG. 8 is a diagram for explaining a technique for suppressing the generation of such noise in setting the alpha value.

  First, in the upper left graph, the maximum value among vector values is shown on the horizontal axis, and the alpha value A determined by the conditional branch shown with reference to FIG. 7 is shown on the vertical axis. In this method, the final alpha value A is determined by converting the alpha value determined in the conditional branch according to a predetermined rule. Therefore, the alpha value of the graph is a provisional value as shown in the figure. As shown in the graph, when the maximum vector value is smaller than the threshold value Th, the provisional value of the alpha value A is zero. When the maximum value becomes equal to the threshold value Th, the provisional value of the alpha value A becomes the maximum value Th.

  In order to suppress such a rapid change of the alpha value, a final alpha value A is calculated by applying a function f (A) as shown in the upper right of FIG. The function f (A), for example, rises gently from 0 at a provisional value of an alpha value that is smaller than the threshold Th by a predetermined width ΔA, and further 1 while the provisional value becomes larger than the threshold Th by a predetermined width ΔA. It is a function that reaches When the value obtained from such a function is multiplied by the provisional value of the alpha value A, a graph shown in the lower part of the figure is obtained.

  That is, when the maximum value of the vector value shown on the horizontal axis is smaller than the threshold value Th, the alpha value A is 0 as in the provisional value. When the maximum value becomes equal to the threshold value Th, the alpha value A is kept smaller than the provisional value as a result of multiplying the provisional value Th by f (Th). As a result, the alpha value does not change abruptly with a slight change in the maximum value of the vector value. The function f (A) is optimized according to the noise generation rate in the actual display image. Note that the alpha value A may be a value output by substituting the provisional value of the alpha value A into the function f.

  So far, the alpha value setting method has been relatively simple based on conditional branching. When it is desired to set the alpha value in a more complex scheme, for example, an alpha value may be prepared in advance for the position coordinates in the N-dimensional space composed of N physical quantities constituting the vector value. For example, when the alpha value is set with a different threshold value for each physical quantity constituting the vector value, the conditional branch may be complicated.

  Therefore, a table in which alpha values for all conditions are associated with position coordinates in the N-dimensional space, that is, combinations of N values, is created in advance. In this way, the alpha value can be directly subtracted from the vector value only by referring to the table without performing conditional branching processing or the like.

  Similarly, when a predetermined function is calculated using one or a combination of physical quantities constituting a vector value as a variable, and an alpha value is determined using the result, similarly, all combinations of N values of vector values are determined. On the other hand, the alpha value is calculated first and prepared as a table. As a result, even when the alpha value is obtained with a complicated scheme, the processing load and the time required for determination can be suppressed.

  FIG. 9 is a flowchart illustrating a processing procedure in which the display processing unit 32 performs display processing of a three-dimensional object. First, when the user inputs a display start instruction via the input device 15 (S30), the slice image management unit 52 sets a predetermined viewpoint coordinate to display an initial image of the three-dimensional object as an initial viewpoint, The slice image data necessary for drawing the three-dimensional object from the viewpoint is loaded into the drawing memory 56 (S32, S34). Actually, by loading sequentially from the necessary slice images as described above, it is possible to make the drawing process of the object in parallel.

  Next, the slice image management unit 52 adjusts the amount of the slice image according to the distance between the viewpoint and the three-dimensional object (S36). When a new slice image for interpolation is generated, it is stored in the drawing memory 56. At this time, the image data itself is inserted between the data of the preceding and succeeding slice images or is associated with the position on the axis so that the position of the insertion destination can be identified.

  Next, the image drawing unit 54 draws an additional object to be displayed inside the three-dimensional object in response to an instruction input from the user via the input device 15 (S38). As described above, the additional object is a cursor that is moved by the user's operation, a marker that is displayed in contact with the object, a line that represents the trajectory of the object, and the like, which can be selected by the user. The image drawing unit 54 determines the position for displaying the additional object based on the designation by the user, the outline of the target obtained by the edge extraction process, and the like, and uses the image data loaded in the drawing memory 56 to use the additional object. Draw.

  Next, the image drawing unit 54 draws a three-dimensional object representing the measurement space by superimposing the slice images loaded in the drawing memory 56 by alpha blending in order from the viewpoint (S40). As will be described later, the drawing process of the additional object in S38 may be performed simultaneously with the alpha blending drawing in S40.

  When the user performs an operation of moving the viewpoint using the input device 15 (Y in S44), the processes from S34 to S40 are repeated in accordance with the change of the viewpoint. While the viewpoint movement or display stop operation is not performed, the display at that time is continued (N in S44, N in S46), and when the display stop operation is performed, the process ends (Y in S46).

  FIG. 10 schematically shows how the image drawing unit 54 performs alpha blending drawing in S40 of FIG. From the side closer to the viewpoint 114, the screen 116, slice images 110a, 110b,. At this time, the image projected on the point 115 on the screen 116 is an intersection 112a, 112b,... Of the line of sight 119 passing through the viewpoint 114 and the point 115 and the slice images 110a, 110b,. , 112n, 117 are superimposed from the viewpoint 114. In the case of the figure, the intersections 117, 112n,..., 112b, 112a are in this order.

FIG. 11 is a flowchart showing a processing procedure for obtaining RGB values at the point 115 on the screen shown in FIG. First, variables Ri, Gi, Bi (i is an integer equal to or greater than −1) are prepared, and RGB values, Rb, Gb, Bb at the intersection 117 of the background 118 are substituted for R ₋₁ , G ₋₁ , B ₋₁ , respectively. (S50). When the RGB value and alpha value of the intersection 112n on the slice image 110n farthest from the viewpoint are (R ₀ , G ₀ , B ₀ , A ₀ ), the intersection 117 of the background 118 and the intersection 112n on the slice image 110n are overlapped. When combined, the RGB values are as follows (S52, S54, S56, S58).

R = Kr × A ₀ × R ₀ + (1−Kr × A ₀ ) × R ₋₁
G = Kg × A ₀ × G ₀ + (1-Kg × A ₀ ) × G ₋₁
B = Kb × A ₀ × B ₀ + (1−Kb × A ₀ ) × B ₋₁
... (Formula 1)

Here, Kr, Kg, and Kb are coefficients of alpha values for red, green, and blue, respectively, when the alpha value held in the slice image is adjusted by color, and in the range of 0 <K <1, as necessary. Set. When adjustment by color is not necessary, Kr = Kg = Kb = 1. By sequentially performing R, G, and B in the above formula as new R ₀ , G ₀ , and B ₀ and incrementing i until the intersection of the last slice image (i = nslice-1) approaches the viewpoint ( (Y in S60, S62), an RGB value obtained by superimposing all the intersections of nslice slice images while allowing the transparent and translucent states is obtained (N in S60).

However, the drawing processing procedure is not limited to this. For example, it is conceivable to obtain RGB values as follows. That is, when Equation 1 is generalized as an RGB value when the k + 1th slice image (i = k) is overlaid, the following is obtained.
R ′ _k = Kr × A _k × R _k + (1−Kr × A _k ) × R ′ _k−1
G ′ _k = Kg × A _k × G _k + (1−Kg × A _k ) × G ′ _k−1
B ′ _k = Kb × A _k × B _k + (1−Kb × A _k ) × B ′ _k−1
... (Formula 2)

In Expression 2, the RGB values after superposition are set as (R ′ _k , G ′ _k , B ′ _k ), and are distinguished from the RGB values (R _k , G _k , B _k ) of only the k + 1-th slice image. Using the relationship of Equation 2, if the final RGB value when superimposed to the last slice image (i = nslice-1) is retroactively developed to the RGB value of the background, the final RGB value (R ' _nslice-1 , G' _nslice-1 , B ' _nslice-1 ) can be expressed by a linear combination of RGB values for each slice image as follows.

Here, (Br _i , Bg _i , Bb _i ) are coefficients of the RGB values (R _i , G _i , B _i ) of the i + 1-th slice image, and are calculated from the alpha values of the slice image. Thus in embodiments to be determining the alpha value of all the slice images before drawing a final factor used _{(Br i, Bg i, Bb} i) Equation 3 in terms of the previously obtained according to the viewpoint RGB values may be calculated.

  FIG. 12 is a diagram for explaining processing in which the slice image management unit 52 determines necessary slice image data in accordance with the viewpoint in S34 of FIG. As described above, the slice image is an image of the distribution of physical quantities at discrete positions with respect to a predetermined axis. When the sensor axis is the z-axis in the vertical direction in the figure, the slice image that is basically generated is a plurality of parallel planes that form the xy plane like the slice image group 120. When using the slice image group 120 to display the three-dimensional object 126 viewed from the upper side or the lower side of the figure, it is possible to draw an image without a sense of incongruity by performing the superposition as described with reference to FIG.

  On the other hand, when the viewpoint is placed at a position close to the x-axis in the horizontal direction or the y-axis in the depth direction, the space between the slice images can be seen. Therefore, as described above, a slice image is generated for an image axis different from the sensor axis, or a physical quantity is measured for a plurality of sensor axes to generate a slice image group for the plurality of axes. The slice image group used by movement is switched. In the example shown in the figure, a slice image group 122 having the x axis as the image axis and a slice image group 124 having the y axis as the image axis are prepared.

  Then, the three-dimensional space around the three-dimensional object 126 to be displayed is divided, and the slice image group used for drawing is switched depending on which section the viewpoint is located in. For example, as indicated by the dotted lines in the figure, from each vertex of the three-dimensional object 126, a boundary line that makes a predetermined angle with the three sides of the three-dimensional object 126 forming the vertex is set. An approximately trapezoidal space having these boundary lines as side edges and each surface of the three-dimensional object as an upper surface is defined as one section.

  For example, in the same figure, when the viewpoint is in the section on the upper surface or the lower surface of the three-dimensional object 126, since the line of sight is closest to the z axis, the slice image group 120 having the z axis as an axis is used for drawing. Similarly, when the viewpoint is in the section on the left surface or the right surface of the three-dimensional object 126, the line of sight is closest to the x axis, and therefore the slice image group 122 having the x axis as an axis is used for drawing. When the viewpoint is in the section on the front surface or the back surface of the three-dimensional object 126, the line of sight is closest to the y axis, and therefore the slice image group 124 with the y axis as an axis is used for drawing.

  FIG. 13 is a diagram for explaining an example of a method for determining the angle of the boundary line. First, consider a slice image group having two orthogonal directions, in the example of FIG. This figure shows a state in which a slice image group 122 (dotted line) with the x axis as an axis and a slice image group 124 (solid line) with the y axis as an axis are arranged on the same xy plane and viewed from the z-axis direction. In the figure, when a three-dimensional object is viewed from the upper left, the line of sight passing through the vertex v changes as indicated by arrows 128, 129, and 130 depending on the position of the viewpoint.

The number of slice image groups having an intersection with each line of sight is as follows in the order of the slice image group 122 having the x axis as the axis and the slice image group 124 having the y axis as the axis.
Line of sight of arrow 128: 4/6 sheets Line of sight of arrow 129: 6/6 sheets Line of sight of arrow 130: 7/6 sheets

  That is, in the line of sight of the arrow 129, the same display image is obtained because the same number of alpha blending processes are performed regardless of which slice image group data is used. Therefore, the display image before and after switching is continuously connected by setting the direction of the arrow 129 as a boundary line. When the interval between the images of the slice image group 122 with the x axis as the axis is Wx and the interval between the images of the slice image group 124 with the y axis as the axis is Wy, the angle θ of the arrow 129 is | tan θ | = Wy / Wx. In order to satisfy, the slice image group is switched under the following conditions.

When tan θ | ≧ Wy / Wx, a group of slice images with the y axis as the axis When | tan θ | <Wy / Wx, a group of slice images with the x axis as the axis

When the above is expanded to three dimensions, and the slice image group with the line-of-sight vector as (X, Y, Z), z-axis, x-axis, and y-axis as Sz, Sx, and Sy, respectively, Become.
Sz when | Z / X | ≧ Wz / Wx and | Z / Y | ≧ Wz / Wy
When | Z / X | <Wz / Wx and | X / Y | ≧ Wx / Wy, Sx
When | Z / Y | <Wz / Wy and | X / Y | <Wx / Wy

  The line-of-sight vector serving as the boundary of the above condition corresponds to the boundary line indicated by the dotted line in FIG. Thereby, even if the slice image group used for display is switched according to a change in viewpoint, it is possible to seamlessly connect before and after switching. In the examples of FIGS. 12 and 13, the sensor axis or the image axis is perpendicular to the slice image. However, as described above, the present embodiment is not limited thereto. For example, in the case of a measurement system in which the slice image is not perpendicular to the sensor axis, the slice image generated by the image axis conversion unit 50 by reconstructing the measurement value may not be perpendicular to the image axis. The sensor axis and the image axis may not be orthogonal. In these cases, the boundary line can be set based on the same theory as described above, and the slice image switching and the alpha blending drawing process according to the viewpoint are the same.

  FIG. 14 is a diagram for explaining another method of drawing a three-dimensional object. In the case of the figure, slice image groups for the x-axis, y-axis, and z-axis are simultaneously used as texture images in three directions. A general computer graphics drawing process can be applied to the method of drawing the three-dimensional object 136 with the texture pasted on each surface in accordance with the position of the viewpoint 132. On the other hand, in the case of the present embodiment, since the slice image in each direction has an alpha channel as described above, superposition from pixels far from the viewpoint 132 is necessary.

  However, when using slice images in three directions at the same time, it is difficult to specify combinations of slice images in order from the viewpoint. Therefore, as shown in the figure, the three-dimensional object 136 is divided into a lattice shape by a slice image group in three directions. In the figure, straight lines intersecting each other represented in the three-dimensional object 136 represent the edge of each slice image. If the target is a three-dimensional array of small rectangular parallelepipeds obtained by such division, the order far from the viewpoint 132 can be easily specified.

  An arrow 138 in the figure schematically shows such an order. Then, the three-dimensional object 136 can be expressed in a transparent and translucent state by superimposing it on the screen using the pixel values of slice images constituting the surfaces of the rectangular parallelepipeds in the order of the arrows 138. In the case of this mode, the slice image management unit 52 loads data of slice image groups in three directions regardless of the position of the viewpoint.

  Next, a method in which the image drawing unit 54 draws the additional object in S38 of FIG. 9 will be described. In order to draw the additional object while floating in the semi-transparent space of the three-dimensional object, it is necessary to consider the drawing order with respect to the alpha blending drawing of the three-dimensional object. FIG. 15 schematically shows a processing procedure for drawing a three-dimensional object after drawing an additional object first as one method.

  First, a virtual rectangular parallelepiped 142 in which a three-dimensional object is finally drawn is prepared according to the position of the viewpoint, and the additional object 140 is drawn at a required position inside the rectangular parallelepiped 142 by a general method (S70). . In the example shown in the figure, the additional object 140 is an arrow-shaped cursor figure. Next, a three-dimensional object is drawn using the slice image by the above method (S72). By drawing the additional object 140 first, the additional object 140 remains as it is in an area where the alpha value is close to 0 and the transmittance is high.

  When the superimposition is completed up to the slice image closest to the viewpoint, it is possible to draw a state where the cursor floats as the additional object 140 in the three-dimensional object 146 (S74). For example, by making the cursor point to the target object 148 existing in the three-dimensional object 146, a presentation using display can be suitably performed. When the moving object is displayed as a moving image with time, the locus of the object may be displayed. Further, another target (product) may be artificially represented at a position predicted from the observed data. In this case, the calculation of the position of the product may be performed separately, and the user may input as an input value.

  FIG. 16 schematically shows a processing procedure of another method for drawing the additional object. In this example, the drawing of the three-dimensional object using the slice image and the drawing of the additional object are advanced in parallel. First, alpha blending drawing using a slice image is advanced to a position where an additional object is drawn (S80). At this time, the additional object portion 152a that should be in the space from the uppermost slice image 150a to the next slice image is drawn (S82).

  Subsequently, the slice image 150b positioned next to the slice image 150a is superimposed (S84). Then, the additional object portion 152b that should be in the space from the slice image 150b to the next slice image is drawn (S86). In the case of the figure, the entire additional object is drawn at this stage. After that, by superimposing the remaining slice images, the state where the cursor is floated as the additional object 140 in the three-dimensional object 146 can be drawn as in FIG. 15 (S88).

  FIG. 17 schematically shows a procedure for reading slice image data when the sensor group 12 displays the measurement result at a predetermined rate as a moving image of a three-dimensional object. In this case, a slice image group is generated in the slice image storage unit 48 for each of the measurement times t = 0, 1,. In practice, a slice image group in three directions is generated for each time, but the illustration is omitted here.

  Especially when displaying moving images using such slice images, it is necessary to load and draw new slice image data in accordance with changes in both the viewpoint and time parameters. Desired. The same applies to the case where the display is changed only by moving the viewpoint without using the moving image. When the slice image storage unit 48 is configured as a secondary storage device, data transfer processing to the drawing memory 56 tends to cause drawing delay.

  Therefore, the slice image management unit 52 loads the slice images necessary for drawing into the drawing memory 56 in order. For example, a slice image farthest from the viewpoint position is specified, and the slice image data is loaded in order. At this time, a load unit composed of a plurality of slice images may be formed according to the bandwidth of the bus serving as a transfer path, and the load may be loaded in the load unit. In the example of the figure, data of three slice images indicated by thick frames are loaded simultaneously as a load unit.

  Immediately after the first slice image data is loaded, the image drawing unit 54 sequentially reads the slice image data from the drawing memory 56 and executes alpha blending drawing. In parallel with this drawing processing, the slice image management unit 52 loads the subsequent slice image data in units of load. When the loading of the slice image group at t = 0 is completed, the slice image group at t = 1 is similarly loaded. In this embodiment, since slice images are sequentially overlapped to draw a three-dimensional object, such parallel processing is possible, and as a result, response to a viewpoint movement request can be improved and smooth display of a moving image can be realized. .

  According to the present embodiment described above, a slice image having an alpha channel is created from layered image information obtained by acquiring cross-sectional information of an object on a plurality of parallel planes, and alpha blending is performed, so that the measurement space is 3 Expressed as a dimensional object. As a result, it is possible to easily visualize the object and its surroundings in an intuitively understandable form using a general image processing function without increasing the processing load and necessary resources.

  Since the alpha value and color information decision rules can be changed variously at the stage of generating the slice image, it is easy for the user who creates the image to perform processing suitable for the display contents and the display purpose. In addition, even if the first acquired laminated image information is for one axis, slice images for a plurality of axes are generated in advance, and the slice image to be used is switched according to the viewpoint position when displaying a three-dimensional object. Furthermore, the amount of slice images is adjusted according to the proximity of the viewpoint. With these configurations, even if the source data is a discrete two-dimensional image, a three-dimensional object can be displayed from all directions without a sense of incongruity. Also, slice images for a plurality of axes can be generated when necessary for drawing. In this way, the memory capacity for storing the slice image can be reduced.

  Furthermore, since a transparent and translucent space can be generated in the three-dimensional object, an additional object such as a cursor can be displayed in the three-dimensional object. As a result, it is easy to point an object, attach a marker that follows the object, or visualize a trajectory, and realize a suitable display in a presentation or the like.

  Further, when displaying a three-dimensional object, the order of slice images to be used can be determined according to the position of the viewpoint. Therefore, if data is loaded from the secondary storage device to the memory in that order, the loading process and the drawing process are performed. Can be parallel. Thereby, the latency to display can be suppressed even in a situation where the loading frequency of the slice image data is increased by moving the viewpoint or making the three-dimensional object a moving image. As a result, a smooth display change can be realized without increasing the memory capacity.

  The present invention has been described based on the embodiments. Those skilled in the art will understand that the above-described embodiment is an exemplification, and that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  For example, the slice plane of the present embodiment is a plane having a predetermined angle with the sensor axis or the image axis, but is not limited to a plane and may be a curved surface such as a spherical surface or a curved surface with fluctuations. FIG. 18 schematically shows the shape of a slice image when measuring a distribution with respect to a partially cylindrical curved surface or a partially spherical surface. In the figure, the sensor group 208 measures the measurement space in which the object 216 exists, and the distribution of the measured values at this time has a sensor axis on the z-axis in the vertical direction in the figure and a plurality of parallel lines centering on the sensor axis. For the curved surfaces 210a, 210b,..., 210n. In the figure, these curved surfaces are indicated by fan-shaped curves in a cross section passing through the sensor axis.

  For the slice curved surfaces 210a, 210b,..., 210n, the slice image generation unit 44 generates a slice image as described in the embodiment. The curved surface is the same as the curved surface. By treating the slice image as a texture, the same processing can be performed regardless of whether the shape is flat or curved. The image axis conversion unit 50 prepares a plurality of planes or curved surfaces that cross the slice curved surfaces 210a, 210b,..., 210n at a predetermined angle, and extracts the measured values on the surfaces to extract the slice curved surfaces 210a, 210b, ..., a slice image group having a different direction from 210n is generated.

  In the example shown in the figure, slice image groups are generated for planes 212a, 212b,..., 212n perpendicular to the drawing that cross the slice curved surfaces 210a, 210b,. Furthermore, a slice image group arranged in the depth direction in the figure is also generated. By performing drawing using the three-direction slice image group generated in this way, it is possible to display a partial cylindrical or partial spherical three-dimensional object. In this case, the drawing is performed in the same order as shown in FIG. 14 for each small solid divided by the slice image group in each direction.

  As described above, in this embodiment, a three-dimensional object can be drawn by a general computer graphics technique by treating an image as a texture regardless of the shape of the slice image. Therefore, the measurement result can be visualized similarly and easily if the shape and the intersecting angle are known.

DESCRIPTION OF SYMBOLS 10 Image processing system, 12 Sensor group, 14 Display apparatus, 15 Input apparatus, 20 Image processing apparatus, 30 Image data generation part, 32 Display processing part, 42 Sensor output data acquisition part, 44 Slice image generation part, 46 Vector value information Storage unit, 48 slice image storage unit, 50 image axis conversion unit, 52 slice image management unit, 54 image drawing unit, 56 drawing memory, 58 additional object image storage unit.

Claims (16)

A slice image generation unit that generates slice image information based on pixel values including an alpha value, determined based on physical quantities acquired at each position on a plurality of slice planes defined in a space including an observation target;
In order to display a three-dimensional object including the observation object, an image drawing unit that draws the slice image information in an alpha blending manner according to the input viewpoint position;
With
The image drawing unit draws a preset additional object inside the three-dimensional object.   The image processing apparatus according to claim 1, wherein the additional object is an object that moves following the observation target inside the three-dimensional object.   The image processing apparatus according to claim 1, wherein the additional object is an object whose position in the three-dimensional object can be selected by a user operation.   The image processing apparatus according to claim 1, wherein the image drawing unit draws the additional object based on the position of the observation target specified inside the three-dimensional object.   Since the image drawing unit displays a three-dimensional object including the observation object at different times, the slice image information acquired at different times is sequentially alpha-blended according to the input viewpoint position. The image processing apparatus according to claim 1, wherein the image processing apparatus performs drawing.   The image processing apparatus according to claim 1, wherein the image drawing unit draws the three-dimensional object by alpha blending drawing after drawing the additional object.   The image drawing unit generates a drawing unit of the additional object by dividing the model of the additional object by the plane of the arrayed slice image information, and draws one of the slice image information sandwiching one drawing unit. Then, the additional object is drawn inside the three-dimensional object by drawing the drawing unit and then drawing the other slice image information. Image processing apparatus.   The image processing apparatus according to claim 1, wherein the slice image generation unit determines the alpha value by a user setting operation.   The image processing apparatus according to claim 1, wherein the slice image generation unit determines an alpha value at the position based on at least one value of a plurality of types of physical quantities acquired at the positions.   The image processing apparatus according to claim 1, wherein the slice image generation unit determines an alpha value at the position based on a combination of a plurality of types of physical quantities acquired at the positions. The slice image generation unit further generates the slice image information for each surface based on physical quantities at positions on a plurality of surfaces in parallel planes in different directions with respect to the plurality of slice surfaces,
The image processing apparatus according to claim 1, wherein the image drawing unit switches slice image information to be used according to a viewpoint position.   As the distance between the viewpoint and the three-dimensional object decreases, new slice image information is generated by interpolating the pixel values of the corresponding pixels of the adjacent slice image information that are arranged. The image processing apparatus according to claim 1, further comprising a slice image management unit for interpolation.   The image drawing unit arranges the slice image information with respect to axes in three directions intersecting at a predetermined angle, and performs alpha blending drawing in the order of distance from the viewpoint in a solid unit delimited by the plane of each slice image information. The image processing apparatus according to any one of claims 1 to 6 and 8 to 12, wherein a dimensional object is displayed.   The image drawing unit draws a plurality of the three-dimensional objects for a plurality of viewpoints having a predetermined positional relationship, and displays the three-dimensional objects stereoscopically by simultaneously or sequentially displaying them. The image processing apparatus according to any one of 1 to 13. At least one sensor for acquiring a physical quantity at each position on a plurality of slice planes defined in a space including an observation object;
In order to display a slice image generation unit that generates slice image information based on a pixel value including an alpha value determined based on the physical quantity, and a three-dimensional object including the observation object, the display unit according to the input viewpoint position An image processing apparatus including an image drawing unit that draws slice image information in an alpha blending manner;
Consists of
The image drawing system draws a preset additional object inside the three-dimensional object. Generating slice image information with pixel values including alpha values determined based on physical quantities acquired at each position on a plurality of slice planes defined in a space including an observation object;
Alpha blending rendering of the slice image information according to the position of the input viewpoint to display a three-dimensional object including the observation object;
Including
The image processing method by the image processing apparatus, wherein the alpha blending drawing step draws a preset additional object inside the three-dimensional object.