JP2013027433A - Image processing device and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique by which a user can easily grasp a three-dimensional structure of a portion corresponding to an ultrasonic tomographic image being imaged.SOLUTION: When opacity in each position in volume data is set in the process of volume rendering, it is assumed that light beam tracing from a viewpoint for calculating the opacity intersects a cross section corresponding to the ultrasonic tomographic image in the volume data. In this case, with respect to the opacity in each position in a section included in the volume data of the light beam, the opacity in a position which is farther from the cross section is set to be smaller.

Description

  The present invention relates to an image processing technique for processing a medical image, and particularly to a technique for processing other volume data and presenting it together with an ultrasonic tomographic image when performing ultrasonic image diagnosis.

  In the medical field, after identifying the position of a lesion in volume data obtained by imaging a subject with a modality such as MRI (hereinafter referred to as reference volume data), the site is imaged and observed by an ultrasonic diagnostic apparatus. In some cases, image diagnosis is performed by the following procedure. In order to support such diagnosis, the position and orientation of the ultrasonic probe relative to the subject are measured, and a two-dimensional tomographic image corresponding to the ultrasonic tomographic image being captured is cut out from the reference volume data. For example, Japanese Patent Application Laid-Open No. H10-228667 proposes a display device. In such an apparatus, it is possible to display the lesioned part in the reference volume data and the ultrasonic tomographic image of the lesioned part in comparison.

  Further, in Patent Document 1, the entire image of the reference volume data is subjected to volume rendering, and the position of the ultrasonic tomographic image being captured is presented in the same space.

JP 2008-279272 A

  However, in such an apparatus, since the tomographic image of the reference volume data corresponding to the ultrasonic tomographic image being captured is presented as a two-dimensional image, there is a problem that it is difficult to grasp the three-dimensional structure of the reference volume data. there were. Further, since volume rendering of the entire image of the reference volume data is performed, it is difficult to grasp the three-dimensional structure of the part corresponding to the ultrasonic tomographic image being captured. In particular, when an ultrasonic tomographic image is taken, the request to compare with the three-dimensional structure of the lesioned part in the reference volume data cannot be satisfied.

  The present invention has been made in view of the above problems, and provides a technique for allowing a user to easily grasp the three-dimensional structure of a part corresponding to an ultrasonic tomographic image being captured.

  In order to achieve the object of the present invention, for example, the image processing apparatus of the present invention performs rendering of volume using volume data of a subject, thereby rendering an image of the volume data viewed from a specified viewpoint. And display means for displaying the image, an acquisition means for acquiring an ultrasonic tomographic image of the subject obtained by an ultrasonic probe, and in the volume data in the volume rendering process When setting the opacity at each of the positions, if the ray traced from the viewpoint for obtaining the opacity intersects the cross section corresponding to the ultrasonic tomographic image in the volume data, The opacity at each position in the section included in the volume data is a position where the distance from the cross section is larger. Opacity definitive is characterized in that it comprises a control means for controlling the rendering means to set so that smaller.

  According to the configuration of the present invention, the user can easily grasp the three-dimensional structure of the part corresponding to the ultrasonic tomographic image being captured.

The figure which shows the structural example of a system. The figure which shows the example of a setting of a projection surface. The figure explaining the opacity in the attention point P. The figure which shows an example of a transfer function. The figure which shows each of MRI volume data, a visual field plane, and an attention cross section. 10 is a flowchart of processing performed by the image processing apparatus 200. The flowchart which shows the detail of the process in step S107. The figure explaining the process in step S205. The figure which shows the attention cross section as a curved surface. The figure which shows the example of arrangement | positioning of the attention cross section in the modification 1-2. The figure explaining the production | generation process of the filter processed image. 10 is a flowchart of processing performed by the image processing apparatus 200. The flowchart which shows the detail of the process in step S300.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiment described below shows an example when the present invention is specifically implemented, and is one of the specific examples of the configurations described in the claims.

[First Embodiment]
The image processing apparatus according to the present embodiment uses MRI volume data of a subject imaged in advance as reference volume data, and a correspondence relationship between an image that has been volume-rendered based on the reference volume data and an ultrasonic tomographic image of the subject Present. In this volume rendering, a transfer function that increases the transparency (decreases the opacity) as the distance from the cross section (the cross section of interest) in the MRI volume data corresponding to the ultrasonic tomogram increases. Thereby, the three-dimensional structure of the part corresponding to the ultrasonic tomographic image can be easily grasped. In addition, for a pixel in which a ray to be traced during volume rendering (in the process of volume rendering) does not intersect the cross section of interest, volume rendering using a normal transfer function is performed. Thereby, it is possible to grasp the three-dimensional structure of the part corresponding to the ultrasonic tomographic image while grasping the three-dimensional position of the ultrasonic tomographic image in the subject.

  First, a configuration example of a system according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, the system according to the present embodiment includes an image processing apparatus 200, a display apparatus 100, a database (DB) 250, an ultrasonic diagnostic apparatus 220, an ultrasonic probe 300, and a position / orientation sensor 301.

  As is well known, the ultrasonic probe 300 irradiates ultrasonic waves, and collects data of a portion of the subject 400 (cross section 302 in FIG. 1) as ultrasonic data. The ultrasonic probe 300 sends the collected ultrasonic data to the ultrasonic diagnostic apparatus 220.

  In addition, a position and orientation sensor 301 is attached to the ultrasonic probe 300. A position and orientation sensor of various measurement methods can be applied to the position and orientation sensor 301, and any position and orientation sensor may be applied in the present embodiment. The position and orientation sensor 301 measures its own position and orientation, and hereinafter, the position and orientation measured by the position and orientation sensor 301 is handled as the position and orientation of the ultrasonic probe 300. However, the position and orientation measured by the position and orientation sensor 301 may be treated as the position and orientation of the ultrasonic probe 300 by applying a specified bias or performing a specified conversion. Furthermore, in this embodiment, it is only necessary to acquire the position and orientation of the ultrasonic probe 300. For example, the position and orientation of the ultrasonic probe 300 is obtained using an image captured by the ultrasonic probe 300 captured by the camera. Such a method may be adopted. However, in the present embodiment, any method may be adopted as long as the position and orientation of the ultrasonic probe 300 can be acquired. In any case, the position and orientation of the ultrasonic probe 300 (in this embodiment, the position and orientation measured by the position and orientation sensor 301) are input to the image processing apparatus 200.

  The ultrasound diagnostic apparatus 220 uses the ultrasound data from the ultrasound probe 300 to generate an ultrasound tomographic image of the site of the subject 400 irradiated with ultrasound (section 302 in FIG. 1). Since a technique for generating an ultrasonic tomographic image from data collected by the ultrasonic probe 300 is well known, a description thereof will be omitted. Then, the ultrasonic diagnostic apparatus 220 inputs the generated ultrasonic tomographic image to the image processing apparatus 200.

  The DB 250 stores volume data of the subject 400 imaged in advance by MRI or the like, and the image processing apparatus 200 can appropriately read this volume data. In this embodiment, this volume data is MRI volume data, but may be volume data collected by other methods.

  Next, the image processing apparatus 200 will be described. The CPU 11 uses the computer program and data stored in the RAM 12 and the ROM 13 to control the operation of the entire image processing apparatus 200 and executes each process described later as what the image processing apparatus 200 performs.

  The RAM 12 is an area for temporarily storing various information received from the computer program and data loaded from the external storage device 15, the DB 250, the ultrasonic diagnostic apparatus 220, and the position / orientation sensor 301 via the I / F 16. Have. Further, the RAM 12 has a work area used when the CPU 11 executes various processes. That is, the RAM 12 can provide various areas as appropriate. The ROM 13 stores setting data and a boot program for the image processing apparatus 200.

  The operation unit 14 is configured by a keyboard, a mouse, and the like, and can input various instructions to the CPU 11 when operated by a user of the image processing apparatus 200.

  The external storage device 15 is a mass information storage device represented by a hard disk drive device. The external storage device 15 stores an OS (Operating System) and computer programs and data for causing the CPU 11 to execute processes described later as those performed by the image processing apparatus 200. The external storage device 15 also stores information handled as known information in the following description. Computer programs and data stored in the external storage device 15 are appropriately loaded into the RAM 12 under the control of the CPU 11 and are processed by the CPU 11.

  The DB 250, the ultrasonic diagnostic apparatus 220, and the position / orientation sensor 301 are connected to the I / F 16 and communicate with each device via the I / F 16. In FIG. 1, for simplification, the I / F with each device is also used as the I / F 16, but an I / F may be provided for each device. Each of the above parts is connected to the bus 17.

  When receiving the ultrasonic tomographic image of the cross section 302 sent from the ultrasonic diagnostic apparatus 220, the image processing apparatus 200 displays this ultrasonic tomographic image as the ultrasonic tomographic image 102 on the display screen of the display device 100.

  In addition, the image processing apparatus 200 performs volume rendering using the MRI volume data of the subject 400 read from the DB 250, thereby generating an image of the MRI volume data of the subject 400 viewed from a specified viewpoint. Then, the image processing device 200 displays the generated image on the display screen of the display device 100 as an overhead view (103).

  Further, the image processing apparatus 200 generates a two-dimensional tomographic image in the MRI volume data corresponding to the cross section 302, and uses the generated two-dimensional tomographic image as a reference image 101 corresponding to the ultrasonic tomographic image 102. Display on the display screen. Note that the display of the reference image 101 is not essential.

  Processing performed by the image processing apparatus 200 will be described in more detail with reference to FIG. 6 showing a flowchart of the processing. The process according to the flowchart of FIG. 6 is executed by the CPU 11 using a computer program or data loaded from the external storage device 15 to the RAM 12. Therefore, unless otherwise specified, the main body of processing in each step shown in FIG.

  In step S101, the MRI volume data of the subject 400 is read (downloaded) from the DB 250 to the RAM 12. This reading may be performed in response to the CPU 11 detecting a reading instruction input by the user using the operation unit 14.

  In step S102, the ultrasonic tomographic image of the cross section 302 sent from the ultrasonic diagnostic apparatus 220 is received and temporarily stored in the RAM 12, and then this ultrasonic tomographic image is used as the ultrasonic tomographic image 102 of the display device 100. Display on the display screen. This display may be performed in response to the CPU 11 detecting a display instruction input by the user using the operation unit 14. Further, display on / off may be switched by this display instruction.

  In step S103, the position and orientation of the ultrasonic probe 300 sent from the position and orientation sensor 301 are received and temporarily stored in the RAM 12.

  In step S104, the MRI corresponding to the cross section 302 is obtained using the position and orientation acquired in step S103 (the position and orientation measured by the position and orientation sensor 301 when the ultrasonic probe 300 is collecting ultrasonic data of the cross section 302). Identify cross sections in volume data. Since the process of specifying the cross section in the MRI volume data corresponding to the cross section 302 using the position and orientation acquired in step S103 is a well-known technique, a description thereof will be omitted. Hereinafter, a cross section in the MRI volume data corresponding to the cross section 302 is referred to as a cross section of interest.

  If the position and orientation relationship between the coordinate system in which the position and orientation sensor 301 measures the position and orientation and the coordinate system of the space (virtual space) in which the MRI volume data is arranged is known, it corresponds to the position and orientation measured by the position and orientation sensor 301. The position and orientation in the virtual space can be specified by a known method. However, if the position / orientation measured by the position / orientation sensor 301 when the ultrasound probe 300 is collecting ultrasound data of the section 302 is used, the section in the MRI volume data corresponding to the section 302 is obtained by a known method. Can be identified.

  Next, in step S105, a two-dimensional tomographic image of the cross section of interest identified in step S104 is generated from the MRI volume data received in step S101 by a known technique. In step S106, the two-dimensional tomographic image generated in step S105 is displayed on the display screen of the display device 100 as the reference image 101.

  In step S107, an image of the MRI volume data of the subject 400 viewed from a specified viewpoint is generated by performing volume rendering using the MRI volume data. When performing volume rendering, a projection plane (view plane) is arranged between the MRI volume data and the viewpoint. The field plane is arranged so that the entire MRI volume data can be volume-rendered. The field plane may be arranged in a specified position and orientation, or the user adjusts the position and orientation using the operation unit 14. May be arranged.

  FIG. 5 shows the MRI volume data, the field plane, and the cross section of interest of the subject 400. As shown in FIG. 5, in step S107, a visual field plane 508 is arranged between the MRI volume data 500 of the subject 400 and a viewpoint (not shown). Reference numeral 501 denotes a cross section of interest in the MRI volume data 500 corresponding to the cross section 302.

  In step S107, volume rendering is performed using the MRI volume data, thereby generating an image of “MRI volume data 500 of the subject 400” viewed from a specified viewpoint on the projection plane.

  In step S108, the image generated in step S107 is displayed on the display screen of the display device 100 as an overhead view (103). The display layouts of the reference image 101, the ultrasonic tomographic image 102, and the overhead view (103) are not limited to the display layout shown in FIG. 1, and any display layout can be considered. In addition, the display method is not limited to a specific display method as long as the correspondence between the ultrasonic tomographic image 102 and the overhead view (103) can be presented to the user. Further, on / off of display of each image may be switched according to a user operation or the like.

  Next, details of the processing in step S107 will be described using the flowchart of FIG. In step S107, the above viewpoint is set in the virtual space, a line segment (light ray) from the set viewpoint to each pixel position on the projection plane is set, and whether the line segment intersects the target cross section. Depending on whether or not, the transfer function for determining the transparency at the pixel position is switched. In volume rendering, the transparency at each pixel position on the projection plane is determined based on this transfer function. That is, the volume rendering of the present embodiment is a function that switches the transfer function according to whether the line segment intersects the target section, and the transfer function when the line segment intersects is a function that controls opacity according to the distance from the target section. Other than that, you can think of it as normal volume rendering. That is, similarly to normal volume rendering, in this embodiment, a rendered image is obtained by performing ray tracing for each pixel position on the projection plane.

  7 is a process for determining a pixel value at the target pixel position on the projection plane. Actually, in step S107, the process according to the flowchart of FIG. 7 is performed for each pixel on the projection plane. Will be done about the position.

  In step S201, a specified viewpoint is set in the virtual space, and a line segment (line-of-sight vector) from the set viewpoint to the target pixel position on the projection plane is set.

  In step S202, it is determined whether or not the line segment set in step S201 intersects the target cross section. Since a technique for determining whether or not a cross section (regardless of a plane or an aspect) in the virtual space intersects with a line segment is a well-known technique, a description thereof will be omitted. As a result of this determination, if it is determined that they intersect, the process proceeds to step S203, and if it is determined that they do not intersect, the process proceeds to step S204.

  In step S204, the transfer function f that returns larger opacity (smaller transparency) f (v) as the value (voxel value) v at each position (each voxel) in the MRI volume data 500 is larger is used as the first function. select. In the case of the first function shown in FIG. 4A, opacity = 0 is returned if 0 ≦ voxel value v <a, and opacity is directly proportional to voxel value v if a ≦ voxel value v <b. , And when b ≦ voxel value v, the function returns opacity = 1. Such a first function can be expressed by the following equation (1).

  In step S203, the larger the voxel value v, the larger the opacity (smaller transparency) f (v, d) is returned, and the smaller the distance d from the cross section of interest, the smaller the opacity (greater transparency) f (v, The transfer function f that returns d) is selected as the second function. In the case of the second function shown in FIG. 4B, when the distance d = 0, the function is the same as the first function, but becomes smaller than the value of the first function as the distance d increases. It is a function that returns a value. Such a second function can be expressed by the following equation (2).

  Of course, the first function and the second function are not limited to these expressions, and any expression may be used as long as it has the above properties. For example, as the second function, a function that returns normal opacity if the distance from the target cross section is within c, and returns small opacity as the distance increases is used. Good. Such a function can be expressed by the following equation (3).

  When the distance from the target cross section is greater than c, a function that returns opacity = 0 may be used as the second function. Such a function can be expressed by the following equation (4).

  Further, the first function and the second function can be implemented as a function program, but may be implemented by a lookup table. The opacity (or transparency) corresponding to each voxel value v is registered in the first look-up table that implements the first function. In addition, the opacity (or transparency) for each combination of the voxel value v and the distance d is registered in the second lookup table that implements the second function.

  In step S205, the intersection position between the line segment obtained in step S201 and the MRI volume data 500 of the subject 400 is obtained. The process in this step will be described with reference to FIG. As shown in FIG. 8, since the intersection of the “line segment 890 passing through the viewpoint 800” obtained in step S201 and the MRI volume data 500 is the incident point 801 and the exit point 802, in the case of FIG. The positions of the incident point 801 and the outgoing point 802 are obtained. Next, a point is set at an appropriate sampling interval between the incident point 801 and the emission point 802 (in the section included in the MRI volume data 500 in the line segment 890). Hereinafter, the function selected in step S203 or step S204 is used for each of the incident point 801 and the emission point 802, and each point set at an appropriate sampling interval between the incidence point 801 and the emission point 802. Set the opacity. Hereinafter, each of the incident point 801 and the emission point 802 and each point set at an appropriate sampling interval between the incidence point 801 and the emission point 802 are referred to as target points.

  In step S206, the incident point 801 among the target points is set as the attention point P. In step S207, the processing differs depending on whether the first function is selected as the transfer function or the second function is selected.

  When the first function is selected as the transfer function, in step S207, the voxel value (voxel value) v at the point of interest P is acquired from the MRI volume data 500 and input to the first function. The return value from is acquired as the opacity for the point of interest P.

  When the second function is selected as the transfer function, in step S207, the voxel value (voxel value) v at the point of interest P is acquired from the MRI volume data 500, and the distance d between the point of interest P and the cross section of interest is obtained. Ask. The acquired voxel value v and the obtained distance d are input to the second function, and the return value from the second function is acquired as the opacity with respect to the attention point P. For example, when the above equation (2) is used as the second function, the opacity at the attention point P is inversely proportional to the distance from the attention section, as shown in FIG.

  Further, in step S207, the result of multiplying the acquired opacity and the color at the point of interest P is added to the pixel value (initially 0) obtained for the pixel of interest at this time, thereby obtaining the pixel of the pixel of interest. Update the value. The color may be obtained from MRI volume data.

  Next, in step S208, a target point adjacent to the target point set as the target point P in step S206 (a target point not yet set as the target point P) is newly set as the target point P.

  In step S209, it is determined whether or not the target point set as the target point P in step S208 is the emission point 802. If the result of this determination is that the point is the exit point 802, the process according to the flowchart of FIG. 7 ends and the process proceeds to step S108. If the point is not the exit point 802, the process returns to step S207.

  In this way, by performing the processing according to the flowchart of FIG. 7, the opacity for each point on the line segment from the viewpoint to the target pixel position on the projection plane can be determined. From the opacity, the pixel value at the target pixel position can be obtained.

  As described above, according to the present embodiment, the three-dimensional structure of the cross section in the MRI volume data 500 corresponding to the ultrasonic tomographic image can be shown in an easily understandable manner. In addition, it can be shown at the same time which area on the subject the ultrasonic tomographic image is taken.

  Further, by applying a transfer function such as Equation 2, a three-dimensional structure around the cross section 501 of interest can be drawn by volume rendering. Further, by switching the transfer function in this way, it is displayed in an easy-to-understand manner which part of the subject 400 is being scanned by the ultrasonic probe 300, and at the same time, 3 of the part corresponding to the ultrasonic tomographic image 102 is displayed. The dimensional structure can be easily grasped.

<Modification 1-1>
In the first embodiment, the cross section of interest is a flat surface, but may be a curved surface. An example in which the cross section of interest is a curved surface is a case where the breast is the subject. In a general imaging protocol in a mammary gland department, imaging by an MRI apparatus is often performed in a prone position (depressed position), and an examination by an ultrasonic diagnostic apparatus is often performed in a supine position (a position facing the chin). For this reason, the shape of the breast is different between MRI imaging and ultrasound examination, and the correspondence between the locations imaged by both is not simple. At this time, even if the section 302 scanned by the ultrasonic probe 300 is a plane, the section on the MRI volume data corresponding to this (representing the same part as the section 302) is a curved surface. That is, if this corresponding cross section is the target cross section, the target cross section 502 corresponding to the cross section 302 is a curved surface as shown in FIG.

  Note that the target cross section 502 can be obtained from the position of the cross section 302 by estimating the deformation of the subject by a known process such as a physical deformation simulation. In addition, when the cross section of interest 502 is a curved surface, it is necessary to correct the transfer function shown in Expression (2). That is, d in Equation (2) may be a distance from each voxel to the closest point on the cross section 502 of interest. Alternatively, in order to simplify the calculation, an approximate plane of the cross section 502 of interest is obtained, and the intersection of the normal section 502 drawn from each voxel to the approximate plane and the cross section of interest 502 is obtained, and this intersection is obtained from each voxel. You may ask for the distance to. Thus, it is clear from the above description that the above-described embodiment can be applied even when the attention cross section 502 is a curved surface.

<Modification 1-2>
In the first embodiment, the cross section 302 scanned with the ultrasonic probe 300 and the cross section of interest 501 determined according to the position of the cross section 302 represent the same part. However, the cross section 302 and the target cross section 501 may indicate different parts.

  As described above, in ultrasonic image diagnosis, there are cases in which a lesion is searched for in MRI volume data and the position is observed with an ultrasonic diagnostic apparatus. In that case, it is desirable that the position of the target cross section 501 always reflects the lesioned part.

  Therefore, in this modified example, as shown in FIG. 10, the position of the target cross section 503 is fixed to the lesioned part, and only the posture of the target cross section 503 is matched with the cross section 302. In this case, in step S101, when the MRI volume data of the subject 400 is read from the DB 250, “positional position information” registered in the DB 250 in association with the MRI volume data is also read. Then, after setting the position of the lesioned part represented by the acquired position information at the center of the attention section 503, only the posture of the attention section 503 is matched with the section 302. As described above, it is apparent from the above description that the above-described embodiment can be applied even in a case where only the posture is matched without matching the positions of the section 302 and the target section 503.

  Of course, more generally, it is clear from the above description that the above-described embodiment can be applied even in a case where some correspondence can be established between the position of the cross section 302 and the position of the target cross section.

[Second Embodiment]
In the present embodiment, in the configuration according to the first embodiment, the transfer function used for volume rendering is adjusted according to the imaging content of the ultrasonic tomographic image. Below, only the part changed from 1st Embodiment is demonstrated. That is, unless otherwise specified below, it is the same as in the first embodiment.

  The image processing apparatus according to the present embodiment determines a volume rendering transfer function according to the distribution of a portion suspected to be a lesion on the ultrasonic tomographic image 102. Thereby, it is possible to present to the user how the portion suspected of being a lesion is imaged on the MRI volume data.

  The present embodiment is different from the first embodiment in processing performed by the image processing apparatus 200. Processing performed by the image processing apparatus 200 in the present embodiment will be described with reference to FIG. 12 showing a flowchart of the processing. The flowchart of FIG. 12 performs the process in step S300 between step S102 and step S103 in the flowchart of FIG. 6, and when using the expression (2) in step S107, this expression corresponds to the process of step S300. It will change. That is, other than that is the same as the flowchart of FIG. However, hereinafter, the processing performed in steps S300 and S107 in the present embodiment will be described.

  In step S300, a weight value for the transfer function is obtained from the ultrasonic tomogram acquired from the ultrasonic diagnostic apparatus 220. Details of the processing in step S300 will be described using the flowchart of FIG.

  In step S301, the well-known image processing for detecting the lesioned part in the ultrasonic tomographic image is performed on the ultrasonic tomographic image acquired from the ultrasonic diagnostic apparatus 220, whereby the region of the lesioned part (affected part) is determined. To detect. In this embodiment, for each pixel constituting the ultrasonic tomographic image, the bit value is 1 when the luminance value of the pixel is smaller than a preset luminance value, and the luminance value of the pixel is larger than the preset luminance value. In this case, the bit value 0 is assigned. As a result, a binary image composed of bit values for each pixel of the ultrasonic tomographic image can be generated, so that an area composed of pixels to which the bit value 1 is assigned can be detected as a lesion area. Note that the method for detecting a region suspected of being a lesion is not limited to this, and various methods can be applied. For example, a region suspected of being a lesion may be detected using a more advanced detection method such as a vector concentration filter or pattern matching. Since all such detection methods are well-known techniques, description thereof will be omitted.

  In step S302, the region detected in step S301 is expanded in consideration of the alignment error between the MRI volume data and the ultrasonic tomographic image and the apparent difference in the size of the lesion. Specifically, the binary image is updated by updating the bit value for each pixel constituting the region including the region detected in step S301 to 1. Of course, this expansion process may be omitted as necessary.

  Next, in step S303, filtering processing using a low-pass filter is performed on the binary image updated in step S302, so that the filtered image (each pixel has a pixel value between 0 and 1). Is generated. Here, the position on the target cross section corresponding to each pixel position on the filtered image (each pixel position on the ultrasonic tomographic image), the pixel position on the filtered image corresponding to each position on the target cross section, Can be identified by a known technique.

  In step S107, the following processing is performed when the second function is selected. That is, for each corresponding point, a coordinate position on the filtered image corresponding to a position (x, y) where a perpendicular line perpendicular to the cross section of interest intersects the cross section of interest is obtained. Let the pixel value be a weight value for the corresponding point. Then, the result of multiplying the opacity calculated using the second function by this weight value is set as the opacity of the corresponding point.

  Here, it is assumed that the ultrasonic tomographic image generated by the ultrasonic diagnostic apparatus 220 is the image shown in FIG. FIG. 11B shows the binary image obtained by performing known image processing for detecting a lesion in an ultrasonic tomographic image on the ultrasonic tomographic image. FIG. 11C shows the result of expanding the lesion area in the binary image. In FIG.11 (c), it has expanded to the elliptical area | region containing the area | region of a lesioned part. FIG. 11D shows a filtered image obtained by performing a filtering process using a low-pass filter on the binary image shown in FIG.

  In this way, by highlighting the suspected lesion area in the ultrasonic tomogram, it is possible to show the part that should be noticed to the user and to clarify the part that should be noticed by not displaying the unnecessary part. Can be displayed.

  As described above, according to the present embodiment, by weighting the opacity in the vicinity of a portion suspected to be a lesion in an ultrasonic tomographic image, how the portion suspected to be a lesion is represented on the MRI volume data. It is possible to present with particular emphasis on whether the image is captured. Thereby, it is possible to more clearly display a notable part without displaying an unnecessary part.

<Modification 2-1>
In the second embodiment, filter processing using a low-pass filter is performed on a binary image, but this filter processing is not essential. In addition, in the case where detection processing is performed in step S301 so that the likelihood of a lesion is obtained for each pixel as a likelihood of 0 to 1, this likelihood image is directly used as a weighted image instead of the filtered image. It may be used.

[Third Embodiment]
In this embodiment, not a two-dimensional tomographic image but a three-dimensional tomographic image by volume rendering is displayed as the reference image 101. By displaying the reference image 101 and the ultrasonic tomographic image 102 in parallel, both can be compared with the same viewpoint / size. Further, by making the reference image 101 a three-dimensional tomographic image, the vicinity of the reference image 101 can be displayed simultaneously. This makes it easy to understand where the lesioned part is reflected in the ultrasonic tomographic image 102 when searching for a place where the lesioned part reflected in the reference image 101 is reflected in the ultrasonic tomographic image 102.

  In particular, if the reference image 101 is a two-dimensional tomographic image when the lesion is small, the reference image 101 shows a lesion in the reference image 101 due to the limit of the alignment accuracy between the MRI volume data and the ultrasonic tomogram 102. The sonic tomographic image 102 may not be reflected. At this time, if the reference image 101 is a three-dimensional tomographic image having a thickness, even if the position of the ultrasonic tomographic image 102 is shifted to some extent, a lesioned part can be reflected on the reference image 101.

  In the following, only the parts different from the first embodiment will be described, and unless otherwise noted, the same as the first embodiment. In the present embodiment, as shown in FIG. 2, the projection plane 507 is set as a plane in which the cross section of interest 501 is moved by a predetermined distance (for example, 10 cm) in the normal direction, and volume rendering using this projection plane is performed. Thus, the reference image 101 is generated on the projection plane.

  In the present embodiment, the following processing is performed in steps S105 and S106 described above. That is, in step S105, after setting the projection plane as described above, volume rendering is performed using the projection plane. In step S106, the result of the volume rendering is displayed on the display screen of the display device 100 as the reference image 101. indicate.

  As described above, according to the present embodiment, by displaying the reference image 101 as the three-dimensional tomographic image and the ultrasonic tomographic image 102 in parallel, both can be compared at the same viewpoint / same size, The vicinity of the reference image 101 can also be displayed at the same time.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (6)

Rendering means for generating an image of the volume data viewed from a prescribed viewpoint by performing volume rendering using the volume data of the subject;
An image processing apparatus having display means for displaying the image,
An acquisition means for acquiring an ultrasonic tomographic image of the subject obtained by an ultrasonic probe;
When setting the opacity at each position in the volume data in the volume rendering process, a ray traced from the viewpoint to obtain the opacity corresponds to the ultrasonic tomographic image in the volume data. The opacity at each position in the section included in the volume data in the ray is set so that the opacity at a position where the distance from the cross section is larger is smaller. An image processing apparatus comprising: control means for controlling The control means includes
Judgment for determining whether or not a ray passing through the viewpoint and the pixel intersects the cross-section for each pixel on the projection plane arranged between the viewpoint and the volume data to generate the image Means,
If it is determined that a ray passing through the viewpoint and the pixel of interest does not intersect the cross section, a first function that returns greater opacity as the voxel value is larger is selected, and a ray passing through the viewpoint and the pixel of interest is And selecting means for selecting a second function that returns greater opacity as the voxel value is larger and returns smaller opacity as the distance from the cross section is greater,
The rendering means is in the process of volume rendering,
When the selection means selects the first function, it is obtained by inputting the voxel value at each position in the section included in the volume data to the first function in the light ray passing through the viewpoint and the target pixel. Set the opacity to the opacity at each position,
When the selection unit selects the second function, the voxel value at each position in the section included in the volume data and the distance from the position to the cross section in the light ray passing through the viewpoint and the target pixel are calculated. 2. The image according to claim 1, wherein the opacity at each position in the volume data is set by setting the opacity obtained by inputting into the function of 2 to the opacity at each position. Processing equipment. Furthermore,
A region composed of pixels having a prescribed pixel value in the ultrasonic tomographic image is detected as a region of a lesion, and a binary image representing the region of the lesion and the region of irrigation with different pixel values; or A generating unit that generates a filtered image by performing a filtering process using a low-pass filter on the binary image;
The rendering means, when the selection means selects the second function, for each position in the section included in the volume data in a light ray passing through the viewpoint and the pixel of interest, a perpendicular line taken from the position to the cross section And the cross-sectional position of the binary image corresponding to the cross-point position, or the pixel value at the coordinate position on the filtered image as the weight value of the position,
The opacity at the position is set as a result of multiplying the opacity obtained from the second function for the position by the weight value of the position for each of the positions. Image processing apparatus.   The image processing apparatus according to claim 1, wherein the display unit further displays the ultrasonic tomographic image. Rendering means for generating an image of the volume data viewed from a prescribed viewpoint by performing volume rendering using the volume data of the subject;
An image processing method performed by an image processing apparatus having display means for displaying the image,
An acquisition step in which the acquisition unit of the image processing apparatus acquires an ultrasonic tomographic image of the subject obtained by an ultrasonic probe;
When the control means of the image processing apparatus sets opacity at each position in the volume data in the volume rendering process, a ray traced from the viewpoint to obtain the opacity is included in the volume data. If it intersects the cross section corresponding to the ultrasonic tomographic image, the opacity at each position in the section included in the volume data in the light ray is reduced, and the opacity at a position where the distance from the cross section is larger is smaller. And a control step of controlling the rendering means so as to set the image processing method.   The computer program for functioning a computer as each means of the image processing apparatus of any one of Claims 1 thru | or 4.