JP6433190B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method, and a program for generating a cross-sectional image from three-dimensional image data.

  In the medical field, a doctor displays a medical image obtained by imaging a subject (three-dimensional image data including a group of tomographic images representing information inside the subject) on a monitor, interprets the displayed medical image, and Make a diagnosis. Medical image acquisition devices (hereinafter referred to as modalities) for imaging 3D image data include magnetic resonance imaging devices (hereinafter referred to as MRI devices), computed tomography devices (hereinafter referred to as CT devices), and super-dimensional images. Examples include an ultrasonic imaging apparatus.

  More specifically, the doctor displays a cross-sectional image in a predetermined posture including a site of interest (hereinafter referred to as a site of interest) such as a lesion in 3D image data on the monitor. Then, observation and diagnosis of the site of interest are performed using the feature of how the image of the site of interest on the cross-sectional image and the image of the periphery thereof (hereinafter referred to as image feature) is a clue. As a cross-sectional image in a predetermined posture, an axial cross-section, sagittal cross-section, coronal cross-section, or a combination thereof (three cross-sections) is generally used.

  Patent Document 1 discloses a method of changing a cross section by rotating a posture of a cross section in an arbitrary direction under a constraint condition that a point specified in three-dimensional image data is always included. A technique for generating and displaying an image from three-dimensional image data is disclosed. Further, Patent Document 2 discloses a method of rotating the posture around a straight line passing through two selected points of interest.

Japanese Patent Laying-Open No. 2005-169070 JP 2011-125568 A

  However, the cross-sectional image in a predetermined posture does not necessarily include the target region and the known image features that serve as a reference, and observation of the target region based on the image features of the target region and its surroundings. It is not always easy to conduct diagnosis. In Patent Documents 1 and 2, it is not considered to generate a cross-sectional image including a specified point indicating a target region specified in three-dimensional image data and including a feature point indicating a known image feature.

  The present invention provides an image processing technique that can generate a plurality of cross-sectional images that include designated points that indicate a region of interest in three-dimensional image data and that include feature points that are sequentially selected.

An image processing apparatus according to an aspect of the present invention includes an acquisition unit that acquires three-dimensional image data and information representing positions of a plurality of feature points in the three-dimensional image data;
A designation unit for designating a designated point indicating a site of interest in the three-dimensional image data;
A selection unit that selects, from the plurality of feature points, a feature point used for generating a cross-sectional image of the three-dimensional image data based on a predetermined selection order;
A generating unit that generates a cross-sectional image including a surface defined based on the position of the designated point and the position of the selected feature point from the three-dimensional image data;
A display control unit for displaying the cross-sectional image on a display unit ,
The selection unit determines an order of selecting feature points from the plurality of feature points based on a positional relationship between the position of the designated point and the positions of the plurality of feature points .

  According to the present invention, it is possible to generate a plurality of cross-sectional images that include designated points that indicate a site of interest in three-dimensional image data and that include feature points that are sequentially selected.

1 is a diagram illustrating a device configuration of an image processing apparatus according to a first embodiment. The figure which shows the example of the cut surface and cross-sectional image in 1st Embodiment. The flowchart which shows the whole process sequence in 1st Embodiment. The figure which shows another example of the cut surface in 1st Embodiment, and a cross-sectional image. The figure which shows the apparatus structure of the image processing apparatus which concerns on 2nd Embodiment. The flowchart which shows the whole process sequence in 2nd Embodiment. The flowchart which shows the procedure of the tiling process in 2nd Embodiment.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the components described in this embodiment are merely examples, and the technical scope of the present invention is determined by the scope of the claims, and is not limited by the following individual embodiments. Absent.

<First Embodiment>
The image processing apparatus according to the present embodiment includes an acquisition unit that acquires three-dimensional image data and positions of a plurality of feature points in the three-dimensional image data, and a designation unit that designates a designated point that indicates a site of interest in the three-dimensional image data. Is provided. In addition, the image processing apparatus includes a selection unit that selects one or more feature points from the plurality of feature points using information determined by the positions of the designated points and the positions of the plurality of feature points, and the designated points and the selection points. A generating unit that generates a cross-sectional image of the surface determined based on the feature points determined from the three-dimensional image data. Further, the image processing apparatus includes a display control unit that displays the generated cross-sectional image on the display unit.

  The feature point indicates a point having a characteristic image feature around it in the three-dimensional image data. The selection unit of the image processing apparatus can select one or more feature points from the plurality of feature points using information determined by the position of the designated point and the positions of the plurality of feature points. For example, the selection unit can select a feature point near the designated point. Alternatively, the selection unit can select two feature points from a plurality of feature points without using the position of the designated point. In this case, the selection unit may select two feature points at random, or may select a plurality of feature points that are close to the position of the reference feature point based on the positional relationship between the plurality of feature points. You may select from feature points. Alternatively, the selection unit can select feature points from a plurality of feature points in order of increasing distance from the reference feature point position.

  The image processing apparatus according to the present embodiment is given two different three-dimensional image data (first three-dimensional image data and second three-dimensional image data) obtained by imaging the same subject. In addition, it has a function as an image viewer that supports an operation of searching for a corresponding part on the other three-dimensional image with respect to a target part on one three-dimensional image. Further, when a pair of points that will correspond on two different three-dimensional image data are input as designated points, the image processing apparatus performs a process of generating a cross-sectional image from the three-dimensional image data. That is, the image processing apparatus captures a cross-sectional image cut out from the first three-dimensional image data by a plane (cut plane) including a designated point on the first three-dimensional image and two feature points selected by the selection unit. The first three-dimensional image data is generated and displayed. Further, a cross-sectional image cut out from the second three-dimensional image data by a plane (cut plane) including a designated point on the second three-dimensional image and corresponding points on the second three-dimensional image of the two feature points. Are generated from the second three-dimensional image data and displayed.

  The two cross-sectional images displayed by the image processing apparatus are cross-sectional images including designated points and corresponding points (anatomically identical feature points). Therefore, image features that serve as a reference for comparing the designated points. A cross-sectional image including In particular, when selecting a feature point in the vicinity of a designated point, it is possible to reduce the shift between the cross sections due to the influence of deformation. As a result, the user can check the two cross-sectional images and confirm whether the designated points on the two different three-dimensional images are anatomically identical points, that is, corresponding points. In addition, it is possible to update the designation of the position of the designated point while viewing the image. By updating the designation of the position of the designated point, the image processing apparatus generates and displays a cross-sectional image of the cut surface determined based on the updated designated point and feature point from the respective three-dimensional image data. Can do.

  In the present embodiment, the breast is the subject, but the gist of the present invention is not limited to this part, and other parts can be the subject. Hereinafter, the configuration and processing of this embodiment will be described with reference to FIGS.

  FIG. 1 is a diagram showing a configuration of a medical image diagnostic system (image processing system) according to the present embodiment. As shown in the figure, the image processing apparatus 100 in this embodiment is connected to a data server 170 and a display unit 180. The image processing apparatus 100 includes a first medical image imaging device 150 (first imaging unit) and a second medical image imaging device 160 (second imaging unit) that function as an imaging unit that captures three-dimensional image data, a data server, and the like. 170 is connected.

  The first medical image imaging apparatus 150 is an MRI apparatus, and images first-prone breasts to obtain first three-dimensional image data (MRI images). The second medical image capturing apparatus 160 is a CT apparatus and captures the breast in the supine position to obtain second 3D image data (CT image). These three-dimensional image data are stored in the data server 170. The modality for capturing 3D image data is not limited to the MRI apparatus and the CT apparatus, but may be a 3D ultrasonic imaging apparatus, a photoacoustic tomography apparatus, a PET / SPECT, an OCT apparatus, or the like. Further, the first and second three-dimensional image data may be captured with the same modality, or may be captured with the same body posture. For example, it may be an image obtained by capturing the same patient with the same modality at different dates and times for follow-up observation.

  Each of the first 3D image data and the second 3D image data held by the data server 170 (holding unit) is a tomographic image group obtained by imaging the subject in advance by the MRI apparatus and the CT apparatus. . The positions and orientations of the tomographic images constituting the first 3D image data and the second 3D image data are converted into a reference coordinate system (a coordinate system in a space with the subject as a reference). It is assumed that the data server 170 holds the data. The first 3D image data and the second 3D image data expressed in the reference coordinate system are input to the image processing apparatus 100 via the acquisition unit 102. Here, the position is represented by three position components (three-dimensional position information) of X, Y, and Z in the three-dimensional space of the reference coordinate system. The posture is a degree of rotation or inclination represented by three angle components (three-dimensional angle information) of a roll (Roll), a pitch (Pitch), and a yaw (Yaw) in a three-dimensional space.

  Further, the data server 170 holds the three-dimensional coordinate values of the corresponding feature point group (corresponding point group) in each three-dimensional image data, and these data are also stored in the image processing apparatus 100 via the acquisition unit 102. Is input. In the present embodiment, the data server 170 holds 3D position information of a plurality of feature points in the reference coordinate system for each 3D image data. In the following, the feature points on the first three-dimensional image are referred to as first feature points. A feature point on the second three-dimensional image is referred to as a second feature point. Each of the first feature points has a second feature point which is a pair (indicating the same anatomical point). Then, as the association information for associating the first feature point with the second feature point, a common ID number (identification information) is assigned to the corresponding feature point. That is, each feature point is represented by a point ID number (identification information) and three position components (three-dimensional position information) of X, Y, and Z in the three-dimensional space of the reference coordinate system. .

  The display unit 180 displays a display image generated by the image processing apparatus 100. The display unit 180 includes a button for determining whether or not to change the cutting plane, a button for determining whether or not to re-designate a designated point, and whether or not to end the designated point designation process. There are buttons and the like for determining whether or not.

(Configuration of the image processing apparatus 100)
The image processing apparatus 100 is composed of the components described below. The acquisition unit 102 acquires first three-dimensional image data and second three-dimensional image data as three-dimensional image data. In addition, the acquisition unit 102 uses, as the positions of the plurality of feature points, the positions of the first plurality of feature points in the first three-dimensional image data and the positions of the second plurality of feature points in the second three-dimensional image data. And get. In addition, the acquisition unit 102 acquires association information that associates the positions of the first plurality of feature points with the positions of the second plurality of feature points. Furthermore, the acquisition unit 102 acquires the three-dimensional coordinate value of the feature point group on the respective three-dimensional image data for the first and second three-dimensional image data. The association information is information for associating the three-dimensional coordinate values of the feature point groups in the first and second three-dimensional image data. The acquisition unit 102 outputs the acquired information to the selection unit 108 and the generation unit 112.

  The change operation unit 104 receives input of a mouse or keyboard operation by the user. Then, the change operation unit 104 changes either one of the first and second 3D image data or both the first and second 3D image data according to the received input. The change parameter for performing the acquisition is acquired and output to the calculation unit 110.

  The designation unit 106 designates the first designated point and the second designated point in the first and second three-dimensional image data in accordance with a user instruction, and acquires the position of each designated point in the reference coordinate system. . The designation unit 106 outputs the acquired position of the first designated point and the position of the second designated point to the selection unit 108 and the calculation unit 110.

  The selection unit 108 selects one or more feature points from the plurality of feature points using information determined by the position of the designated point and the positions of the plurality of feature points. Alternatively, the selection unit 108 selects two feature points from a plurality of feature points without using the position of the designated point. In the first three-dimensional image data, the selection unit 108 selects two features from the first feature point group based on a comparison between the position of the first designated point and the position of the first feature point. Select a point. For example, two feature points close to the position of the designated point are selected. Further, the selection unit 108 selects a corresponding point of the selected first feature point from the second feature point group using the association information in the second three-dimensional image data. Note that the feature point selected by the selection unit 108 may include a reference point indicating a specific part in the subject.

  In the first three-dimensional image data, when the first feature point is selected from the first plurality of feature points, the selection unit 108 selects the second feature point based on the association information. Then, the second feature point corresponding to the selected first feature point is selected. Alternatively, when the second feature point is selected from the second plurality of feature points in the second three-dimensional image data, the selection unit 108 selects the first plurality of feature points based on the association information. A first feature point corresponding to the selected second feature point is selected.

  In the first and second three-dimensional image data, the calculation unit 110 calculates a surface determined based on the designated point designated by the designation unit 106 and the two feature points selected by the selection unit 108 of the three-dimensional image data. Calculated as a cut surface. In addition, the calculation unit 110, based on the change parameter for changing the cut surface acquired by the change operation unit 104, the cut surface of either one of the first and second three-dimensional image data, or the first and second Both cut surfaces of the second three-dimensional image data are changed. Then, the calculation unit 110 outputs the calculated or changed cut surface to the generation unit 112.

  The generation unit 112 cuts out two-dimensional cross-sectional images from the first three-dimensional image data and the second three-dimensional image data using the cut surface calculated or changed by the calculation unit 110, and supplies the cut-out images to the display control unit 114. Is output. For example, the generation unit 112 generates a first cross-sectional image of the first cut surface determined based on the position of the first designated point and the position of the first feature point, and the position of the second designated point And a second cross-sectional image of the second cut surface determined based on the position of the second feature point.

  The display control unit 114 creates display data in which the generated two cross-sectional images are arranged, and performs display control to display the data on the display unit 180. That is, the display control unit 114 causes the display unit 180 to display the first cross-sectional image and the second cross-sectional image. When the user confirms the cross-sectional image displayed by the display control unit 114 and determines that the positions of the designated points designated in the respective three-dimensional image data are matched (corresponding), it is not shown. An instruction is input to the image processing apparatus via the UI. When this instruction is input, each position of the designated point is stored in a storage unit (not shown) in the image processing apparatus 100 as a corresponding point in the two three-dimensional image data, and the designated point designation process is completed. To do.

  The outline of the cross-sectional image generation process executed by the above-described image processing apparatus 100 will be described with reference to FIG. In FIG. 2A, the first three-dimensional image data (MRI image of the prone breast) obtained from the first medical imaging device 150 and the second three-dimensional image obtained from the second medical imaging device 160 are shown. Three-dimensional image data (CT image of the supine breast) is shown. These are the three-dimensional image data to be subjected to designated point designation processing. Further, a first cross section obtained by cutting out a display target region from each three-dimensional image data by a plane (first cut surface and second cut surface) passing through each three-dimensional image data and each cut surface. An image and a second cross-sectional image are shown (FIG. 2 (b)).

  Further, in each cross-sectional image, a target lesion portion on the first three-dimensional image data and a corresponding lesion portion corresponding to the target lesion portion on the second three-dimensional image data are displayed. In each three-dimensional image data, it is assumed that the position of the part considered to be a lesion is designated by the designation unit 106 as the first and second designated points. In FIG. 2A, designated points are indicated by crosses. The target lesion on the first three-dimensional image data is shaded and displayed inside the designated point, and the corresponding lesion on the second three-dimensional image data is not shaded. The display of both is changed by way of example, showing that even if the same subject is imaged by the MRI apparatus and the CT apparatus, the appearance is different. In addition, it is assumed that two or more characteristic positions, such as a blood vessel bifurcation portion and a mammary fat boundary portion, which can be associated with each 3D image data, are designated in advance as feature points. In FIG. 2A, the feature point group is indicated by a circle.

  The selection unit 108 selects the first based on the positional relationship obtained by comparing the position of the first designated point (target lesion part) and the position of the first feature point on the first three-dimensional image data. A feature point A closest to the specified point and a feature point B closest to the first specified point are selected. On the other hand, on the second 3D image data, the selection unit 108 associates the feature point A with the feature point B from the second feature points on the second 3D image data based on the association information. Feature points A ′ and B ′ are selected as points. At this time, the first cut surface can be defined as a plane passing through the first designated point, the feature point A, and the feature point B. The second cut surface can be defined as a plane that passes through the second designated point, the feature point A ′, and the feature point B ′. FIG. 2B shows a display example of the first and second cross-sectional images cut out from the first three-dimensional image data and the second three-dimensional image data by the first and second cut surfaces. The user can confirm whether or not the designated point is correctly designated by evaluating the appearance of the lesioned portion indicated by the designated point and the peripheral portion indicated by the feature point from various aspects.

  FIG. 3 is a flowchart showing an overall processing procedure performed by the image processing apparatus 100.

(Data acquisition)
In step S300, the acquisition unit 102 of the image processing apparatus 100 receives, from the data server 170, each three-dimensional image data (an MRI image of the prone breast and a CT image of the supine breast) and corresponding feature point groups. The three-dimensional coordinate value of (corresponding point group) is acquired.

(Designation of designated points)
In step S301, the designation unit 106 of the image processing apparatus 100 determines the position of the first designated point designated by the user on the first three-dimensional image and the second designated by the user on the second three-dimensional image. Get the position of the specified point.

  Specifically, first, the display control unit 114 displays each of the tomographic images constituting each three-dimensional image data on the screen. The tomographic image to be displayed can be arbitrarily selected from the group of tomographic images, and the change operation unit 104 can accept the operation of the mouse and the keyboard by the user and can be selectively displayed according to the input. it can. The user operates the mouse and keyboard to select and display a tomographic image that seems to include a lesion from each of the tomographic image groups of the MRI image and the CT image. Then, on the displayed tomographic image, an input for designating a position of a part that is considered to be a lesion as a designated point is performed. In response to the input, the designation unit 106 acquires the position of the first designated point and the position of the second designated point in the reference coordinate system using the position and orientation of the tomographic image.

  Here, the designation of the position of the designated point is performed by the following procedure, for example. First, the display control unit 114 switches and displays tomographic images on the display unit 180 in accordance with a user instruction. Then, when a site that seems to be a lesion of interest appears on the tomographic image, the user clicks the position of the site that seems to be a lesion on the display screen with a mouse or the like, so that the position of the designated point in the reference coordinate system is determined. specify. Note that the position of the designated point is not limited to being designated on the tomographic image constituting the three-dimensional image data, and may be designated on a cross-sectional image obtained by cutting the three-dimensional image data with a cutting plane having an arbitrary posture.

  The designation of the designated point does not need to be accurate because it designates the initial position. In order to finish the designated point designation process in a short time, it is also possible to designate a position near the center of the corresponding lesion. In this embodiment, since the objective is to observe the lesioned part of the subject, the center position of the lesioned part is used as the designated point, but a characteristic point other than the center position or other than the lesioned part is used as the designated point. It is also possible to specify.

  Note that since the first three-dimensional image data and the second three-dimensional image data are both expressed in the same reference coordinate system, the positions of the respective lesions in the reference coordinate system are not greatly different. Therefore, the position (initial position) of the second designated point may be configured to use the position of the first designated point in the reference coordinate system without being designated by the user.

  Furthermore, the designation unit 106 can select the first designated point and the second designated point from the feature point group acquired by the acquisition unit 102. In this case, since a common ID number (identification information) is assigned to the feature point, either the first three-dimensional image data or the second three-dimensional image data is selected as the designated point. For example, the designated point on the other side can be determined. Here, the feature point selected as the designated point is stored in a storage unit (not shown) as a point not selected by the selection unit 108. Note that the designation order of designated points may be reversed. That is, the second designated point (for example, the center position of the lesion) in the second 3D image data may be designated first.

(Select feature points)
In step S302, the selection unit 108 of the image processing apparatus 100 selects two feature points from the first feature point group on the first three-dimensional image data (first selection). In the present embodiment, two feature points are selected using information determined by the position of the first designated point and the respective positions of the first feature point group. For example, the selection is made based on the distance between the designated point and the feature point. As an example, a feature point located in the vicinity of a designated point can be selected. Specifically, the feature point closest to the first designated point (feature point A) and the second closest feature point (feature point B) are selected. Further, the selection unit 108 selects a corresponding point between the feature point A and the feature point B from the second feature point group on the second three-dimensional image data based on the feature point association information (FIG. 2 ( The feature point A ′ and feature point B ′) of a) are selected (second selection). Note that the selection unit 108 can perform the first selection on the second 3D image data and can perform the second selection on the first 3D image data. The three-dimensional image data to be subjected to the first selection may be selected by the user using a UI or the like, or the first selection may be performed on the three-dimensional image data in which the designated point has been previously specified.

  Here, the selection method of the two feature points in the first selection is not limited to the feature point closest to the designated point and the second closest feature point. The selection unit 108 may select a feature point such that the positional relationship between the feature point, the designated point, and the reference point satisfies a predetermined condition when a reference point indicating a specific part in the subject exists. Good. For example, based on a comparison between a straight line connecting a reference point and a designated point and the positions of a plurality of feature points, it is possible to sequentially select feature points that are close to the straight line from the plurality of feature points. . For example, with the nipple or the like as a reference point, the feature point closest to the straight line connecting the reference point and the designated point may be selected and the feature point closest to the second. At that time, the distance from the foot of the perpendicular line drawn straight from the feature point to the designated point may be considered together. In this case, it becomes easier to select feature points along the travel of the mammary gland from the nipple toward the lesion. The generation unit 112 generates a cross-sectional image of the surface determined based on the position of the designated point, the position of the reference point indicating a specific part in the subject, and the position of the feature point selected by the selection unit 108. Generate from dimensional image data.

  Alternatively, it is possible to add a condition that the distance from the designated point is limited to a feature point that is a predetermined distance or more (a feature point that is too close to the designated point is not selected). According to this, the problem that the cut surface becomes unstable when the selected feature point is too close to the designated point can be avoided. When selecting the second feature point, the angle between the straight line connecting the designated point and the first feature point and the straight line connecting the designated point and the second feature point is limited to the feature point within a predetermined range. It is possible to add a condition to do. According to this, the subject that a cut surface becomes unstable can be avoided.

  Further, the selection unit 108 can select a feature point from a plurality of feature points based on a comparison between information determined using the position of the designated point and the positions of the plurality of feature points and a threshold value. For example, the selection unit 108 can select a feature point using an area of a surface constituted by a designated point and two feature points. For example, the selection unit 108 selects a feature point from among a plurality of feature points such that the area of the surface constituted by the designated point and the two feature points is equal to or greater than a predetermined value (more than the area threshold). Is possible. For example, the feature points can be selected so that the area of the triangle formed by the designated points and the two feature points is greater than or equal to a predetermined value.

  The selection unit 108 can select a feature point using an angle formed by a straight line connecting the designated point and the first feature point and a straight line connecting the designated point and the second feature point. For example, a feature in which an angle formed by a straight line connecting the specified point and the first feature point and a straight line connecting the specified point and the second feature point is within a predetermined angle (threshold: threshold of the linear angle). A point can be selected from a plurality of feature points. For example, the angle formed by the straight line connecting the designated point and the first feature point and the straight line connecting the designated point and the second feature point is close to 90 degrees (the absolute value of the difference between the angle and 90 degrees is predetermined). It is also possible to select two feature points that are smaller than the angle (1). In this case, when the position of the designated point is slightly changed (redesignated), it is possible to prevent the angle of the cut surface from greatly changing from before the redesignation and becoming difficult to understand.

  Alternatively, the selection unit 108 is the second closest to the feature point where the distance from the predetermined curved surface such as the body surface or the pectoral muscle surface to the designated point and the distance from the curved surface to the feature point are closest. It is possible to select such feature points. In this case, feature points at substantially the same depth as the designated point are easily selected.

  The selection unit 108 can select a feature point using an angle formed by a reference plane and a plane defined by the designated point and the two feature points. For example, the selection unit 108 selects a feature point such that an angle between a surface defined by the designated point and two feature points and a predetermined reference plane is smaller than a predetermined angle (threshold: threshold of plane angle). It is possible. Alternatively, the selection unit 108 can select a feature point that minimizes the angle between the surface defined by the designated point and the two feature points and the reference plane. Here, an axial section, a sagittal section, a coronal section, or the like can be used as the reference plane. In this case, it is possible to observe a cross-sectional image of a cross section close to a familiar cross-section that is usually used for medical treatment.

  Further, the selection unit 108 can select a feature point having the highest texture complexity in a plane cross-sectional image including a designated point and two feature points. In this case, it is expected that an image feature that serves as a reference when diagnosing a lesion is included in the cross-sectional image.

  Also, one of the two feature points is independent of the distance from the designated point, as shown in FIG. 4A, a fixed reference point designated in advance (for example, an anatomy such as a nipple position). Can be used. At that time, a reference point closest to the designated point may be selected from a plurality of reference points and used. In this case, the lesioned part can be diagnosed while confirming the relationship with the reference point. At this time, as another feature point, it is desirable to select a feature point whose positional relationship between the reference point, the designated point, and the feature point satisfies a predetermined condition. For example, it is possible to select a feature point whose distance from a line segment connecting the reference point and the designated point is within a predetermined range. Further, the two feature points may be configured such that the user selects from a plurality of candidates within a predetermined distance from the designated point using a UI or the like. The selection unit 108 can also select two feature points from among a plurality of feature points without using the position of the designated point as a feature point selection method in the first selection. In this case, the selection unit 108 may randomly select two feature points, or select a plurality of feature points close to the position of the reference feature point based on the positional relationship between the plurality of feature points. It is possible to select from among the feature points. Alternatively, the selection unit 108 can select feature points from a plurality of feature points in order of increasing distance from the reference feature point position.

(Calculation of cut surface)
In step S303, the calculation unit 110 of the image processing apparatus 100 determines or changes the cut surface. When the process proceeds from S302 or S309 to this step, the calculation unit 110 converts the designated point designated by the designation unit 106 and the two feature points selected by the selection unit 108 in the first and second three-dimensional image data. A cutting plane determined based on the cutting plane is determined. Since the plane passing through the three points (designated point and two feature points) can be obtained by a known method, detailed description thereof is omitted.

  On the other hand, when transitioning from step S307 to this processing step, the calculation unit 110 applies the change parameter acquired in step S307 to the cut surface calculated by the processing of step S303 executed earlier, Determine a new cutting plane. In the present embodiment, the calculation unit 110 performs a rotation transformation about each straight line connecting two feature points on each cut surface.

(Cross section image generation)
In step S304, the generation unit 112 of the image processing apparatus 100 generates cross-sectional images of the first three-dimensional image data and the second three-dimensional image data by using the respective cut planes obtained in step S303. Specifically, in the cut surface, a range in which the image is cut out and a rotation axis of in-plane rotation are determined, and a cross-sectional image in the range is generated. Here, the generation unit 112 can generate a cross-sectional image within a range where the three-dimensional image data and the cut surface intersect without limiting the range to be cut out. In FIG. 2 (a), this corresponds to the area within the dotted line. Further, the generation unit 112 can set a range to be cut out based on the position of the designated point. For example, it is possible to obtain a predetermined range in the cutting plane centering on the position of the specified point and generate a cross-sectional image within the range. Similarly, it is possible to set a range to be cut out based on the position of the selected feature point or reference point. It should be noted that the range to be cut out can be enlarged by, for example, the user clicking an enlarge button arranged on the display unit 180 with a mouse so as to include a wider range.

  On the other hand, the rotation axis of in-plane rotation may be determined, for example, so that the X-axis direction of the cross-sectional image coordinate system is orthogonal to the Y-axis direction (gravity direction) of the reference coordinate system. Further, the rotation axis of the in-plane rotation may be determined so that the straight line connecting the designated point and the reference point matches the reference direction (for example, the Y-axis direction) of the coordinate system of the cross-sectional image. FIG. 4B shows a display example when the rotation axis of the in-plane rotation is determined so that the straight line connecting the designated point and the reference point (nipple) coincides with the Y-axis direction of the cross-sectional image coordinate system.

  Note that one cross-sectional image may be subjected to affine transformation so that the distance and direction from the designated point to the two feature points coincide in the first cross-sectional image and the second cross-sectional image. Since the affine transformation matrix can be obtained by a known method from a set of three points before and after the transformation (designated point and two feature points), detailed description is omitted.

(Display of cross-sectional image)
In step S305, the display control unit 114 of the image processing apparatus 100 causes the display unit 180 to display two cross-sectional images. A user who is a diagnostician can check whether or not the position of each designated point matches the position of the lesion (the lesion of interest and the corresponding lesion) by looking at the two cross-sectional images.

  As an example of display, the display control unit 114 can display, for example, a first cross-sectional image and a second cross-sectional image by dividing one screen vertically or horizontally, respectively, on two screens. It is also possible to display a cross-sectional image. Alternatively, the display control unit 114 can draw the first cross-sectional image and the second cross-sectional image in different colors and display them in a superimposed manner, and any one of the first cross-sectional image and the second cross-sectional image can be displayed. It is also possible to select and display only one of them. FIG. 2B shows an example in which one screen is divided in the horizontal direction and the first cross-sectional image and the second cross-sectional image are displayed side by side.

  Furthermore, the display control unit 114 can also display a cross shape, a square, a circle, or the like indicating the position of the designated point or feature point on the first cross-sectional image and the second cross-sectional image. is there. In addition, the display control unit 114 can perform rendering by volume-rendering one volume data and overlaying a figure such as a plane representing the position and orientation of the other cross-sectional image. The display control unit 114 can also perform display control so that whether or not to superimpose various graphics can be selected. FIG. 2B shows an example in which a mark “X” indicating the position of the designated point and a circle indicating the position of the selected feature point are superimposed and displayed.

(Change of cutting plane)
In step S306, the image processing apparatus 100 changes either one of the cut surfaces of the first and second 3D image data or both of the cut surfaces of the first and second 3D image data. It is determined whether or not to continue the confirmation. For example, the user inputs a change determination by clicking a cutting plane change button (cutting plane changing section) arranged on the display unit 180 with a mouse. If a change determination is input, the process proceeds to step S307. On the other hand, if the change determination is not input, the process proceeds to step S308.

(Obtain cut surface change parameters)
In step S307, the change operation unit 104 changes either one of the cut surfaces of the first and second 3D image data, or both of the cut surfaces of the first and second 3D image data. The change parameter is acquired, and the process returns to step S303. As a change parameter, a rotation angle L about an axis about a straight line connecting two feature points is acquired. This parameter can be acquired by operating a mouse constituting the change operation unit 104. For example, the change operation unit 104 obtains the rotation angle L based on the change operation of the change operation unit 104 (for example, the position, movement amount, and movement direction of the mouse while the mouse button is pressed). For example, by a series of operations of the change operation unit 104 such as start of change operation (starting to press the mouse button), move operation (moving while pressing the mouse button), and end of change operation (release the press of the mouse button), The direction vector specified by the change operation can be determined. The change operation start position becomes the start point of the direction vector, and the change operation end position becomes the end point of the direction vector, so that the direction vector can be determined. Further, the change operation unit 104 can acquire an angle formed by the direction vector and a straight line (rotation axis) connecting two feature points as a rotation angle L around the axis.

  Here, the change operation unit 104 can select whether to change one of the cut surfaces of the first three-dimensional image data and the second three-dimensional image data. For example, when the left mouse button is pressed and operated, the cut surface of the first three-dimensional image can be changed. Similarly, when the right mouse button is pressed and operated, the cut surface of the second three-dimensional image can be changed. When both buttons of the mouse are pressed and operated, both the cut surfaces of the first 3D image and the second 3D image can be changed.

(Determining whether to re-specify the specified point)
In step S308, the image processing apparatus 100 determines whether or not to re-designate the designated point of the first or second 3D image data to a different designated point. If the user who has viewed the cross-sectional image displayed in step S305 determines that the specified point does not correspond to the user, an input to re-specify the specified point is performed. For example, a re-designation determination is input by the user clicking a re-designation button (re-designation unit) arranged on the display unit 180 with a mouse. If re-designation determination is input, the process proceeds to step S309. On the other hand, if the re-designation determination is not input, the process proceeds to step S310.

(Redesignation of designated points)
In step S309, the designation unit 106 of the image processing apparatus 100 designates a designated point in the three-dimensional image data that is different from the designated point designated in step S301 when the designated point is designated again. The designation unit 106 re-designates the position of the designated point of the first or second 3D image data to a position of a different designated point, returns the process to step S303, and each unit of the image processing apparatus 100 performs step S303. The same processing is performed thereafter. The user observes the first and second cross-sectional images, and evaluates the similarity between the lesion part indicated by the designated point and the peripheral part indicated by the feature point in a multifaceted manner, or the target lesion part or the corresponding lesion. The position of the part can be specified more accurately.

(Determine the end of the specified process)
In step S <b> 310, the image processing apparatus 100 finishes the designation processing of one designated point of the first and second 3D image data or both designated points of the first and second 3D image data. It is determined whether or not to do so. For example, an end determination is input by, for example, the user clicking the end button arranged on the display unit 180 with the mouse. If it is determined that the designation process is to be terminated, the process proceeds to step S311. On the other hand, if it is determined not to end, the process returns to step S305, and each unit of the image processing apparatus 100 performs the same process after step S305. Here, when there is no input for changing the cutting plane or re-designating the designated point, the apparatus enters a standby state with the cross-sectional image displayed until an end input is received, and accepts the input.

(Output of specified point)
In step S311, the image processing apparatus 100 stores the position of the designated point in a storage unit (not shown) as a corresponding point in the two three-dimensional image data. Further, the display control unit 114 outputs information on the corresponding designated points to the display unit 180, and causes the display unit 180 to display the information to that effect. Thus, the process executed by the image processing apparatus 100 ends.

  The image processing apparatus according to the present embodiment is based on designated points determined for respective lesions in two three-dimensional image data and two feature points selected based on the positional relationship with the designated points. It is possible to generate and display a cross-sectional image of the defined cut surface. As a result, a cross-sectional image including an image feature that serves as a reference for comparing the designated points is displayed. Therefore, the position of the corresponding point in a plurality of three-dimensional image data can be correctly specified, and the diagnosis of the lesion can be supported.

  In this embodiment, a case where a specified set of designated points is specified has been described. However, when a plurality of specified points are specified, for example, when there are a plurality of lesions, What is necessary is just to repeat the process of step S301 to step S311.

  In this embodiment, the case where two feature points are selected from a plurality of feature points has been described as an example. However, the present invention is not limited to this, and three or more feature points may be selected. In this case, since the surface including the selected feature point and the designated point is a curved surface, a cross-sectional image (curved cross-sectional image) with the curved surface as a cut surface may be displayed.

Second Embodiment
In the present embodiment, a configuration will be described in which the selection order is determined according to a criterion (rule) for selecting one or more feature points, and the selection is performed sequentially based on the determined selection order. Hereinafter, the difference between the image processing apparatus according to the present embodiment and the first embodiment will be described. FIG. 5 is a diagram showing a configuration of a medical image diagnostic system (image processing system) according to the present embodiment. In the figure, the same parts as those in FIG. 1 are given the same numbers and symbols, and the explanation of the overlapping parts is omitted.

  The image processing apparatus according to the present embodiment includes an acquisition unit that acquires three-dimensional image data and positions of a plurality of feature points in the three-dimensional image data, and a designation unit that designates a designated point that indicates an attention site in the three-dimensional image data Is provided. The image processing apparatus further includes a selection unit that selects one or more feature points used for generating a cross-sectional image of the three-dimensional image data from a plurality of feature points based on a predetermined selection order. In addition, the image processing apparatus generates a cross-sectional image of a surface determined based on the designated point and the selected feature point from the three-dimensional image data, and display control for displaying the generated cross-sectional image on the display unit A part.

  The selection unit 508 in this embodiment determines the order of selection according to a criterion (rule) for selecting feature points. Then, the selection unit 508 sequentially selects and calculates two feature points from the feature point group in the first three-dimensional image data or the second three-dimensional image data based on a predetermined selection order. Output to the unit 110.

  The display mode selection unit 516 includes a pair display mode (first display mode) for arranging a set of cross-sectional images and a tiled display mode (second display mode) for arranging a plurality of sets of cross-sectional images in a tile shape. Either one of the display modes is selected and output to the display control unit 514. The display control unit 514 controls the display of the cross-sectional image according to the selected display mode.

  In accordance with an operation input for selecting a set of cross-sectional images, the cross-sectional image selection unit 518 selects a set of cross-sectional images from a plurality of sets of cross-sectional images displayed in a tile shape. When the cross-sectional image selection unit 518 selects a set of cross-sectional images, the display mode selection unit 516 switches the display mode from the tiled display mode to the pair display mode, and the display control unit 514 controls the display of the cross-sectional image according to the switching. .

  The display control unit 514 creates display data in which a set of cross-sectional images are arranged, or display data in which a set of a plurality of cross-sectional images are arranged in a tile shape according to the display mode. Is displayed on the display unit 180.

  FIG. 6 is a flowchart showing an overall processing procedure performed by the image processing apparatus 500. Since the processing from step S600, S608 and step S611 to step S616 overlaps with the processing from step S300, S305 and step S306 to step S311 in FIG. 3 in the first embodiment, a description thereof will be omitted.

(Select feature points)
In step S602, the selection unit 508 of the image processing apparatus 500 determines the selection order according to a criterion (rule) for selecting feature points. Then, the selection unit 508 selects two feature points from the feature point group of the first 3D image data or the feature point group of the second 3D image data based on the determined order of selection. select. Here, which of the first three-dimensional image data and the second three-dimensional image data is to be used for feature point selection can be determined in the same manner as in the first embodiment. .

  The selection unit 508 determines the order of selecting two feature points from the plurality of feature points based on the positional relationship obtained by comparing the position of the designated point and the positions of the plurality of feature points. For example, the selection unit 508 can sequentially select feature points from a plurality of feature points in the order closest to the position of the designated point based on the determined order of selection. Alternatively, the selection unit 508 can sequentially select feature points from a plurality of feature points in order of increasing distance from the position of the designated point based on the determined order of selection.

  In this step, the feature points are rearranged and arranged in order using the positional relationship between the position of the designated point and the positions of the plurality of feature points as a reference for determining the order of selection. For example, the feature points are rearranged and arranged in order of increasing distance from the designated point or in increasing order. Next, the feature point selection number N is initialized to 1, and the Nth and N + 1th, that is, the first and second feature points are selected.

  Here, the selection unit 508 is not limited to the case where the positional relationship between the position of the designated point and the positions of the plurality of feature points is used as a reference for determining the order of selection. It is possible to select feature points by comparing positional relationships using predetermined feature points. For example, the feature points may be rearranged in order of increasing or decreasing distance from a reference point such as a nipple or a predetermined feature point. Alternatively, the reference points indicating specific parts in the subject may be rearranged in order of increasing or decreasing distance from a predetermined curved surface such as the body surface or the pectoralis major surface.

  When a reference point indicating a specific part in the subject is used, the selection unit 508 selects two features based on the positional relationship between the reference point indicating the specific part in the subject and the positions of a plurality of feature points. The order of selecting points from a plurality of feature points is determined. The selection unit 508 can sequentially select feature points from a plurality of feature points in the order closer to the position of the reference point based on the determined order. Alternatively, the selection unit 508 can sequentially select two feature points from a plurality of feature points in order of increasing distance from the position of the reference point based on the determined order.

  When a predetermined feature point is used, the selection unit 508 determines an order of selecting one or more feature points from the plurality of feature points based on the positional relationship between the plurality of feature points. The selection unit 508 can sequentially select two feature points from a plurality of feature points in the order closest to the position of the reference feature point based on the determined order. Alternatively, the selection unit 508 can sequentially select two feature points from a plurality of feature points in order of distance from the position of the reference feature point based on the determined order.

  Further, a random order may be used as a criterion for determining the order of selection. In that case, the feature points may be rearranged at random. Note that the order of the ID numbers (identification information) of the feature points (ascending order or descending order) may be used as a reference for determining the order of selection, in which case rearrangement is not necessary.

  Here, when the selection unit 508 selects two first feature points from the first plurality of feature points, the selection unit 508 selects the second feature points corresponding to the first feature points based on the association information. Two second feature points are selected from the second plurality of feature points. On the other hand, when the selection unit 508 selects two second feature points from the second plurality of feature points, the selection unit 508 selects the two feature points corresponding to the second feature points based on the association information. The first feature point is selected from the first plurality of feature points.

(Calculation of cut surface)
In step S603, the calculation unit 110 of the image processing apparatus 500 determines or changes the cut surface. When transitioning to this step from S602, S610, or S614, the calculation unit 110 uses the designated point designated by the designation unit 106 and the two feature points selected by the selection unit 508 in the first and second 3D image data. The cutting plane determined based on is determined.

  On the other hand, when the process proceeds from step S612 to this process step, the calculation unit 110 applies the change parameter acquired in step S612 to the cut surface calculated by the process of step S603 executed previously, Determine a new cutting plane.

(Cross section image generation)
In step S604, the generation unit 112 of the image processing apparatus 500 sequentially generates a cross-sectional image of a surface determined based on the designated point and the sequentially selected feature points from the three-dimensional image data. The display control unit 514 causes the display unit 180 to display a plurality of sequentially generated cross-sectional images.

(Determination of display mode)
In step S605, the display mode selection unit 516 of the image processing apparatus 500 determines whether or not to change the display mode, and then determines whether or not the display mode is a tiled display mode. First, when the user wants to change the display mode, an input for changing the display mode is made. For example, the user inputs a determination to change the display mode by clicking a display mode change button (display mode change unit) arranged on the display unit 180 with a mouse. When the display mode change determination is input, the display mode selection unit 516 changes the display mode to the pair display mode if the display mode is the tile display mode, and conversely displays the tile display if the display mode is the pair display mode. Change to mode. On the other hand, when the determination to change the display mode is not input, the display mode selection unit 516 does not change the display mode.

  If the display mode is a tiled display mode, the process proceeds to step S606. On the other hand, if the display mode is not the tile display mode (the pair display mode), the process proceeds to step S608.

(Tile display)
In step S <b> 606, the display control unit 514 of the image processing apparatus 500 creates (tils) display data by arranging tile image pairs (pairs) of the number of feature point combinations in a tile shape, and displays them. Displayed on the unit 180. When the tiled display mode is selected by the display mode selection unit 516, the display control unit 514 causes the display unit 180 to display sequentially generated cross-sectional images. A user who is a diagnostician can check whether or not the position of each designated point matches the position of the lesion (the lesion of interest and the corresponding lesion) by looking at the cross-sectional images arranged in a tile shape. . Details of the tiling process will be described later with reference to a flowchart shown in FIG.

(Cross section image selection)
In step S607, the cross-sectional image selection unit 518 of the image processing apparatus 500 determines whether or not to select a set of cross-sectional images of interest from a plurality of sets of cross-sectional images displayed in a tile shape. When the cross-sectional image selection unit 518 selects a set of cross-sectional images, the display mode selection unit 516 switches the display mode from the tiled display mode to the pair display mode, and the display control unit 514 selects the cross-sectional image selected according to the switching. Control the display of. In other words, when a user who has viewed a plurality of sets of cross-sectional images displayed in a tile shape wants to observe a detailed set of cross-sectional images in detail, an input indicating that a set of cross-sectional images is to be selected. I do. For example, the selection determination is input by, for example, clicking a cross-sectional image of interest among the cross-sectional images displayed in a tile shape on the display unit 180 with a mouse. When the selection determination is input, the display mode is switched to the pair display mode, and the selected set of cross-sectional images is output to the display control unit 514, and the process proceeds to step S608. On the other hand, if the selection determination is not input, the process proceeds to step S616.

(Determining whether to reselect feature points)
In step S609, the image processing apparatus 500 determines whether to reselect feature points. That is, when the user who has viewed the cross-sectional image displayed in step S608 determines to view a cross-sectional image including another feature point, the user inputs that the feature point is selected again. For example, the user selects a reselection determination by clicking a reselect button (reselection unit) arranged on the display unit 180 with a mouse. If the reselection determination is input, the process proceeds to step S610. On the other hand, if the determination of feature point reselection is not input, the process proceeds to step S611.

(Reselect feature points)
In step S610, the selection unit 508 of the image processing apparatus 500 adds 1 to N when the feature point selection number N is 2 or more smaller than the feature point number M, and initializes N to 1 otherwise. To do. Then, the Nth and N + 1th feature points are selected. Thus, the processing of the image processing apparatus 500 is performed.

(Tiling procedure)
FIG. 7 is a flowchart illustrating a tiling process procedure performed by the image processing apparatus 500. Hereinafter, processing of each unit of the image processing apparatus 500 will be described.

(Initialization of selection number)
In step S700, the selection unit 508 of the image processing apparatus 500 initializes the feature point selection number N to 1.

(Select feature points)
In step S <b> 701, the selection unit 508 of the image processing apparatus 500 includes a group of feature points that are rearranged based on a criterion (rule) for selecting feature points (for example, in order of increasing distance from the designated point). The Nth and N + 1th feature points are selected.

(Calculation of cut surface)
In step S <b> 702, the calculation unit 110 of the image processing apparatus 500 determines a cut plane determined based on the designated point designated by the designation unit 106 and the two feature points selected by the selection unit 508.

(Cross section image generation)
In step S <b> 703, the generation unit 112 of the image processing apparatus 500 generates cross-sectional images of the first and second three-dimensional image data using the cut plane determined by the calculation unit 110 and outputs the cross-sectional images to the display control unit 114. .

(Judgment of end of feature point selection)
In step S704, the selection unit 508 of the image processing apparatus 500 determines whether or not the feature point selection is to be ended. If the feature point selection number N is two or more smaller than the feature point number M, the process returns to step S701 after adding 1 to N. In other cases, the selection unit 508 determines that the feature point selection is to end, and proceeds to step S705.

(Tile display)
In step S <b> 705, the display control unit 514 of the image processing apparatus 500 performs control for creating display data in which (M−1) sets of cross-sectional images are arranged in a tile shape and displaying the data on the display unit 180. As an example of display, the display control unit 514 divides a display area of one screen into tiled (M−1) partial display areas, and first and second cross-sectional images are displayed in each partial display area. Each pair can be displayed. Alternatively, the display control unit 514 can display an image in which the first and second cross-sectional images are drawn in different colors and overlapped with each other in a tile shape, and any one of the first and second cross-sectional images can be displayed. It is also possible to select only one and display it in the tiled partial display area. Furthermore, the display control unit 514 can display various figures superimposed on each cross-sectional image. Thus, the tiling process executed by the image processing apparatus 500 is completed.

  The image processing apparatus according to the present embodiment generates a cross-sectional image of a cut surface determined based on designated points determined for respective lesions in two three-dimensional image data and two feature points that are sequentially selected. Can be displayed. Thereby, diagnosis of a lesioned part can be supported.

<Third Embodiment>
In the first embodiment and the second embodiment, the configuration in which two feature points are selected has been described as an example. However, the number of feature points to be selected is not limited to two. This embodiment demonstrates the structure which selects one feature point based on the positional relationship with a designated point. Hereinafter, the difference between the image processing apparatus according to the present embodiment and the first embodiment will be described. The configuration of the medical image diagnostic system (image processing system) according to the present embodiment is the same as that in FIG. 1, but the processing contents of the selection unit 108 and the calculation unit 110 are different.

  The selection unit 108 selects one feature point based on the positional relationship with the designated point designated by the designation unit 106, and outputs it to the calculation unit 110. For example, the selection unit 108 uses the first feature point group based on the comparison (positional relationship) between the position of the first designated point and each position of the first feature point in the first three-dimensional image data. Select one feature point from. For example, the feature point closest to the first designated point is selected. Further, the selection unit 108 selects a corresponding point of the selected first feature point from the second feature point group using the association information in the second three-dimensional image data. And it outputs to the calculation part 110.

  The calculation unit 110 calculates a cut plane determined based on the designated point designated by the designation unit 106 and one feature point selected by the selection unit 108 in the first and second three-dimensional image data. In addition, the calculation unit 110, based on the change parameter for changing the cut surface acquired by the change operation unit 104, the cut surface of either one of the first and second three-dimensional image data, or the first and second Both cut surfaces of the second three-dimensional image data are changed. Then, the calculation unit 110 outputs the calculated or changed cut surface to the generation unit 112.

  The overall processing procedure performed by the image processing apparatus 100 is the same as the flowchart shown in FIG. However, the processing contents of step S302, step S303, and step S307 are different.

(Select feature points)
In step S302, the selection unit 108 selects one feature point from the first feature point group on the first three-dimensional image data (first selection). In the present embodiment, one feature point (for example, feature point A) closest to the first designated point is selected. Further, the selection unit 108 selects a corresponding point of the feature point A (for example, FIG. 2A) from the second feature point group on the second three-dimensional image data based on the feature point association information. Feature point A ′) is selected (second selection). Note that the feature point selection method in the first selection is not limited to the method of selecting the feature point closest to the designated point, and various methods described in step S302 of the first embodiment may be used. it can.

(Calculation of cut surface)
In step S303, the calculation unit 110 determines or changes the cut surface. When the process proceeds from S302 or S309 to this step, the calculation unit 110 converts the designated point designated by the designation unit 106 and one feature point selected by the selection unit 108 in the first and second three-dimensional image data. A cutting plane determined based on the determined cutting plane is determined. A plane passing through two points (a specified point and one feature point) may be calculated as follows, for example. The calculation unit 110 presets a reference direction (for example, a Y-axis direction representing a gravity direction) of a reference coordinate system (a coordinate system in a space with the subject as a reference), and selects one feature selected as a designated point. A plane including a point and having a normal line orthogonal to the reference direction may be calculated as a cut plane.

  On the other hand, when the process transitions from step S307 to this processing step, the calculation unit 110 performs rotation conversion around the straight line connecting the designated point designated by the designation unit 106 and one feature point selected by the selection unit 108. Apply to each cut surface.

(Obtain cut surface change parameters)
In step S307, the change operation unit 104 acquires a change parameter for changing the cut surface. The calculation unit 110 can change the cut surface of the three-dimensional image data by changing the rotation angle based on the change parameter. The change operation unit 104 acquires the direction vector D of the rotation axis and the rotation angle L around the axis as change parameters. For example, the change operation unit 104 rotates the direction vector D of the rotation axis and the rotation based on the change operation of the change operation unit 104 (for example, the position, movement amount, and movement direction of the mouse while the mouse button is pressed). The angle L is acquired. For example, by a series of operations of the change operation unit 104 such as start of change operation (starting to press the mouse button), move operation (moving while pressing the mouse button), and end of change operation (release the press of the mouse button), A direction vector D can be defined. The change operation start position becomes the start point of the direction vector D, the change operation end position becomes the end point of the direction vector D, and the direction vector D can be determined. Further, the change operation unit 104 rotates the angle formed between the direction vector D and a straight line (rotation axis) connecting the specified point specified by the specifying unit 106 and one feature point selected by the selecting unit 108 around the axis. It can be acquired as the angle L.

  Note that in the cross-sectional image generation processing in step S304, the generation unit 112 causes the one cross-section so that the distance and direction from the designated point to one feature point match in the first cross-sectional image and the second cross-sectional image. It is possible to enlarge, reduce, and rotate the image.

  According to this embodiment, the reference direction is aligned, including the designated point determined for each lesioned part of the two three-dimensional image data and one feature point selected based on the positional relationship between the designated point. A cross-sectional image by the cut surface can be generated and displayed.

<Modification>
In the first to third embodiments, a case has been described in which two different three-dimensional image data obtained by imaging the same subject are to be processed, but there are three or more three-dimensional image data. Also good. In addition, those three-dimensional image data may be captured with different modalities, or part or all of them may be captured with the same modality. According to this modification, it is possible to compare lesions of three or more three-dimensional image data, so that more reliable diagnosis of the lesion can be supported.

<Fourth embodiment>
In the first to third embodiments, an example has been described in which two different three-dimensional image data obtained by imaging the same subject are processed, but the number of three-dimensional image data is two. Not limited to. In the present embodiment, a case where there is one piece of three-dimensional image data, that is, a case where only the first three-dimensional image data is acquired and used will be described. Hereinafter, the difference between the image processing apparatus according to the present embodiment and the first embodiment will be described.

  The configuration of the medical image diagnostic system (image processing system) according to the present embodiment is the same as that shown in FIG. However, one of the medical imaging devices is not used. In the present embodiment, a configuration example in which the first medical image imaging device 150 (first imaging unit) is used and the second medical image imaging device 160 (second imaging unit) is not used will be described. The medical image capturing apparatus to be used is not limited to this example, and the second medical image capturing apparatus 160 may be used and the first medical image capturing apparatus 150 may not be used.

  In addition, the processing content of each unit constituting the image processing apparatus is different. Hereinafter, the processing content of each part is demonstrated concretely.

  The data server 170 (holding unit) holds, as first three-dimensional image data, a tomographic image group obtained by imaging a subject in advance with an MRI apparatus. Also, the three-dimensional coordinate value of the feature point group in the three-dimensional image data is held. However, the feature point ID number (identification information) is not required. That is, each feature point is represented by three position components (three-dimensional position information) of X, Y, and Z in the three-dimensional space of the reference coordinate system.

  The acquisition unit 102 acquires the first three-dimensional image data input to the image processing apparatus 100 and the three-dimensional coordinate values of the feature point group, and outputs the acquired information to the selection unit 108 and the generation unit 112.

  The change operation unit 104 receives an input of a mouse or keyboard operation by the user, acquires a change parameter for changing the cut surface of the first three-dimensional image data according to the received input, and calculates the acquired change parameter To 110.

  The designation unit 106 designates, for example, the center position of a part that is considered to be a lesion part (target lesion part) in the first three-dimensional image data as a first designated point in accordance with a user instruction, in the reference coordinate system. It can be acquired as the position of the first designated point. The designation unit 106 outputs the acquired position of the first designated point to the selection unit 108 and the calculation unit 110.

  In the first three-dimensional image data, the selection unit 108 selects from the first feature point group based on the positional relationship obtained by comparing the position of the first designated point and the position of the first feature point. Two feature points are selected and output to the calculation unit 110.

  In the first three-dimensional image data, the calculation unit 110 calculates a cutting plane determined based on the designated point designated by the designation unit 106 and the two feature points selected by the selection unit 108. Further, the calculation unit 110 changes the cut surface of the first three-dimensional image data based on the change parameter for changing the cut surface acquired by the change operation unit 104. Then, the calculation unit 110 outputs the calculated or changed cut surface to the generation unit 112.

  The generation unit 112 cuts out a two-dimensional cross-sectional image from the first three-dimensional image data using the cut surface calculated or changed by the calculation unit 110, and outputs the cut-out image to the display control unit 114.

  The display control unit 114 performs display control for causing the display unit 180 to display the generated cross-sectional image. When the user confirms the cross-sectional image displayed by the display control unit 114 and determines that the position of the designated point designated in the three-dimensional image data is correct, the instruction is processed through the UI (not shown). Input to the device. When this instruction is input, the position of the designated point is stored in a storage unit (not shown) in the image processing apparatus 100, and the designated point designation process is completed.

  The overall processing procedure performed by the image processing apparatus 100 is the same as the flowchart shown in FIG. However, the processing content of each step is different. Specific processing contents will be described below.

(Data acquisition)
In step S300, the acquisition unit 102 acquires first three-dimensional image data (an MRI image of a prone breast) and a three-dimensional coordinate value of a feature point group.

(Designation of designated points)
In step S301, the position of the first designated point in the first three-dimensional image data is determined as processing of the designation unit 106.

(Select feature points)
In step S302, the selection unit 108 selects the feature point closest to the specified point (feature point A) and the second closest feature point (feature point B) from the feature point group of the first three-dimensional image data. To do. Note that the feature point selection method is not limited to this, and various methods can be used as in the first selection described in step S302 of the first embodiment.

(Calculation of cut surface)
In step S303, the calculation unit 110 determines or changes the cut surface. In the case of transition from S302 or S309 to this processing step, the calculation unit 110 converts the designated point designated by the designation unit 106 and the two feature points selected by the selection unit 108 in the first three-dimensional image data. A cutting plane determined based on the cutting plane is determined.

  On the other hand, when transitioning from step S307 to this processing step, the calculation unit 110 applies the change parameter acquired in step S307 to the cut surface calculated by the processing of step S303 executed earlier, Determine a new cutting plane.

(Cross section image generation)
In step S <b> 304, the generation unit 112 generates a cross-sectional image of the first three-dimensional image data using the cut surface determined or changed by the calculation unit 110.

(Display of cross-sectional image)
In step S305, the display control unit 114 causes the display unit 180 to display a cross-sectional image. A user who is a diagnostician can check the cross-sectional image to confirm whether the position of the designated point matches the position of the lesion. The display control unit 114 can also display various figures superimposed on the cross-sectional image.

(Change of cutting plane)
In step S306, the image processing apparatus 100 determines whether to change the cut surface of the first three-dimensional image data and continue the confirmation. For example, the user inputs a change determination by clicking a cutting plane change button (cutting plane changing section) arranged on the display unit 180 with a mouse. If a change determination is input, the process proceeds to step S307. On the other hand, if the change determination is not input, the process proceeds to step S308.

(Obtain cut surface change parameters)
In step S307, the change operation unit 104 acquires a change parameter for changing the cut surface of the first three-dimensional image data, and returns the process to step S303. As a change parameter, a rotation angle L about an axis about a straight line connecting two feature points is acquired. This parameter can be acquired by operating a mouse constituting the change operation unit 104. For example, the change operation unit 104 obtains the rotation angle L based on the change operation of the change operation unit 104 (for example, the position, movement amount, and movement direction of the mouse while the mouse button is pressed). Details of acquisition of the rotation angle L are as described in the first embodiment.

(Determining whether to re-specify the specified point)
In step S308, the image processing apparatus 100 determines whether or not to respecify the designated point of the first three-dimensional image data. If the user viewing the cross-sectional image displayed in step S305 determines that the designated point does not correspond, an input is made to re-designate the designated point. If re-designation determination is input, the process proceeds to step S309. On the other hand, if the re-designation determination is not input, the process proceeds to step S310.

(Redesignation of designated points)
In step S309, the designation unit 106 of the image processing apparatus 100 designates a designated point in the three-dimensional image data that is different from the designated point designated in step S301 when the designated point is designated again. The designation unit 106 re-designates the position of the designated point of the first or second 3D image data to a position of a different designated point, returns the process to step S303, and each unit of the image processing apparatus 100 performs step S303. The same processing is performed thereafter. The user observes the first cross-sectional image and evaluates the similarity between the lesion part indicated by the designated point and the peripheral part indicated by the feature point in a multifaceted manner, and the position of the target lesion part or the corresponding lesion part. Can be specified more accurately.

(Determine the end of the specified process)
In step S310, the image processing apparatus 100 determines whether or not to end the designated point designation processing of the first three-dimensional image data. For example, an end determination is input by, for example, the user clicking the end button arranged on the display unit 180 with the mouse. If it is determined that the designation process is to be terminated, the process proceeds to step S311. On the other hand, if it is determined not to end, the process returns to step S305, and each unit of the image processing apparatus 100 performs the same process after step S305. Here, when there is no input for changing the cutting plane or re-designating the designated point, the apparatus enters a standby state with the cross-sectional image displayed until an end input is received, and accepts the input.

(Output of specified point)
In step S311, the image processing apparatus 100 stores the position of the designated point in a storage unit (not shown). Further, the display control unit 114 outputs information on the designated point to the display unit 180 and causes the display unit 180 to display the information.

  According to the present embodiment, a cross-sectional image including a designated point determined for a lesion portion of one three-dimensional image data and two feature points selected based on the positional relationship between the designated points is generated and displayed. be able to. Thereby, diagnosis of a lesioned part can be supported.

  Although each embodiment has been described in detail above, the present invention can take an embodiment as a system, apparatus, method, program, storage medium, or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices by distributing the functions of the image processing apparatus, or may be applied to an apparatus composed of a single device.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

100 medical image processing apparatus, 102 acquisition unit, 104 change operation unit,
106 designation unit, 108 selection unit, 110 calculation unit,
112 generator, 114 display controller,
150 first medical imaging device, 160 second medical imaging device,
170 Data server, 180 display unit

Claims (19)

An acquisition unit for acquiring three-dimensional image data and information representing positions of a plurality of feature points in the three-dimensional image data;
A designation unit for designating a designated point indicating a site of interest in the three-dimensional image data;
A selection unit that selects, from the plurality of feature points, a feature point used for generating a cross-sectional image of the three-dimensional image data based on a predetermined selection order;
A generating unit that generates a cross-sectional image including a surface defined based on the position of the designated point and the position of the selected feature point from the three-dimensional image data;
A display control unit for displaying the cross-sectional image on a display unit;
With
The selection unit, based on the positional relationship between the positions of the plurality of feature points and the position of the designated point, you and determines the order in which to select a feature point from among the plurality of feature points image Image processing device. An acquisition unit for acquiring three-dimensional image data and information representing positions of a plurality of feature points in the three-dimensional image data;
A designation unit for designating a designated point indicating a site of interest in the three-dimensional image data;
A selection unit that selects, from the plurality of feature points, a feature point used for generating a cross-sectional image of the three-dimensional image data based on a predetermined selection order;
A generating unit that generates a cross-sectional image including a surface defined based on the position of the designated point and the position of the selected feature point from the three-dimensional image data;
A display control unit for displaying the cross-sectional image on a display unit;
With
The selection unit determines an order of selecting feature points from the plurality of feature points based on a positional relationship between a reference point indicating a specific part in the subject and the positions of the plurality of feature points. images processor you wherein a. An acquisition unit for acquiring three-dimensional image data and information representing positions of a plurality of feature points in the three-dimensional image data;
A designation unit for designating a designated point indicating a site of interest in the three-dimensional image data;
A selection unit that selects, from the plurality of feature points, a feature point used for generating a cross-sectional image of the three-dimensional image data based on a predetermined selection order;
A generating unit that generates a cross-sectional image including a surface defined based on the position of the designated point and the position of the selected feature point from the three-dimensional image data;
A display control unit for displaying the cross-sectional image on a display unit;
With
The selection unit, based on the respective positional relationship of the plurality of feature points, said plurality of images processing device shall be the determining means determines the order of selecting the feature points from the feature point. Wherein the feature point selected by the selection unit, an image processing apparatus according to any one of claims 1 to 3, characterized in that includes a reference point indicating a specific site in the subject. The generation unit sequentially generates a cross-sectional image including a surface defined based on the designated point and the feature points sequentially selected from the three-dimensional image data,
The display controller, an image processing apparatus according to any one of claims 1 to 4, characterized in that to display a plurality of tomographic images wherein are sequentially generated on the display unit. A calculation unit that calculates a surface determined based on the specified designated point and the selected feature point as a cut surface of the three-dimensional image data;
The generating unit, an image processing apparatus according to cross-sectional images in a cutting plane to any one of claims 1 to 5, characterized in that generating from said three-dimensional image data. An acquisition unit for acquiring three-dimensional image data and information representing positions of a plurality of feature points in the three-dimensional image data;
A designation unit for designating a designated point indicating a site of interest in the three-dimensional image data;
A selection unit that selects, from the plurality of feature points, a feature point used for generating a cross-sectional image of the three-dimensional image data based on a predetermined selection order;
A generating unit that generates a cross-sectional image including a surface defined based on the position of the designated point and the position of the selected feature point from the three-dimensional image data;
A display control unit for displaying the cross-sectional image on a display unit;
A calculation unit that calculates a surface determined based on the specified designated point and the selected feature point as a cut surface of the three-dimensional image data;
A change operation unit for obtaining a change parameter for changing the cutting plane ;
Bei to give a,
The calculating unit, the change parameter images processor you and changes the cutting plane of the 3-dimensional image data by changing the rotation angle based on. The generation unit generates, from the three-dimensional image data, a cross-sectional image in the cut plane calculated by the calculation unit or in the cut plane changed by the calculation unit based on the change parameter. The image processing apparatus according to claim 7 . The designation unit re-designates a designated point different from the designated point in the three-dimensional image data when the designated point is designated again.
The calculation unit calculates a surface determined based on the redesignated designated point and the selected feature point as a cut surface of the three-dimensional image data,
The image processing apparatus according to claim 7 , wherein the generation unit generates a cross-sectional image including the calculated cut surface from the three-dimensional image data. The selection unit reselects a feature point different from the feature point from the plurality of feature points when reselection of the feature point is instructed,
The calculation unit calculates a surface determined based on the designated point and the reselected feature point as a cut surface of the three-dimensional image data,
The image processing apparatus according to claim 7 , wherein the generation unit generates a cross-sectional image including the calculated cut surface from the three-dimensional image data. The acquisition unit
As the three-dimensional image data, first three-dimensional image data and second three-dimensional image data are acquired,
Information representing the positions of the plurality of feature points includes information representing the positions of the first plurality of feature points in the first three-dimensional image data and the second plurality of feature points in the second three-dimensional image data. Get information that represents the location of
Obtaining association information associating information representing the positions of the first plurality of feature points and information representing the positions of the second plurality of feature points;
The designation unit is:
Designating the position of the first designated point in the first three-dimensional image data and the position of the second designated point in the second three-dimensional image data;
The selection unit includes:
Selecting a first feature point used for generating a cross-sectional image of the first three-dimensional image data from the first plurality of feature points based on a predetermined selection order;
Based on the association information, a second feature point corresponding to the selected first feature point is selected from the second plurality of feature points,
The generator is
Generating a first cross-sectional image including a first cut surface defined based on the position of the first designated point and the position of the first feature point;
Generating a second cross-sectional image including a second cut surface determined based on the position of the second designated point and the position of the second feature point;
The display controller, an image processing apparatus according to any one of claims 1 to 10, characterized in that for displaying the first sectional image and the second cross-sectional image on the display unit. A display mode selection unit that selects any one of a pair display mode in which a set of cross-sectional images are arranged and a tiled display mode in which a plurality of sets of cross-sectional images are arranged in a tile;
The display controller, an image processing apparatus according to any one of claims 1 to 11, characterized in that for controlling the display of the cross-sectional image in accordance with the selected display mode. 13. The display control unit according to claim 12 , wherein, when the tiled display mode is selected by the display mode selection unit, the display control unit collectively displays sequentially generated cross-sectional images on the display unit. The image processing apparatus described. A cross-sectional image selection unit for selecting a set of cross-sectional images from a plurality of sets of cross-sectional images displayed in a tiled manner according to an operation input for selecting a set of cross-sectional images;
When the cross-sectional image selection unit selects the set of cross-sectional images, the display mode selection unit switches the display mode from the tiled display mode to the pair display mode,
The image processing apparatus according to claim 12, wherein the display control unit controls display of the selected cross-sectional image according to the switching. An acquisition step of acquiring three-dimensional image data and information representing positions of a plurality of feature points in the three-dimensional image data;
A designating step of designating a designated point indicating a site of interest in the three-dimensional image data ;
A selection step of selecting, from the plurality of feature points, a feature point used for generating a cross-sectional image of the three-dimensional image data based on a predetermined selection order ;
Generating a cross-sectional image including a surface defined based on the position of the designated point and the position of the selected feature point from the three-dimensional image data ;
A display control step of displaying the cross-sectional image on a display unit ,
In the selecting step, image processing is characterized in that an order of selecting feature points from the plurality of feature points is determined based on a positional relationship between the positions of the designated points and the positions of the plurality of feature points. Method. An acquisition step of acquiring three-dimensional image data and information representing positions of a plurality of feature points in the three-dimensional image data;
  A designating step of designating a designated point indicating a site of interest in the three-dimensional image data;
  A selection step of selecting, from the plurality of feature points, a feature point used for generating a cross-sectional image of the three-dimensional image data based on a predetermined selection order;
  Generating a cross-sectional image including a surface defined based on the position of the designated point and the position of the selected feature point from the three-dimensional image data;
  A display control step of displaying the cross-sectional image on a display unit,
  In the selection step, an order of selecting feature points from the plurality of feature points is determined based on a positional relationship between a reference point indicating a specific part in the subject and the positions of the plurality of feature points. An image processing method. An acquisition step of acquiring three-dimensional image data and information representing positions of a plurality of feature points in the three-dimensional image data;
  A designating step of designating a designated point indicating a site of interest in the three-dimensional image data;
  A selection step of selecting, from the plurality of feature points, a feature point used for generating a cross-sectional image of the three-dimensional image data based on a predetermined selection order;
  Generating a cross-sectional image including a surface defined based on the position of the designated point and the position of the selected feature point from the three-dimensional image data;
  A display control step of displaying the cross-sectional image on a display unit,
  In the selecting step, an order of selecting feature points from the plurality of feature points is determined based on a positional relationship between the plurality of feature points. An acquisition step of acquiring three-dimensional image data and information representing positions of a plurality of feature points in the three-dimensional image data;
  A designating step of designating a designated point indicating a site of interest in the three-dimensional image data;
  A selection step of selecting, from the plurality of feature points, a feature point used for generating a cross-sectional image of the three-dimensional image data based on a predetermined selection order;
  Generating a cross-sectional image including a surface defined based on the position of the designated point and the position of the selected feature point from the three-dimensional image data;
  A display control step of displaying the cross-sectional image on a display unit;
  A calculation step of calculating a surface determined based on the specified designated point and the selected feature point as a cut surface of the three-dimensional image data;
  A change operation step of obtaining a change parameter for changing the cut surface,
  In the calculation step, the cut surface of the three-dimensional image data is changed by changing a rotation angle based on the change parameter. A program for causing a computer to execute the image processing method according to any one of claims 15 to 18 .