JP6847011B2 - 3D image processing equipment and methods and programs - Google Patents

Description

The present invention relates to a three-dimensional image processing device, a method, and a program that support the insertion of an endoscope into a duct in the body such as a bronchus by using a three-dimensional image obtained by photographing a subject. ..

In recent years, with the progress of the photographing apparatus, the resolution of the image data photographed by the photographing apparatus has been improved, and detailed analysis of the subject can be performed based on the image data. For example, multi-slice CT (Multi Detector-row Computed Tomography) can take a plurality of tomographic images at a time, and can take a tomographic image with a thin slice thickness. By reducing the slice thickness, the resolution in the body axis direction of the three-dimensional image obtained by stacking a plurality of tomographic images is increased, and a more detailed three-dimensional image can be obtained. By displaying and analyzing such a three-dimensional image, it is possible to find lesions and the like that have been difficult to find until now.

As one of the display methods using the three-dimensional image as described above, there is a virtual endoscopic display (see, for example, Patent Document 1 and Patent Document 2). The virtual endoscopic display is a method in which a viewpoint position is set inside the cavity and a perspective projection image is generated and displayed based on the viewpoint position. In the virtual endoscope display, the user can sequentially change the viewpoint position to provide an image as if the camera of the endoscope took a picture while moving inside the body.

In particular, in bronchoscopy, which is an examination using an endoscope, the branching of the bronchus is very complicated, so insert the endoscope while referring to the above-mentioned virtual endoscopic display as a "map". Is being done. At this time, since it is troublesome to manually change the virtual endoscope display according to the actual movement of the endoscope, it is estimated where the tip of the endoscope will be in the body, and the virtual endoscope is used. An image is being created.

For example, in Patent Document 1, it is proposed to perform an alignment process between a virtual endoscope image and an endoscope image actually taken by the endoscope in order to estimate the tip position of the endoscope. ing.

International Publication No. 2014/141968 Japanese Unexamined Patent Publication No. 11-12327 Japanese Unexamined Patent Publication No. 2013-192741

Here, in order to estimate the tip position of the endoscope for navigation, virtual endoscope images from a plurality of viewpoints are generated, and they are aligned with the actually captured endoscope image. And choose the one that best matches.

However, in order to improve the estimation accuracy and stability of the tip position of the endoscope, it is necessary to generate a large number of virtual endoscope images, but the volume rendering process, which is the process of generating virtual endoscope images, is extremely difficult. It takes time.

In Patent Document 3, a graph structure is generated from a three-dimensional image of the bronchus, and for each of a plurality of branch points included in the graph structure, the branch destination from the branch point is a two-dimensional plane including the branch point. It has been proposed to generate a projected image projected above, but the projected image is a schematic image showing the branch destination, not an image that accurately represents the shape of the bronchial hole, etc. It is not suitable for the alignment process with the endoscopic image as described above.

In view of the above circumstances, the present invention provides a three-dimensional image processing apparatus, method, and program capable of generating an image with simple features so that it can be aligned with an endoscopic image by high-speed processing. It is intended to be provided.

The three-dimensional image processing apparatus of the present invention has a three-dimensional image acquisition unit that acquires a three-dimensional image obtained by photographing a subject, and a graph structure generation that generates a graph structure of a tubular structure included in the three-dimensional image. Based on the part, the contour information acquisition part that acquires the contour information of the tubular structure at each point on the graph structure, the viewpoint information acquisition part that acquires the viewpoint information in the tubular structure, and the viewpoint information and the graph structure. It is provided with a projection point specifying unit that specifies a projection point from each point on the graph structure, and a projection image generation unit that generates a projection image in which contour information of the projection points is projected on a two-dimensional plane.

Further, in the three-dimensional image processing apparatus of the present invention, the projection point specifying unit identifies one point on the graph structure as a starting point, traces the graph structure from the starting point, and includes points within a preset range. Can be specified as a projection candidate point, and the projection point can be specified from the projection candidate points.

Further, in the three-dimensional image processing apparatus of the present invention, the projection point specifying unit can specify the projection point based on the shape information of the contour information.

Further, in the three-dimensional image processing apparatus of the present invention, the projection image generation unit can acquire information on the branch to which the projection point belongs and add the branch information to the projection image.

Further, in the three-dimensional image processing apparatus of the present invention, the viewpoint information of different images is based on the result of the alignment processing between the projected image and the image different from the projected image and the viewpoint information of the projected image. It is possible to provide a viewpoint information estimation unit for estimating.

Further, in the three-dimensional image processing apparatus of the present invention, different images corresponding to the contour information included in the projected image are obtained based on the result of the alignment processing between the projected image and the image different from the projected image. The contour information estimation unit for estimating the contour information included in the above can be provided.

Further, in the three-dimensional image processing apparatus of the present invention, an image different from the projected image can be a two-dimensional image obtained by photographing the subject.

Further, in the three-dimensional image processing apparatus of the present invention, the two-dimensional image can be an image taken by an endoscope.

Further, the three-dimensional image processing device of the present invention may include a display control unit for displaying a projected image on the display device.

Further, in the three-dimensional image processing apparatus of the present invention, the tubular structure can be a bronchus.

The three-dimensional image processing method of the present invention acquires a three-dimensional image obtained by photographing a subject, generates a graph structure of a tubular structure included in the three-dimensional image, and is tubular at each point on the graph structure. The contour information of the structure is acquired, the viewpoint information in the tubular structure is acquired, the projection point is specified from each point on the graph structure based on the viewpoint information and the graph structure, and the contour information of the projection point is obtained. Is projected onto a two-dimensional plane to generate a projected image.

The three-dimensional image processing program of the present invention uses a computer to generate a three-dimensional image acquisition unit that acquires a three-dimensional image obtained by photographing a subject and a graph structure of a tubular structure included in the three-dimensional image. A graph structure generation unit, a contour information acquisition unit that acquires contour information of a tubular structure at each point on the graph structure, a viewpoint information acquisition unit that acquires viewpoint information in the tubular structure, and a viewpoint information and a graph structure. Based on the above, it functions as a projection point identification unit that specifies a projection point from each point on the graph structure and a projection image generation unit that generates a projection image in which contour information of the projection point is projected on a two-dimensional plane.

The other three-dimensional image processing apparatus of the present invention includes a memory for storing instructions to be executed by a computer and a processor configured to execute the stored instructions, and the processor obtains a photograph of a subject. The process of acquiring the obtained 3D image and generating the graph structure of the tubular structure included in the 3D image, the process of acquiring the contour information of the tubular structure at each point on the graph structure, and the inside of the tubular structure. The process of acquiring the viewpoint information in the above, the process of specifying the projection point from each point on the graph structure based on the viewpoint information and the graph structure, and the projection of the contour information of the projection point projected on the two-dimensional plane. It is configured to perform the process of generating an image.

According to the three-dimensional image processing apparatus and method sequence program of the present invention, a three-dimensional image obtained by photographing a subject is acquired, and a graph structure of a tubular structure included in the three-dimensional image is generated to generate a graph structure. Obtain contour information of the tubular structure at each of the above points. Then, the viewpoint information in the tubular structure is acquired, the projection point is specified from each point on the graph structure based on the viewpoint information and the graph structure, and the contour information of the projection point is projected on the two-dimensional plane. Generate the projected image.

Therefore, it is possible to generate a projected image with a simple feature so that the alignment process with the endoscopic image can be performed by high-speed processing.

A block diagram showing a schematic configuration of an endoscopic image diagnosis support system using an embodiment of the three-dimensional image processing apparatus of the present invention. A diagram showing an example of a graph structure A diagram showing an example of bronchial contour information at each point on the graph structure. Flowchart to explain how to identify the projection point Explanatory drawing for explaining the method of specifying a projection point A diagram showing an example of contour information of each point on the graph structure. Diagram showing an example of a projected image The figure which shows an example of the projection image generated using the contour information of the bronchus of the first branch. The figure which shows an example of the projection image generated so that the contour information does not overlap. The figure which shows an example of the projection image generated using the contour information of the finally identified projection point. The figure which shows the example which added the branch information to the projected image and displayed it. A block diagram showing a schematic configuration of an endoscopic image diagnosis support system using another embodiment of the three-dimensional image processing apparatus of the present invention. Explanatory drawing for explaining a method of estimating the viewpoint information of an endoscopic image based on the amount of deviation between a projected image and an endoscopic image and the viewpoint information of the projected image. Explanatory drawing for explaining a method of estimating a hole (contour information) included in an endoscopic image corresponding to the contour information included in a projected image. The figure which shows an example of the contour image which extracted the contour of a hole from an endoscopic image

Hereinafter, the endoscopic image diagnosis support system using the three-dimensional image processing apparatus and method of the present invention and one embodiment of the program will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of the endoscopic image diagnosis support system of the present embodiment.

As shown in FIG. 1, the endoscopic image diagnosis support system 1 of the present embodiment includes a three-dimensional image processing device 10, a display device 20, an input device 30, and a three-dimensional image storage server 40. ..

The three-dimensional image processing device 10 is configured by installing the three-dimensional image processing program of the present embodiment on a computer.

The three-dimensional image processing device 10 includes a central processing unit (CPU), a semiconductor memory, a hard disk in which the above-mentioned three-dimensional image processing program is installed, a storage device such as an SSD (Solid State Drive), and the like. By hardware

, 3D image acquisition unit 11, graph structure generation unit 12, contour information acquisition unit 13, viewpoint information acquisition unit 14, projection point identification unit 15, projection image generation unit 16, bronchial 3D image generation unit as shown in FIG. 17 and a display control unit 18 are configured. Then, each of the above parts operates by executing the three-dimensional image processing program installed on the hard disk by the central processing unit.

The three-dimensional image acquisition unit 11 acquires a three-dimensional image of the subject taken in advance before surgery or examination using an endoscope device. Examples of the three-dimensional image include volume data reconstructed from slice data output from a CT device and an MRI (Magnetic Resonance Imaging) device, and an MS (Multi Slice) CT device and Cornbee.

There is volume data output from the CT device. The three-dimensional image is stored in advance in the three-dimensional image storage server 40 together with the identification information of the subject, and the three-dimensional image acquisition unit 11 corresponds to the identification information of the subject input in the input device 30. The three-dimensional image is read from the three-dimensional image storage server 40.

The graph structure generation unit 12 inputs a three-dimensional image acquired by the three-dimensional image acquisition unit 11, and generates a graph structure of a tubular structure included in the input three-dimensional image. In this embodiment, a bronchial graph structure is generated as a graph structure of a tubular structure. Hereinafter, an example of a method for generating this graph structure will be described.

The bronchi included in the three-dimensional image appear as a region showing a low CT value (pixel value) on the CT image because the pixels inside the bronchus correspond to the air region, but the bronchial wall is a cylinder showing a relatively high CT value. Alternatively, it is considered to be a tubular structure. Therefore, the bronchi are extracted by performing structural analysis of the shape based on the distribution of CT values for each pixel.

The bronchus branches in multiple stages, and the diameter of the bronchus becomes smaller as it approaches the end. A Gaussian pyramid image obtained by converting a three-dimensional image into multiple resolutions so that bronchi of different sizes can be detected, that is, a plurality of three-dimensional images having different resolutions are generated in advance, and a detection algorithm is used for each generated Gaussian pyramid image. Detects tubular structures of different sizes by scanning.

First, the Hessian matrix of each pixel of the three-dimensional image of each resolution is calculated, and it is determined whether or not the pixels are in the tubular structure from the magnitude relation of the eigenvalues of the Hessian matrix. The Hessian matrix is a matrix whose elements are the second-order partial differential coefficients of the density values in each axis (x-axis, y-axis, z-axis of the three-dimensional image) direction, and is a 3 × 3 matrix as shown in the following equation. ..

When the eigenvalues of the Hessian matrix in an arbitrary pixel are λ1, λ2, and λ3, two of the eigenvalues are large and one eigenvalue is close to 0, for example, when λ3, λ2 >> λ1, λ1≈0 is satisfied. , The pixels are known to be tubular structures. Further, the eigenvectors corresponding to the minimum eigenvalues (λ1≈0) of the Hessian matrix coincide with the spindle direction of the tubular structure.

Although the bronchi can be represented by a graph structure, the tubular structure extracted in this way is not always detected as one graph structure in which all the tubular structures are connected due to the influence of a tumor or the like. Therefore, after the discrimination of the entire three-dimensional image is completed, the detected tubular structure is within a certain distance, and the direction of the basic line connecting arbitrary points on the two extracted tubular structures and each tubular structure. By evaluating whether the angle formed by the main axis direction of the object is within a certain angle, it is determined whether or not a plurality of tubular structures are connected, and the connection relationship of the extracted tubular structures is determined. Rebuild. This reconstruction completes the extraction of the bronchial graph structure.

Then, the extracted graph structure is classified into a start point, an end point, a branch point, and a branch, and by connecting the start point, the end point, and the branch point with a branch, a graph structure representing a bronchi can be obtained. In the present embodiment, features such as the diameter of the bronchus and the length of each branch (the length between the bifurcation points of the bronchi) at each position of the graph structure are also acquired together with the graph structure. FIG. 2 shows an example of a graph structure. In FIG. 2, Sp is a start point, a branch point Bp is represented by a white circle, an end point Ep is represented by a black circle, and a branch E is represented by a line.

The graph structure generation method is not limited to the above-mentioned method, and other known methods can be used.

The contour information acquisition unit 13 acquires contour information of the bronchi at each point on the graph structure. The contour information acquisition unit 13 of the present embodiment acquires the contour information of the tubular structure detected when the graph structure of the bronchi is generated as described above. The contour information is acquired for each point in the graph structure, and the interval between the points can be arbitrarily set to be smaller than the interval between the bronchial bifurcation points. For example, it is preferable to set it to about 1 mm to 2 mm. FIG. 3 is a diagram showing an example of bronchial contour information at each point on the graph structure. Note that FIG. 3 also includes a bronchial graph structure.

The viewpoint information acquisition unit 14 acquires viewpoint information in the bronchus. The viewpoint information is arbitrarily set and input by the user using the input device 30, and the viewpoint information acquisition unit 14 acquires the viewpoint information received by the input device 30. The setting input of the viewpoint information may be specified by the user using an input device 30 such as a mouse on a three-dimensional image of the bronchus displayed on the display device 20, for example.

In the present embodiment, the viewpoint information is set and input by the user, but the present invention is not limited to this, and the viewpoint information may be automatically set based on preset conditions. Specifically, for example, the viewpoint information may be set at the base end of the bronchus, or the viewpoint information may be set at the first branch from the base end.

The projection point specifying unit 15 identifies one point on the graph structure as a starting point based on the viewpoint information, traces the graph structure from the starting point, and specifies the projection point from each point on the graph structure. .. Hereinafter, the identification of the projection point by the projection point specifying unit 15 will be described with reference to the flowchart shown in FIG. 4 and FIGS. 5 to 10.

As shown in FIG. 5, the projection point specifying unit 15 first identifies the point on the graph structure closest to the viewpoint information S set and input by the user as the starting point Ns (S10). Then, the graph structure is traced from the starting point Ns toward the downstream side of the bronchus (the side opposite to the proximal end side), and points included in the preset range NR are specified as projection candidate points (S12). The range NR can be, for example, a range of a preset distance from the starting point Ns, or a range in which the number of branch points to be passed is a preset number when the graph structure is traced from the starting point Ns.

Next, the projection point specifying unit 15 identifies a part of the projection points from the plurality of projection candidate points included in the preset range NR based on the preset projection point conditions (S14). .. As preset conditions, for example, the central point, the first point, or the last point of each branch in the graph structure in the range NR can be specified as a projection point. The first point and the last point are the first point and the last point of each side when the graph structure is traced to the downstream side. Further, for each branch in the graph structure in the range NR, the first point separated from the branch point by a preset distance or more can be specified as a projection point. FIG. 4 shows an example of the projection point Np specified in S14 with a dotted circle.

Next, the projection point specifying unit 15 confirms whether or not the projection point specified in S14 satisfies the preset projected image condition (S16). FIG. 6 is a diagram showing an example of contour information of each point on the graph structure within the range NR. In FIG. 6, contour information of some points is not shown for easy viewing. The numerical values 0 to 6 shown in FIG. 6 are the branch numbers added to each branch, and the numerical values shown in the vicinity of each contour information are the node numbers added to each point on the graph structure. Here, for example, a method of specifying the projection point when the above-mentioned starting point Ns is at the node 80 shown in FIG. 6 will be described.

First, when the projection points specified in S14 are the node 98 and the node 99 shown in FIG. 6, the contour information of these two node points is projected onto a two-dimensional plane to generate a projected image, which is shown in FIG. It becomes a projected image like this. The two-dimensional plane is a plane orthogonal to the body axis direction of the subject. In the projected image shown in FIG. 7, since the two contour information overlaps each other, it is not very preferable as the projected image. This is because the projection point specified by S14 is too close to the branch point.

Therefore, when the two contour information overlaps with each other as in the projected image shown in FIG. 7, the projection point specifying unit 15 changes the projection point and uses the contour information of the changed projection point to re-project the projected image. Is generated, and it is confirmed whether or not the contour information of the projected image overlaps. Specifically, at least one of the projection points of the node 98 and the node 99 is changed to a projection point far from the branch point, and the projection image is generated again using the contour information of the changed projection point. For example, the projection point of node 98 is changed to the projection point of node 107, and the projection point of node 99 is changed to the projection point of node 110.

That is, the projection point specifying unit 15 confirms whether or not the projection image condition that the contour information of the projection images does not overlap is satisfied, and if the projection image condition is not satisfied (S16, NO), the projection point is set in advance. Change based on the set conditions (S18). Then, using the contour information of the changed projection point, the projection image is generated again, it is confirmed whether or not the projection image condition is satisfied, and the projection point is changed and the projection image is generated until the projection image condition is satisfied. repeat. FIG. 8 is a diagram showing a projected image generated by using the contour information of the projection points of the node 123 and the node 119 that satisfy the projected image condition.

As for the branch 3 and the branch 4 connected to the tip of the branch 1 shown in FIG. 6, and the branch 5 and the branch 6 connected to the branch 2, the projection image condition that the contour information of the projected image does not overlap is the same as described above. The projection point is changed and the projection image is generated repeatedly until the condition is satisfied. As a result, assuming that a projected image as shown in FIG. 9 is generated, the node 289 of the branch 4 and the node 296 of the branch 3 protruding from the node 123 of the branch 1 which is the parent branch, and the branch 2 which is the parent branch The contour information of the node 214 of the branch 6 protruding from the contour information of the node 119 of the above is the contour information that cannot be seen on the endoscopic image actually taken from the node 80, which is the starting point, and should be deleted. Is preferable.

Therefore, the projection point specifying unit 15 specifies a projection point that satisfies the projection image condition that the contour information of the child branch is included in the contour information of the node of the parent branch for the child branch. The projection image condition for the child branch is not limited to this, and for example, even if the contour information of the child branch is not included in the contour information of the node of the parent branch, the contour of the node of the parent branch is not included. If the distance between the information and the contour information of the child branch is within a preset threshold, it may be left as the final projection point without being deleted. Specifically, the contour information of the node 214 of the branch 6 shown in FIG. 9 may be left because it is close to the contour information of the node 119 of the branch 2. Further, even if the contour information of the node of the child branch is included in the contour information of the node of the parent branch, the shape of the contour information has a major axis like the contour information of the node 289 shown in FIG. If the ratio of the minor axis to is an ellipse below a preset threshold, or if the contour information of the child branch node cannot be projected, or if the contour information of the parent branch node is greater than the size of the child branch node. If the ratio of the size of the contour information is less than or equal to the threshold value and the contour information of the node of the child branch is too small, the contour information may be deleted.

As described above, the final projection point is specified by repeatedly changing and deleting the projection point so as to satisfy the projection image condition (S20). FIG. 10 is a diagram showing a projected image generated using the contour information of the finally identified projection point.

For confirmation of whether or not the above-mentioned projection image conditions are satisfied, the shape information of the contour information of the projection point may be used. Specifically, as the shape information, the diameter (radius, diameter, minor axis, major axis, etc.) of the contour information, the distance between the centers, and the like may be used.

Further, in the above description, the final projection point is specified by temporarily generating a projection image using the contour information of the tentatively specified projection point and confirming whether or not the projection image satisfies the projection image condition. However, it is not always necessary to generate a projected image, and the projected image condition is satisfied based on the positional relationship of the tentatively specified projection point on the three-dimensional coordinate space and the magnitude of the contour information of the projection point. You may want to check if it is.

Further, in the above description, the final projection point is specified by confirming whether or not the contour information of the specified projection point satisfies the projection image condition, but the projection image condition is not necessarily confirmed. Alternatively, the projection point specified based on the projection point condition in S14 may be used as the final projection point, and the projection image may be generated using the contour information.

Returning to FIG. 1, the bronchial three-dimensional image generation unit 17 generates a bronchial three-dimensional image representing the morphology of the bronchi by performing volume rendering processing on the three-dimensional image acquired by the three-dimensional image acquisition unit 11. , This bronchial three-dimensional image is output to the display control unit 18.

The display control unit 18 causes the display device 20 to display the projection image generated by the projection image generation unit 16 and the bronchial three-dimensional image generated by the bronchial three-dimensional image generation unit 17.

The display device 20 includes, for example, a liquid crystal display. Further, the display device 20 may be configured by a touch panel and may also be used as the input device 43.

The input device 30 includes a mouse, a keyboard, and the like, and accepts various setting inputs by the user.

According to the endoscopic image diagnosis support system of the above embodiment, a three-dimensional image obtained by photographing a subject is acquired, a graph structure of a tubular structure included in the three-dimensional image is generated, and the graph structure is displayed. The contour information of the tubular structure at each point of is acquired. Then, the viewpoint information in the tubular structure is acquired, one point on the graph structure is specified as the starting point based on the viewpoint information, the graph structure is traced from the starting point, and the projection point is selected from each point on the graph structure. Is specified, and a projected image in which the contour information of the projection point is projected on a two-dimensional plane is generated.

Therefore, it is possible to generate a projected image with a simple feature so that the alignment process with the endoscopic image can be performed by high-speed processing.

Further, in the above embodiment, the projection image generation unit 16 acquires the information of the branch to which the projection point belongs and adds the information of the branch to the projection image so that the information of the branch is displayed on the projection image. It may be. FIG. 11 is a diagram showing an example in which "A" to "D" are added as branch information to the projected image and displayed.

Further, in the above embodiment, an endoscopic image of a two-dimensional image actually taken in the bronchi by the endoscope device 50 is acquired, and endoscopy is performed based on the relationship between the endoscopic image and the projected image. The viewpoint information of the mirror image (corresponding to the tip position of the endoscope) may be estimated. That is, as shown in FIG. 12, the viewpoint information estimation unit 21 may be provided. Specifically, the viewpoint information estimation unit 21 performs alignment processing between the projection image A generated by the projection image generation unit 16 and the endoscope image B output from the endoscope device 50. As the alignment process, for example, rigid body registration or non-rigid body registration can be used.

Then, as shown in FIG. 13, the viewpoint information estimation unit 21 is based on the amount of deviation between the projected image A and the endoscopic image B obtained by the alignment process and the viewpoint information of the projected image A. The viewpoint information b of the endoscopic image B is estimated. The projection image A to be subjected to the alignment processing is generated for each branch point of the graph structure, for example, and is based on the projection image A having the smallest deviation amount among the plurality of projection images A and its viewpoint information. Therefore, it is assumed that the viewpoint information b of the endoscopic image is estimated. As a method of estimating the viewpoint information b of the endoscopic image B, for example, when only the size of the contour information included in the projected image A and the size of the bronchial hole included in the endoscopic image B are different, the size thereof. The viewpoint information b can be estimated by moving the viewpoint information a with respect to the projection surface based on the scaling factor and the distance between the viewpoint information a of the projected image A and the projection surface. The method for estimating the viewpoint information b is not limited to such a method, and various estimation methods based on geometrical relationships can be used.

The viewpoint information b estimated by the viewpoint information estimation unit 21 may, for example, display its three-dimensional coordinates on the display device 20, or may be displayed on the bronchial three-dimensional image displayed on the display device 20. You may.

Further, based on the contour information included in the projected image and the branch information added to the contour information, branch information is added to the holes included in the endoscope image output from the endoscope device 50. You may try to do it. That is, as shown in FIG. 12, the contour information estimation unit 22 may be provided. Specifically, the contour information estimation unit 22 performs an alignment process between the projection image A generated by the projection image generation unit 16 and the endoscope image B output from the endoscope device 50. As the alignment process, for example, rigid body registration or non-rigid body registration can be used. Then, the contour information estimation unit 22 estimates the hole (contour information) included in the endoscope image B corresponding to the contour information included in the projected image A as shown in FIG. 14 by performing the alignment process. To do. Then, the contour information estimation unit 22 adds the branch information “A” to “D” added to each contour information of the projected image A to the corresponding holes included in the endoscopic image. The branch information added to each hole of the endoscopic image is displayed on the display device 20 together with the endoscopic image, for example.

Further, in the above description, the viewpoint information estimation unit 21 and the contour information estimation unit 22 use the endoscope image itself output from the endoscope device 50 when performing the alignment processing, but the present invention is limited to this. Instead, as shown in FIG. 15, a contour image obtained by extracting the contour of the hole from the endoscopic image may be used. As a method of generating a contour image, for example, a region in which pixels having a brightness of less than a threshold value are distributed in a circle is detected from the endoscopic image, and the contour of the region is extracted as the contour of a hole to generate a contour image. You can do it like this.

1 Endoscopic image diagnosis support system 10 3D image processing device 11 3D image acquisition unit 12 Graph structure generation unit 13 Contour information acquisition unit 14 Viewpoint information acquisition unit 15 Projection point identification unit 16 Projection image generation unit 17 Bronchial 3D image Generation unit 18 Display control unit 20 Display device 21 Viewpoint information estimation unit 22 Contour information estimation unit 30 Input device 40 3D image storage server 43 Input device 50 Endoscope device Bp Branch point E Branch Ep End point Np Projection point NR Range Ns Origin S Viewpoint information

Claims (13)

A 3D image acquisition unit that acquires a 3D image obtained by photographing a subject, and
A graph structure generator that generates a graph structure of a tubular structure included in the three-dimensional image,
A contour information acquisition unit that acquires contour information representing the diameter and shape of the tubular structure at each point on the graph structure, and
A viewpoint information acquisition unit that acquires viewpoint information in the tubular structure,
When the contour information of each point is projected onto a two-dimensional plane in the direction of the viewpoint based on the viewpoint information from each point on the graph structure based on the viewpoint information and the graph structure. , A projection point specifying unit that specifies a point that satisfies the condition that the contour information does not overlap as a projection point,
A three-dimensional image processing apparatus including a projection image generation unit that generates a projection image obtained by projecting the contour information of the projection point onto the two-dimensional plane. The three-dimensional image processing according to claim 1, wherein the projection image generation unit acquires information for identifying a branch to which the projection point belongs, and adds information for identifying the branch to the contour information included in the projection image. apparatus. The projection point specifying unit identifies one point on the graph structure as a starting point, traces the graph structure from the starting point, and sets a point on the graph structure included in a preset range as a projection candidate point. The three-dimensional image processing apparatus according to claim 1 or 2 , wherein the projection point is specified and the projection point is specified from the projection candidate points. Claim that the projection point specifying portion identifies the projection point based on the diameter of the contour of the tubular structure represented by the contour information at each point on the graph structure and the distance between the centers of the contour. The three-dimensional image processing apparatus according to any one of 1 to 3. A claim including a viewpoint information estimation unit that estimates the viewpoint information of the different image based on the result of the alignment process between the projected image and an image different from the projected image and the viewpoint information of the projected image. Item 3. The three-dimensional image processing apparatus according to any one of Items 1 to 4. Contour information that estimates contour information contained in the different image corresponding to the contour information included in the projected image based on the result of alignment processing between the projected image and an image different from the projected image. The three-dimensional image processing apparatus according to any one of claims 1 to 5, further comprising an estimation unit. The three-dimensional image processing apparatus according to any one of claims 1 to 6 , wherein an image different from the projected image is a two-dimensional image obtained by photographing the subject. The three-dimensional image processing apparatus according to claim 7 , wherein the two-dimensional image is an image taken by an endoscope. The three-dimensional image processing device according to any one of claims 1 to 8, further comprising a display control unit for displaying the projected image on the display device. The three-dimensional image processing apparatus according to any one of claims 1 to 9 , wherein the tubular structure is a bronchus. The three-dimensional image processing apparatus according to any one of claims 1 to 10 , wherein the projected image generation unit generates the projected image so as to satisfy the condition that the plurality of contour information does not overlap. Acquire the three-dimensional image obtained by photographing the subject,
A graph structure of the tubular structure included in the three-dimensional image is generated.
The contour information representing the diameter and shape of the tubular structure at each point on the graph structure is acquired.
Obtaining viewpoint information in the tubular structure,
When the contour information of each point is projected onto a two-dimensional plane in the direction of the viewpoint based on the viewpoint information from each point on the graph structure based on the viewpoint information and the graph structure. , The point that satisfies the condition that the contour information does not overlap is specified as the projection point.
A three-dimensional image processing method for generating a projected image in which the contour information of the projection point is projected onto the two-dimensional plane. Computer,
A 3D image acquisition unit that acquires a 3D image obtained by photographing a subject, and
A graph structure generator that generates a graph structure of a tubular structure included in the three-dimensional image,
A contour information acquisition unit that acquires contour information representing the diameter and shape of the tubular structure at each point on the graph structure, and
A viewpoint information acquisition unit that acquires viewpoint information in the tubular structure,
When the contour information of each point is projected onto a two-dimensional plane in the direction of the viewpoint based on the viewpoint information from each point on the graph structure based on the viewpoint information and the graph structure. , A projection point specifying unit that specifies a point that satisfies the condition that the contour information does not overlap as a projection point,
A three-dimensional image processing program that functions as a projection image generation unit that generates a projection image obtained by projecting the contour information of the projection point onto the two-dimensional plane.

Images