JP2008093287A - Medical image processing apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a medical image processing apparatus and a method capable of reducing the burden on a user. <P>SOLUTION: This medical image processing apparatus is provided with an occlusion area detection section for detecting an occlusion area of an object image out of images of biotissue input from a medical capturing apparatus, and an occlusion complement section for complementing the occlusion area of the object image using information acquired from one or a plurality of non-object images, which are different from the object image and have at least parts of the area corresponding to the occlusion area of the object image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a medical image processing apparatus and a medical image processing method, and more particularly to a medical image processing apparatus and a medical image processing method capable of estimating a three-dimensional model corresponding to a two-dimensional image. .

  An endoscope system including an endoscope and a medical image processing apparatus is widely used in the medical field and the like. Specifically, the endoscope system is formed by, for example, an insertion portion that is inserted into a body cavity as a living body, an objective optical system that is disposed at a distal end portion of the insertion portion, and an image formed by the objective optical system. An endoscope configured to include an imaging unit that captures an image of a body cavity and outputs the image as an imaging signal, and displays the image of the body cavity on a monitor or the like as a display unit based on the imaging signal And a medical image processing apparatus that performs processing for this purpose. Then, the user observes an organ or the like as a subject in the body cavity based on the image in the body cavity displayed on the monitor or the like as the display unit.

  In addition, the endoscope system having the above-described configuration can also take an image of the digestive tract mucous membrane such as the large intestine as a subject in the body cavity. Therefore, the user can comprehensively observe various findings such as the color of the mucous membrane, the shape of the lesion, and the fine structure of the mucosal surface.

Further, in recent years, it is possible to estimate a three-dimensional model of a subject based on data of a two-dimensional image corresponding to an imaging signal of an image of the subject imaged by an endoscope. Has been proposed.
JP 11-337845 A

  In particular, as an image of a living tissue having a raised shape imaged by an endoscope, for example, a two-dimensional image including an image of a living tissue such as a fold of the large intestine or a polyp is in the viewing direction of the endoscope. Often includes invisible areas. The invisible region in the above-described two-dimensional image is generally called occlusion and is a region where it is difficult to estimate an accurate three-dimensional model. Therefore, in the portion where the occlusion of the 2D image occurs, for example, there is no estimation result at the corresponding position of the 3D model based on the 2D image, or the 3D model based on the 2D image There is a problem that an estimation result with low reliability is calculated at a corresponding position. In a three-dimensional model generated based on an estimation result with low reliability, it may be difficult to find a lesion site such as a polyp.

  The present invention has been made in view of the above-described points. When a three-dimensional model is estimated based on a two-dimensional image, a portion where data is missing and a portion where data with low reliability exist are compared with the conventional one. The purpose of the present invention is to provide a medical image processing apparatus and a medical image processing method capable of estimating a highly accurate three-dimensional model by reducing the number of lesions and, as a result, easily finding a lesion site such as a polyp. Yes.

  The first medical image processing apparatus according to the present invention is an image different from the occlusion area detection unit that detects an occlusion area of a target image among images of biological tissue input from the medical imaging apparatus, and the target image. And using information obtained from one or more non-target images, which is an image in which at least a part of a region corresponding to the occlusion region of the target image exists, and the occlusion region of the target image is And an occlusion complementation unit for complementation.

  According to a second medical image processing apparatus of the present invention, in the first medical image processing apparatus, each of the one or more non-target images is a two-dimensional image or a tertiary generated based on the two-dimensional image. It is an original model.

  According to a third medical image processing apparatus of the present invention, in the first or second medical image processing apparatus, each of the one or more non-target images is time-sequentially before or after the target image. It is an image of a positioned frame.

  According to a fourth medical image processing apparatus of the present invention, in the first to third medical image processing apparatuses, a three-dimensional model of the biological tissue is estimated based on the target image as a two-dimensional image. A first three-dimensional model estimation unit, and a second three-dimensional model estimation unit that estimates a three-dimensional model of each of the one or more non-target images, and the occlusion region detection unit includes a two-dimensional A first occlusion area detection unit for detecting an occlusion area of the target image as an image; and a second occlusion area detection unit for detecting each voxel corresponding to the occlusion area in a three-dimensional model of the target image. And the occlusion complementation unit is configured to generate the second octocc based on the data of the 3D model estimated by the second 3D model estimation unit. Wherein said detected by, Version area detection unit complements the data for each voxel.

  According to a fifth medical image processing apparatus of the present invention, in the fourth medical image processing apparatus, the fifth medical image processing apparatus further includes an edge extraction unit that extracts an edge of the target image, and the first occlusion region detection unit includes The occlusion area is detected based on a difference in brightness components between areas existing on both sides of the edge.

  According to a sixth medical image processing apparatus of the present invention, in the fourth or fifth medical image processing apparatus, a predetermined region existing at a position corresponding to the occlusion region is detected in the target image. On the basis of the image region detection unit that performs the detection, the position where the predetermined region exists in the target image, and the position where the predetermined region exists in one non-target image among the one or more non-target images. An image area movement detecting unit for detecting a distance and direction in which the predetermined area has moved in the two-dimensional image; and the second occlusion area based on the distance and direction in which the predetermined area has moved in the two-dimensional image. Among the voxel data detected by the detection unit, an occlusion complementation corresponding area is detected as an area where complementary voxel data exists. And a second occlusion region detection unit based on a three-dimensional model of the non-target image estimated by the second three-dimensional model estimation unit. The voxel data existing in the occlusion complementation corresponding area among the data of the respective voxels detected by the above is supplemented.

  The first medical image processing method according to the present invention is an occlusion region detection step for detecting an occlusion region of a target image among images of biological tissue input from a medical imaging device, and an image different from the target image. And using information obtained from one or more non-target images, which is an image in which at least a part of a region corresponding to the occlusion region of the target image exists, and the occlusion region of the target image is An occlusion complementing step for complementing.

  According to a second medical image processing method of the present invention, in the first medical image processing method, each of the one or more non-target images is a two-dimensional image or a tertiary generated based on the two-dimensional image. It is an original model.

  According to a third medical image processing method of the present invention, in the first or second medical image processing method, each of the one or more non-target images may be time-sequentially before or after the target image. It is an image of a positioned frame.

  According to a fourth medical image processing method of the present invention, in the first to third medical image processing methods, a three-dimensional model of the biological tissue is estimated based on the target image as a two-dimensional image. A first three-dimensional model estimation step, and a second three-dimensional model estimation step for estimating a three-dimensional model of each of the one or more non-target images, and the occlusion region detection step includes: A first occlusion area detecting step for detecting an occlusion area of the target image as an image; and a second occlusion area detecting step for detecting each voxel corresponding to the occlusion area in a three-dimensional model of the target image. And the occlusion complementation step includes three steps estimated by the second three-dimensional model estimation step. Based on the data in the original model, characterized by complementing the data of the detected each voxel by the second occlusion region detection step.

  According to a fifth medical image processing method of the present invention, in the fourth medical image processing method, the method further includes an edge extraction step of extracting an edge of the target image, and the first occlusion region detection step includes The occlusion area is detected based on a difference in brightness components between areas existing on both sides of the edge.

  According to a sixth medical image processing method of the present invention, in the fourth or fifth medical image processing method, a predetermined region existing at a position corresponding to the occlusion region is detected in the target image. An image area detecting step, a position where the predetermined area exists in the target image, and a position where the predetermined area exists in one non-target image among the one or more non-target images. An image area movement detecting step for detecting a distance and direction in which the predetermined area has moved in the two-dimensional image; and the second occlusion area based on the distance and direction in which the predetermined area has moved in the two-dimensional image. Of the data of each voxel detected by the detection unit, an area where complementary voxel data exists is defined as an occlusion completion compatible area. An occlusion complementation corresponding region detection step for detecting, wherein the occlusion complementation step is based on the three-dimensional model of the non-target image estimated by the second three-dimensional model estimation step. Of the voxel data detected in the detection step, the voxel data existing in the occlusion complementation corresponding region is complemented.

  According to the medical image processing apparatus and the medical image processing method of the present invention, when a three-dimensional model is estimated based on a two-dimensional image, a missing portion of data and a portion where data with low reliability exist are conventionally used. By reducing the number, it is possible to estimate a highly accurate three-dimensional model, and as a result, it is possible to easily find a lesion site such as a polyp.

  Embodiments of the present invention will be described below with reference to the drawings.

  1 to 9 relate to an embodiment of the present invention. FIG. 1 is a diagram illustrating an example of an overall configuration of an endoscope system in which a medical image processing apparatus according to an embodiment of the present invention is used. FIG. 2 is a schematic diagram showing images of a tubular organ and a living tissue imaged by the endoscope of FIG. FIG. 3 is a flowchart illustrating an example of a process for detecting an occlusion region in a three-dimensional model estimated from a first two-dimensional image, which is a process performed by the medical image processing apparatus in FIG. FIG. 4 is a schematic diagram showing an example of a raised lesion site that can be imaged by the endoscope of FIG. FIG. 5 is a schematic diagram showing an example different from FIG. 4 of a raised lesion site that can be imaged by the endoscope of FIG. FIG. 6 is a diagram illustrating a positional relationship between one edge, the center-of-gravity coordinates G1, and one occlusion end coordinate O1 in the first two-dimensional image acquired by the medical image processing apparatus in FIG. FIG. 7 is a flowchart illustrating an example of processing for complementing an occlusion region in a three-dimensional model estimated from a first two-dimensional image, which is processing performed by the medical image processing apparatus in FIG. FIG. 8 is a schematic diagram illustrating an example when a three-dimensional model based on the first two-dimensional image is viewed from the side with respect to the visual field direction of the endoscope of FIG. 1. FIG. 9 is a schematic diagram illustrating an example in which the insertion portion of the endoscope of FIG. 1 is operated from the position illustrated in FIG. 8 in a direction in which occlusion exists.

  As shown in FIG. 1, the endoscope system 1 includes a medical observation device 2 that captures an image of a subject and outputs a two-dimensional image of the image of the subject, a personal computer, and the like, and the medical observation device 2. From the medical image processing device 3 that performs image processing on the video signal of the two-dimensional image output from the image signal and outputs the image signal after the image processing as an image signal, and the medical image processing device 3 And a monitor 4 that displays an image based on the output image signal.

  In addition, the medical observation apparatus 2 is inserted into a body cavity, images the subject in the body cavity and outputs the image as an imaging signal, and illuminates the subject imaged by the endoscope 6. The light source device 7 that supplies the illumination light and the camera that performs various controls on the endoscope 6, performs signal processing on the imaging signal output from the endoscope 6, and outputs it as a video signal of a two-dimensional image A control unit (hereinafter abbreviated as CCU) 8, and a monitor 9 that displays an image of a subject imaged by the endoscope 6 based on a video signal of a two-dimensional image output from the CCU 8. The main part is configured.

  The endoscope 6 as a medical imaging apparatus includes an insertion unit 11 that is inserted into a body cavity and an operation unit 12 that is provided on the proximal end side of the insertion unit 11. A light guide 13 for transmitting illumination light supplied from the light source device 7 is inserted into a portion from the proximal end side in the insertion portion 11 to the distal end portion 14 on the distal end side in the insertion portion 11. Yes.

  The light guide 13 has a distal end side disposed at the distal end portion 14 of the endoscope 6 and a rear end side connected to the light source device 7. Since the light guide 13 has such a configuration, the illumination light supplied from the light source device 7 is transmitted by the light guide 13 and then provided on the distal end surface of the distal end portion 14 of the insertion portion 11 (not shown). It is emitted from the illumination window. Then, illumination light is emitted from an illumination window (not shown) to illuminate a living tissue or the like as a subject.

  At the distal end portion 14 of the endoscope 6, an objective optical system 15 attached to an observation window (not shown) adjacent to an illumination window (not shown) and an imaging position of the objective optical system 15 are arranged. An image pickup unit 17 having an image pickup element 16 constituted by (element) or the like is provided. With such a configuration, the subject image formed by the objective optical system 15 is captured by the image sensor 16 and then output as an image signal.

  The image sensor 16 is connected to the CCU 8 via a signal line. The image sensor 16 is driven based on a drive signal output from the CCU 8 and outputs an image signal to the CCU 8.

  Further, the image pickup signal input to the CCU 8 is subjected to signal processing in a signal processing circuit (not shown) provided inside the CCU 8, thereby being converted and output as a video signal of a two-dimensional image. The video signal of the two-dimensional image output from the CCU 8 is output to the monitor 9 and the medical image processing apparatus 3. As a result, an image of the subject based on the video signal output from the CCU 8 is displayed on the monitor 9 as a two-dimensional image.

  The medical image processing device 3 performs an A / D conversion on the video signal of the two-dimensional image output from the medical observation device 2 and outputs the video output from the image input unit 21. CPU 22 as a central processing unit that performs image processing on a signal, a processing program storage unit 23 in which a processing program related to the image processing is written, and an image that stores a video signal output from the image input unit 21 It has the memory | storage part 24 and the information storage part 25 which memorize | stores the image data etc. as an image processing result of CPU22.

  Further, the medical image processing apparatus 3 stores a storage device interface 26, image data and the like as image processing results of the CPU 22 via the storage device interface 26, a hard disk 27 as a storage device, and image processing results of the CPU 22. The CPU 22 performs a display process for displaying the image data on the monitor 4 based on the image data, and a display processing unit 28 that outputs the image data after the display process as an image signal. An input operation unit 29 configured by a keyboard or the like that allows a user to input parameters for image processing and operation instructions for the medical image processing apparatus 3; The monitor 4 displays an image based on the image signal output from the display processing unit 28.

  Note that the image input unit 21, CPU 22, processing program storage unit 23, image storage unit 24, information storage unit 25, storage device interface 26, display processing unit 28, and input operation unit 29 of the medical image processing apparatus 3 are connected to a data bus. 30 to each other.

  Next, the operation of the endoscope system 1 will be described.

  First, the user inserts the insertion portion 11 of the endoscope 6 into the body cavity. When the insertion portion 11 is inserted into the body cavity by the user, for example, an image of the living tissue 31A that is a raised (having a raised shape) lesion site on the inner wall of the tubular organ 31 in the body cavity. Then, an image as shown in FIG. 2 is picked up by the image pickup unit 17 provided at the tip portion 14. Then, the images of the tubular organ 31 and the living tissue 31A captured by the imaging unit 17 as an image as illustrated in FIG. 2 are output to the CCU 8 as an imaging signal.

  The CCU 8 converts the imaging signal as a video signal of a two-dimensional image by performing signal processing on the imaging signal output from the imaging device 16 of the imaging unit 17 in a signal processing circuit (not shown). The monitor 9 displays the images of the tubular organ 31 and the living tissue 31A as, for example, a two-dimensional image as shown in FIG. 2 based on the video signal output from the CCU 8. Further, the CCU 8 outputs a video signal of a two-dimensional image obtained by performing signal processing on the imaging signal output from the imaging element 16 of the imaging unit 17 to the medical image processing apparatus 3.

  The video signal of the two-dimensional image output to the medical image processing apparatus 3 is A / D converted by the image input unit 21 and then input to the CPU 22.

  Based on the video signal of the two-dimensional image output from the image input unit 21, the CPU 22 performs, for example, processing described below as each processing illustrated in FIG. 3, thereby starting from the first two-dimensional image as the target image. The occlusion area in the estimated 3D model is detected.

  First, the CPU 22 as an edge extraction unit applies a bandpass filter to the red component of the first two-dimensional image, for example, based on the video signal of the first two-dimensional image output from the image input unit 21. As a result, all edges included in the first two-dimensional image are extracted (step S1 in FIG. 3).

  Next, the CPU 22 connects both end points of the edge that matches a predetermined condition among the edges extracted from the first two-dimensional image to form a closed curve (step S2 in FIG. 3). Specifically, for example, the distance between both end points of one edge among the edges extracted from the first two-dimensional image is less than 65% with respect to the line segment length of the one edge. In this case, both end points of the one edge are connected by a straight line, and the one edge is a closed curve.

  The CPU 22 detects whether or not a closed curve exists in the first two-dimensional image after the process of step S2 in FIG. 3 is performed. When the CPU 22 detects that no closed curve exists in the first two-dimensional image after the process of step S2 in FIG. 3 is performed (step S3 in FIG. 3), the first two-dimensional image Is determined not to have a raised lesion that causes occlusion, and the series of processes shown in FIG. 3 is terminated. Further, when the CPU 22 detects that a closed curve exists in the first two-dimensional image after the processing of step S2 in FIG. 3 is performed (step S3 in FIG. 3), the first two-dimensional image It is determined that there is a raised lesion that causes occlusion, and the process shown in step S4 of FIG.

  Thereafter, the CPU 22 serving as the image area detection unit detects the center-of-gravity coordinates G1 (x, y) in the area surrounded by the closed curve, which is present in the first two-dimensional image, and calculates the peripheral length L1 of the closed curve. Calculate (step S4 in FIG. 3). Then, the CPU 22 determines whether or not the value of the peripheral length L1 is equal to or less than a predetermined value.

  Then, the CPU 22 determines a closed curve whose peripheral length L1 is equal to or less than a predetermined first value among the closed curves existing in the first two-dimensional image (determining that there is a high possibility of being generated by noise or the like). (Step S5 in FIG. 3).

  The CPU 22 calculates a brightness component (for example, a luminance value of R) in each of the pixels that are present on opposite sides of the edge as a part of the closed curve and that are adjacent to the edge (in FIG. 3, for example). Step S6). Further, the CPU 22 compares the brightness components of pixels existing on both sides of the edge.

  As a result, the CPU 22 determines that the difference in the brightness component between the pixels located on opposite sides of the edge as part of the closed curve is smaller than a predetermined threshold value (not to cause occlusion). )) Is excluded from the target of subsequent processing (step S7 in FIG. 3).

  The CPU 22 as the occlusion area detection unit detects an area where the difference in brightness components between pixels located on opposite sides across the edge as a part of the closed curve is equal to or greater than a predetermined threshold as an occlusion area. Thereafter, it is further determined whether another closed curve (or edge) exists in each closed curve in which the occlusion region is detected.

  When the CPU 22 detects that another closed curve (or edge) exists within one closed curve (step S8 in FIG. 3), the CPU 22 performs the above-described step in FIG. 3 for the other closed curve (or edge). The process of S6 and step S7 is performed again. When the CPU 22 detects that another closed curve (or edge) does not exist within one closed curve (step S8 in FIG. 3), the CPU 22 continues the process shown in step S9 in FIG.

  Here, step S8 in FIG. 3 will be described.

  For example, in a two-dimensional image in which a raised lesion site 31B is imaged in a state as shown in FIG. 4, another closed curve or edge may be generated within one closed curve only by artifacts such as halation. . However, in a two-dimensional image in which a specially shaped lesion site 31C having both bulging and ulcerative properties is imaged in a state as shown in FIG. There may be another closed curve or edge within the closed curve, and for example, occlusion may occur in a recessed portion as shown in FIG. That is, the CPU 22 detects the occlusion that may occur in the recessed portion as shown in FIG. 5 by performing the processing in step S6 and step S7 in FIG. 3 again according to the processing result in step S8 in FIG. ing.

  In the following, for the sake of simplicity of explanation, among the closed curves existing in the first two-dimensional image, one closed curve and one edge as a part of the one closed curve will be mainly described. To do. In the following, a pixel having a relatively large brightness component (or an inner side of the edge) among pixels adjacent to the one edge and located on opposite sides of the one edge. Pixel) is represented as occlusion start coordinates, and a pixel having a relatively small brightness component (or a pixel outside the edge) is represented as occlusion end coordinates. Further, in the following, for the sake of simplicity, one occlusion end coordinate O1 (x0, y0) is mainly described among the respective occlusion end coordinates of the one edge constituting the one closed curve. . The positional relationship between one edge as a part of one closed curve, the center-of-gravity coordinates G1 (x, y), and one occlusion end coordinate O1 (x0, y0) is, for example, as shown in FIG. It will be something.

  The CPU 22 detects in which direction the one occlusion end coordinate O1 (x0, y0) is present in the first two-dimensional image with respect to the center-of-gravity coordinate G1 (x, y), and determines the direction as the occlusion direction. (Step S9 in FIG. 3).

  The CPU 22 as the three-dimensional model estimation unit uses, for example, a process such as geometric conversion based on the luminance information of the video signal of the first two-dimensional image output from the image input unit 21 and uses the tubular organ 31. The three-dimensional model of the living tissue 31A is estimated (step S10 in FIG. 3). By this processing, the CPU 22 as the occlusion area detection unit corresponds to the occlusion start voxel that is a voxel of the three-dimensional model corresponding to the occlusion start coordinates in the first two-dimensional image and the occlusion end coordinates in the first two-dimensional image. Data with the end-of-occlusion voxel that is the voxel of the three-dimensional model to be acquired is acquired. Then, the CPU 22 detects the two voxels of the occlusion start voxel and the occlusion end voxel and each voxel existing in the region between the two voxels in the three-dimensional model estimated from the first two-dimensional image. (Step S11 in FIG. 3).

  The CPU 22 detects the occlusion area in the three-dimensional model estimated from the first two-dimensional image by performing each process shown in FIG. 3 described above.

  Note that the center-of-gravity coordinates G1, the occlusion start coordinates, and the occlusion end coordinates in the processing of FIG. 3 are not limited to those determined according to individual pixels, and include, for example, (N × N) pixels. It may be determined according to each area to be processed.

  The CPU 22 is an image of a frame located in time series after the first two-dimensional image by performing processing described below as each processing shown in FIG. 7 following the processing shown in FIG. 3, for example. The occlusion region in the three-dimensional model estimated from the first two-dimensional image is complemented while using the three-dimensional model estimated from the second two-dimensional image as the non-target image.

  First, the CPU 22 outputs, for example, the second two-dimensional image based on the video signal of the second two-dimensional image that is output from the image input unit 21 and is a frame image after the first two-dimensional image. By applying a band pass filter to the red component, all edges included in the second two-dimensional image are extracted (step S21 in FIG. 7).

  Then, the CPU 22 connects both end points of the edge that matches the predetermined condition among the edges extracted from the second two-dimensional image to form a closed curve (step S22 in FIG. 7). Specifically, for example, the distance between both end points of one edge among the edges extracted from the second two-dimensional image is less than 65% with respect to the line segment length of the one edge. In this case, both end points of the one edge are connected by a straight line, and the one edge is a closed curve.

  The CPU 22 detects whether or not a closed curve exists in the second two-dimensional image after the process of step S2 of FIG. 3 is performed. When the CPU 22 detects that no closed curve exists in the second two-dimensional image after the processing of step S2 in FIG. 3 is performed (step S23 in FIG. 7), the second two-dimensional image If the image is not the final frame image, the processing of step S21 and subsequent steps in FIG. 7 is performed on the next frame image, and if the second two-dimensional image is the final frame image, it is shown in FIG. A series of processing ends (steps S24 and S25 in FIG. 7).

  Further, when the CPU 22 detects that a closed curve exists in the second two-dimensional image after the process of step S2 of FIG. 3 is performed (step S23 of FIG. 7), the process is shown in step S4 of FIG. The center of gravity coordinates G2 (X, Y) of the closed curve are detected by the same process as the process, and the circumference L2 of the closed curve is calculated (step S26 in FIG. 7).

  Thereafter, the CPU 22 serving as the image region movement detection unit has a predetermined distance from the center-of-gravity coordinates G1 (x, y) in the first two-dimensional image to the center-of-gravity coordinates G2 (X, Y) in the second two-dimensional image. The first closed curve having one occlusion end coordinate O1 (x0, y0) is detected by detecting whether or not the peripheral length L1 and the peripheral length L2 are substantially the same value. It is determined whether one closed curve in the two-dimensional image exists also in the second two-dimensional image.

  The CPU 22 determines that the distance from the center of gravity coordinate G1 (x, y) to the center of gravity coordinate G2 (X, Y) is longer than a predetermined distance, or that the peripheral length L1 and the peripheral length L2 are not substantially the same value. (Step S27 in FIG. 7), it is determined that a closed curve having one occlusion end coordinate O1 (x0, y0) does not exist in the second two-dimensional image. Thereafter, if the second two-dimensional image having the center-of-gravity coordinates G2 (X, Y) is not the image of the final frame, the CPU 22 performs the processing from step S21 onward in FIG. If the second two-dimensional image is the image of the final frame, the series of processes shown in FIG. 7 is terminated (Step S24 and Step S25 in FIG. 7).

  Further, the CPU 22 determines that the distance from the centroid coordinate G1 (x, y) in the first two-dimensional image to the centroid coordinate G2 (X, Y) in the second two-dimensional image is shorter than a predetermined distance, and When it is detected that the length L1 and the peripheral length L2 are substantially the same value (step S27 in FIG. 7), the closed curve having the centroid coordinates G2 (X, Y) and the peripheral length L2 is one occlusion end coordinate O1. It is determined that the closed curve suggests the presence of (x0, y0).

  Thereafter, the CPU 22 as the image region movement detection unit determines that the moving direction from the center of gravity coordinate G1 (x, y) to the center of gravity coordinate G2 (X, Y) is opposite to the occlusion direction calculated in the process of step S9 in FIG. It is determined whether or not. Specifically, the CPU 22 sets the coordinates of the center of gravity when the coordinates obtained by moving the one occlusion end coordinate O1 (x0, y0) to a position symmetrical to the center of gravity coordinates G1 (x, y) are O2 (X0, Y0). If the angle θ formed by G2, the centroid coordinates G1, and the coordinates O2 is within a predetermined angle (for example, 60 degrees), the movement direction from the centroid coordinates G1 (x, y) to the centroid coordinates G2 (X, Y) is Then, it is detected that the direction is opposite to the occlusion direction calculated in the process of step S9 in FIG.

  When the CPU 22 detects that the moving direction from the center-of-gravity coordinates G1 to the center-of-gravity coordinates G2 is not the opposite direction to the occlusion direction calculated in the process of step S9 in FIG. 3 (step S28 in FIG. 7), the first And determining that the data for complementing the occlusion region in the three-dimensional model estimated from the two-dimensional image cannot be obtained from the second two-dimensional image, and has the second center of gravity coordinates G2 (X, Y) If the two-dimensional image is not the final frame image, the image of the next frame is subjected to the processing from step S21 onward in FIG. 7, and if the second two-dimensional image is the final frame image, 7 is terminated (step S24 and step S25 in FIG. 7).

  Further, when the CPU 22 detects that the moving direction from the center-of-gravity coordinates G1 to the center-of-gravity coordinates G2 is opposite to the occlusion direction calculated in the process of step S9 in FIG. 3 (step S28 in FIG. 7), the first It is determined that data for complementing the occlusion region in the three-dimensional model estimated from the two-dimensional image is obtained from the second two-dimensional image. Thereafter, the CPU 22 as the three-dimensional model estimation unit uses, for example, a process such as a geometric transformation based on the luminance information of the video signal of the second two-dimensional image output from the image input unit 21 to form a tubular shape. A three-dimensional model of the organ 31 and the living tissue 31A is estimated (step S29 in FIG. 7).

  The CPU 22 as the occlusion complement corresponding region detection unit is based on the three-dimensional model estimated from the second two-dimensional image, and among the data of each voxel included in the occlusion region in the three-dimensional model estimated from the first two-dimensional image, A voxel included in the occlusion complementation corresponding region, which is a region where voxel data that can be complemented by the data of the three-dimensional model estimated from the second two-dimensional image, is detected (step S30 in FIG. 7).

  Here, details of the process shown in step S30 of FIG. 7 will be described.

  The three-dimensional model of the living tissue 31A based on the first two-dimensional image is estimated as having a state as shown in FIG. 8 when viewed from the side with respect to the viewing direction of the endoscope 6, for example. As shown in FIG. 8, in the three-dimensional model of one voxel existing on one edge sandwiched between the occlusion start voxel and the occlusion end voxel, the voxel E as the occlusion end voxel, and the living tissue 31A, When one voxel existing on the same plane as the occlusion side and voxel OE is voxel B1, the occlusion region of the three-dimensional model of the living tissue 31A based on the first two-dimensional image is included from voxel E to voxel B1. And voxels included from voxel B1 to voxel OE.

  Thereafter, when the first two-dimensional image is acquired, the imaging device 16 (imaging surface thereof) at the position C1 is operated by operating the insertion unit 11 of the endoscope 6 as shown in FIG. When acquiring the second two-dimensional image, it is moved to the position C2. Also, as shown in FIG. 9, when a straight line is drawn from the position C2 via the voxel E toward the occlusion area, the voxel OE exists in the voxel OC included in the occlusion area. Intersect with a plane.

  For example, when an image of 60 frames per second is acquired by the endoscope 6, the difference between the first two-dimensional image and the second two-dimensional image is small. The distance (the distance from the position C1 to the position C2) that the imaging device 16 (the imaging surface) has moved from the time of image acquisition to the time of the second two-dimensional image acquisition, and the movement distance from the center of gravity coordinates G1 to the center of gravity coordinates G2 The assumption that they are equal holds. In addition, for example, when an image for 60 frames per second is acquired by the endoscope 6, the difference between the first two-dimensional image and the second two-dimensional image is small. The distance (the distance from the position C1 to the position C2) that the imaging element 16 (the imaging surface thereof) has moved from the time when the two-dimensional image is acquired to the time when the second two-dimensional image is acquired, and the distance corresponding to the occlusion complement corresponding region The assumption that they are equal holds. Furthermore, when an image of 60 frames per second is acquired by the endoscope 6, for example, since the difference between the first two-dimensional image and the second two-dimensional image is small, the position C1 and It is assumed that the position C2 is on the same plane.

  It should be noted that the above assumptions hold in terms of geometrically, among the parts shown in FIG. 9, a triangle formed by connecting three points of position C1, position C2, and voxel E by a straight line, voxel OE, voxel OC, and voxel. The triangle in which E is connected by a straight line is indicated by being congruent triangles formed by diagonal lines of parallelograms connecting the position C1, the position C2, the voxel OE, and the voxel OC by a straight line.

  Since each of the assumptions described above holds, the movement distance from the center-of-gravity coordinate G1 to the center-of-gravity coordinate G2 is equal to the distance from the voxel OE to the voxel OC. Therefore, the CPU 22 has two voxels VO and OC. And each voxel included between the two voxels is detected as a voxel included in the occlusion complementation corresponding region.

  Then, the CPU 22 as the occlusion complementation unit uses the data of each voxel included in the occlusion complementation corresponding area detected by the process shown in step S30 of FIG. 7, and uses the three-dimensional model of the living tissue 31A based on the first two-dimensional image. The data of the occlusion area at is supplemented (step S31 in FIG. 7). Specifically, the CPU 22 supports occlusion completion among the data of each voxel included in the occlusion area as a process of complementing the data of the occlusion area in the three-dimensional model of the living tissue 31A based on the first two-dimensional image. A process of replacing the data of each voxel included in the region with the data of the voxel included in the three-dimensional model of the living tissue 31A based on the second two-dimensional image is performed.

  By the process shown in step S31 of FIG. 7, the CPU 22 uses the occlusion region voxel data in the three-dimensional model of the living tissue 31A based on the first two-dimensional image as the tertiary of the living tissue 31A based on the second two-dimensional image. After complementing based on voxel data of the original model, it is further determined whether or not the occlusion area remains. The CPU 22 still has an occlusion area in the three-dimensional model of the living tissue 31A based on the first two-dimensional image (step S32 in FIG. 7), and the first used for the detection of the above-described occlusion complementation corresponding area. If the 2D image of 2 is not the image of the last frame, the remaining occlusion area is complemented by performing the processing of step S21 and subsequent steps in FIG. 7 on the image of the next frame of the second 2D image. Is performed (step S24 and step S25 in FIG. 7). Further, the CPU 22 still has an occlusion area in the three-dimensional model of the living tissue 31A based on the first two-dimensional image (step S32 in FIG. 7), and the first used for the detection of the occlusion complementation corresponding area described above. If the two-dimensional image 2 is the image of the final frame, the series of processes shown in FIG.

  Further, when the CPU 22 detects that no occlusion region remains in the three-dimensional model of the living tissue 31A based on the first two-dimensional image (step S32 in FIG. 7), the series of processing illustrated in FIG. 7 is terminated. .

  The center-of-gravity coordinates G2 and the occlusion end coordinates in the processing of FIG. 7 are not limited to those determined according to individual pixels. For example, each of the regions configured to have (N × N) pixels It may be determined according to.

  Further, in the process shown in step S32 in FIG. 7, the CPU 22 performs each process after step S21 in FIG. 7 until the occlusion region in the three-dimensional model of the living tissue 31A based on the first two-dimensional image is completely complemented. For example, the processing may be terminated when it is detected that the original occlusion area detected in each processing of FIG. 3 is complemented by a predetermined ratio or more. Thereby, the medical image processing apparatus 3 can perform the process for complementing the occlusion area at higher speed.

  Further, the CPU 22 has a first occlusion area included in the three-dimensional model estimated from the two-dimensional image of the first frame, and two second frames that are temporally continuous to the first frame. When the first occlusion area is smaller than the second occlusion area while performing the process of comparing the second occlusion area of the three-dimensional model estimated from the three-dimensional image in addition to the process of FIG. Alternatively, the processing shown in FIG. 7 may be performed on the three-dimensional model estimated from the two-dimensional image of the second frame. Thereby, the medical image processing apparatus 3 can perform the process for complementing the occlusion area at higher speed and higher accuracy.

  In addition, the process for complementing the occlusion area in the present embodiment is based on a non-target image based on a non-target image located in time series after the target image as described in the series of processes shown in FIG. For example, if the image includes at least a part of the occlusion region to be complemented, the occlusion region is used while using a non-target image positioned in time series from the target image. It may be complementary.

  As described above, when the 3D model is estimated based on the 2D image, the medical image processing apparatus 3 according to the present embodiment detects the missing data and the location where data with low reliability exists. By reducing the amount compared to the conventional case, it is possible to estimate a highly accurate three-dimensional model, and as a result, it is possible to easily find a lesion site such as a polyp.

  In addition, this invention is not limited to embodiment mentioned above, Of course, a various change and application are possible in the range which does not deviate from the meaning of invention.

The figure which shows an example of the whole structure of the endoscope system in which the medical image processing apparatus which concerns on embodiment of this invention is used. The schematic diagram which shows the image of the tubular organ and biological tissue imaged with the endoscope of FIG. The flowchart which shows an example of the process for detecting the occlusion area | region in the three-dimensional model estimated from the 1st two-dimensional image which is a process which the medical image processing apparatus of FIG. 1 performs. The schematic diagram which shows an example of the protruding lesion part which can be imaged with the endoscope of FIG. The schematic diagram which shows the example different from FIG. 4 of the protruding lesion part which can be imaged with the endoscope of FIG. The figure which shows the positional relationship of one edge, the gravity center coordinate G1, and the one occlusion end coordinate O1 in the 1st two-dimensional image which the medical image processing apparatus of FIG. 1 acquired. The flowchart which shows an example of the process for complementing the occlusion area | region in the three-dimensional model estimated from the 1st two-dimensional image which is the process which the medical image processing apparatus of FIG. 1 performs. The schematic diagram which shows an example at the time of seeing the three-dimensional model based on a 1st two-dimensional image from the side with respect to the visual field direction of the endoscope of FIG. The schematic diagram which shows an example when the insertion part of the endoscope of FIG. 1 is operated to the direction where an occlusion exists from the position shown in FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Endoscope system, 2 ... Medical observation apparatus, 3 ... Medical image processing apparatus, 4, 9 ... Monitor, 6 ... Endoscope, 7 ... Light source device 8 ... CCU, 11 ... insertion part, 12 ... operation part, 13 ... light guide, 14 ... tip part, 15 ... objective optical system, 16 ... imaging device, 17 ... Imaging unit, 21 ... Image input unit, 22 ... CPU, 23 ... Processing program storage unit, 24 ... Image storage unit, 25 ... Information storage unit, 26 ... Storage device interface, 27... Hard disk, 28... Display processing unit, 29... Input operation unit, 30... Data bus, 31 ... tubular organ, 31 A ... biological tissue, 31 B, 31 C ... Lesion site

Claims (12)

An occlusion area detection unit that detects an occlusion area of a target image among images of biological tissue input from a medical imaging device;
Using information obtained from one or a plurality of non-target images, which is an image different from the target image and is an image in which at least a part of a region corresponding to the occlusion region of the target image exists. An occlusion complementing unit that complements the occlusion region of the target image;
A medical image processing apparatus comprising:   The medical image processing apparatus according to claim 1, wherein each of the one or more non-target images is a two-dimensional image or a three-dimensional model generated based on the two-dimensional image.   The medical image processing apparatus according to claim 1, wherein each of the one or more non-target images is an image of a frame positioned before or after the target image in time series. A first three-dimensional model estimation unit that estimates a three-dimensional model of the living tissue based on the target image as a two-dimensional image;
A second three-dimensional model estimation unit that estimates a three-dimensional model of each of the one or more non-target images,
The occlusion area detection unit detects a first occlusion area detection unit that detects an occlusion area of the target image as a two-dimensional image, and detects each voxel corresponding to the occlusion area in a three-dimensional model of the target image. Two occlusion area detectors,
The occlusion complementation unit supplements the data of each voxel detected by the second occlusion region detection unit based on the data of the 3D model estimated by the second 3D model estimation unit. The medical image processing apparatus according to any one of claims 1 to 3. Furthermore, it has an edge extraction unit for extracting the edge of the target image,
The medical image according to claim 4, wherein the first occlusion area detection unit detects the occlusion area based on a difference in brightness components between areas existing on both sides of the edge. Processing equipment. Further, in the target image, an image area detection unit that detects a predetermined area existing at a position corresponding to the occlusion area;
Based on the position where the predetermined area exists in the target image and the position where the predetermined area exists in one non-target image among the one or more non-target images, the predetermined area is two. An image region movement detection unit for detecting a distance and direction moved in the three-dimensional image;
Based on the distance and the direction in which the predetermined area has moved in the two-dimensional image, an occlusion is performed among areas of the voxel data detected by the second occlusion area detector that includes complementary voxel data. An occlusion complementary corresponding area detection unit that detects as a complementary corresponding area,
The occlusion complementation unit is based on the three-dimensional model of the non-target image estimated by the second three-dimensional model estimation unit, and among the data of each voxel detected by the second occlusion region detection unit, 6. The medical image processing apparatus according to claim 4, wherein voxel data existing in the occlusion complementation corresponding region is supplemented. An occlusion area detection step for detecting an occlusion area of a target image among images of biological tissue input from a medical imaging device;
Using information obtained from one or a plurality of non-target images, which is an image different from the target image and is an image in which at least a part of a region corresponding to the occlusion region of the target image exists. An occlusion completion step of complementing the occlusion area of the target image;
A medical image processing method comprising:   The medical image processing method according to claim 7, wherein each of the one or more non-target images is a two-dimensional image or a three-dimensional model generated based on the two-dimensional image.   9. The medical image processing method according to claim 7, wherein each of the one or more non-target images is an image of a frame positioned before or after the target image in time series. A first three-dimensional model estimation step for estimating a three-dimensional model of the living tissue based on the target image as a two-dimensional image;
A second three-dimensional model estimation step for estimating a three-dimensional model of each of the one or more non-target images,
The occlusion area detecting step includes a first occlusion area detecting step for detecting an occlusion area of the target image as a two-dimensional image, and a first occlusion area detecting step for detecting each voxel corresponding to the occlusion area in the three-dimensional model of the target image. Two occlusion area detection steps,
The occlusion complementation step supplements the data of each voxel detected by the second occlusion region detection step based on the data of the three-dimensional model estimated by the second three-dimensional model estimation step. A medical image processing method according to any one of claims 7 to 9. And an edge extraction step for extracting an edge of the target image,
The medical image according to claim 10, wherein the first occlusion area detection step detects the occlusion area based on a difference in brightness components between areas existing on both sides of the edge. Processing method. Further, in the target image, an image region detection step for detecting a predetermined region existing at a position corresponding to the occlusion region;
Based on the position where the predetermined area exists in the target image and the position where the predetermined area exists in one non-target image among the one or more non-target images, the predetermined area is two. An image region movement detection step for detecting the distance and direction moved in the three-dimensional image;
Based on the distance and the direction in which the predetermined area has moved in the two-dimensional image, an occlusion is performed among areas of the voxel data detected by the second occlusion area detector that includes complementary voxel data. An occlusion complementary area detection step for detecting as a complementary area,
The occlusion complementation step is based on the three-dimensional model of the non-target image estimated by the second three-dimensional model estimation step, and among the data of each voxel detected by the second occlusion region detection step, The medical image processing method according to claim 10 or 11, wherein voxel data existing in the occlusion complementation corresponding region is complemented.

Images