JP6671747B2 - Medical image processing apparatus, control method thereof, and program - Google Patents

Description

  The present invention relates to a medical image processing apparatus, a control method thereof, and a program.

  Due to the high mortality from colorectal cancer, various screenings have been planned. Endoscopy is a typical example. In recent years, use of a capsule endoscope with less burden on a subject or use of a virtual endoscope image generated from a medical image obtained by colorectal CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) for a medical examination. Has become possible.

  The capsule endoscope is a type in which a subject swallows a capsule containing a small camera, and continues imaging by the camera while moving inside the body. Therefore, antennas for detecting the position and orientation of the capsule in the body and receiving an image captured by the capsule are arranged at a plurality of preset positions of the clothes worn by the patient. Then, the subject wears an extracorporeal device that is connected to the antenna and records the image captured by the capsule and the position and orientation of the capsule (Patent Document 1).

  Also, there is known a technique of displaying a capsule endoscope image corresponding to a viewpoint position and a line of sight of an intestinal lumen image (virtual endoscopic image by X-ray CT) using the position and orientation of the capsule endoscope. (Patent Document 2). The virtual endoscopic image generated by CT, for example, can check the presence or absence of a polyp, but in order to diagnose whether the polyp is malignant, an actually captured image is required. is there.

JP-A-2007-608 JP 2013-9956 A

  As shown in Patent Document 2, when a capsule endoscope image corresponding to a viewpoint position and a line of sight of an intestinal lumen image is displayed using the position and orientation of the capsule endoscope, the capsule endoscope image can be obtained by a large intestine CT examination. It is necessary to generate a three-dimensional large intestine model from a CT image, and to specify at which position on the three-dimensional large intestine model the capsule endoscope image has been captured.

  However, since a large intestine is inflated by injecting gas or the like into the large intestine of the patient in the large intestine CT examination, the large intestine when the capsule endoscope is moved and the large intestine when the large intestine CT examination is performed. Then, their lengths differ greatly. Therefore, there is a problem that it is difficult to accurately map (align) the imaging position of the image captured by the capsule endoscope and the position on the colon line of the three-dimensional colon model.

  The present invention has been made in view of such a problem, and has a sufficient positional relationship between an actual captured image captured by a capsule endoscope and a tubular structure included in a plurality of medical images obtained by a medical image diagnostic apparatus. It is intended to provide a technique capable of estimating with high accuracy.

In order to solve this problem, for example, an image diagnosis support apparatus of the present invention has the following configuration. That is, a medical image processing apparatus capable of displaying a virtual endoscopic image of a tubular structure included in a plurality of medical images obtained by a medical image diagnostic apparatus, comprising: an obtaining unit, a first specifying unit, A second specifying unit, a third specifying unit, and a notifying unit. The acquisition unit includes a plurality of capsule endoscope image information including a capsule endoscope image captured by a capsule endoscope and coordinate information indicating an imaging position of the capsule endoscope image; Acquire a medical image . The first specifying unit is configured to determine a first trajectory corresponding to the shape of the large intestine from a movement trajectory of the capsule endoscope configured by the imaging positions included in the plurality of capsule endoscope image information acquired by the acquisition unit. Identify the section . The second specifying unit specifies a second section corresponding to the first section according to the shape of the large intestine from a core line of the large intestine included in the plurality of medical images acquired by the acquiring unit . The third specific means, said first and journey across points of the identified said first interval by a specific means, between end points of said identified in the second identifying unit the second section The position on the center line of the large intestine , which corresponds to the imaging position of the capsule endoscope image captured in the first section, is specified using the parameter based on the distance . The notifying means, after the position on the center line of the large intestine corresponding to the imaging position of the capsule endoscope image is specified, moves the virtual line-of-sight position on the center line of the large intestine with a virtual endoscope. When going, among the plurality of capsule endoscope images, when the difference between the imaging position of the key image set as an image of interest to the user and the virtual gaze position is equal to or less than a threshold, the virtual gaze The user is notified that the position has approached the imaging position of the key image.

  ADVANTAGE OF THE INVENTION According to this invention, the positional relationship of the real imaging | photography image imaged by the capsule endoscope with respect to the tubular structure contained in the some medical image obtained by the medical image diagnostic apparatus can be estimated with sufficient precision.

It is a block configuration diagram of an image diagnosis eye gaze device in an embodiment. FIG. 4 is a diagram illustrating an example of a management table generated in a process according to the embodiment. FIG. 6 is a diagram showing an example of a GUI for setting a landmark with respect to a movement locus of a capsule endoscope. FIG. 11 is a diagram illustrating an example of a GUI for setting a landmark for a three-dimensional colon model. It is a figure which shows the large intestine core line obtained from the three-dimensional large intestine model. FIG. 11 is a diagram illustrating an example of a GUI for selecting a key image. FIG. 3 is a diagram showing a relationship between the center of an image captured by a capsule endoscope and the inner wall of the large intestine. FIG. 8 is a diagram illustrating an example of a GUI when functioning as a virtual endoscope. FIG. 8 is a diagram illustrating an example of a GUI when functioning as a virtual endoscope. It is a flowchart which shows the processing procedure of the image diagnosis support apparatus of embodiment. It is a flowchart which shows a virtual endoscope browsing process. FIG. 4 is a diagram illustrating an example of a data structure of capsule endoscope image information. It is a flowchart which shows the detailed processing procedure which determines the imaging position of the key image on a core line. It is a figure which shows the conceptual diagram at the time of dividing the movement path of a colon endoscope and a capsule endoscope.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Device configuration]
An example in which the image diagnosis support apparatus (medical image processing apparatus) according to the embodiment is realized by an information processing apparatus represented by a personal computer and an application program operating on the information processing apparatus will be described.

  FIG. 1 is a block diagram of the information processing apparatus 100 according to the embodiment. The information processing apparatus 100 includes a CPU 101, a ROM that stores a BIOS and a boot program, a RAM 103 that stores an OS (Operating System) of the CPU 101 and various applications, and that is used as a work area of the CPU 101. Further, the information processing apparatus 100 includes an HDD (hard disk drive) 104 as an external storage device, an I / F 105 for communicating with an external device, a display unit 106 and an operation unit 107 functioning as a user interface.

  The HDD 104 stores an OS, applications for the apparatus to function as an image diagnosis support apparatus, and various data files. The I / F 105 is, for example, a network interface, and can communicate with an X-ray CT device (not shown) or a capsule endoscope device. The X-ray CT apparatus and the capsule endoscope are assumed to be connected via a network. However, it is only necessary that this apparatus can acquire the information obtained thereby. It does not need to be connected. For example, the I / F 105 may be an interface that connects a storage medium that stores information obtained by scanning with an X-ray CT apparatus and information obtained with a capsule endoscope apparatus. The display unit 106, the liquid crystal display, and the operation unit 107 capable of displaying various information are configured by a mouse and a keyboard. However, the operation unit may be a touch panel.

  In the configuration described above, the capsule endoscope image information 110 obtained from the capsule endoscope apparatus and the CT image information 120 (medical image data) captured by the X-ray CT apparatus are already stored in the HDD 104. It will be described as.

[Mapping of the movement trajectory of the colon capsule endoscope to the 3D colon model]
FIG. 12 shows an example of the capsule endoscope image information 110. This information includes a large number of image files captured by the capsule endoscope apparatus and a management table for managing each image file. The management table includes an elapsed time from the start of imaging by the capsule endoscope, an imaging position (position in the body of the capsule endoscope, coordinate information), an imaging direction (a line of sight of the camera of the capsule endoscope), and an image. Consists of a file name field. Generally, the imaging interval of the capsule endoscope is about two or three images per second, but for simplicity of description, the capsule endoscope employed in the embodiment is two images / second (the imaging time interval is 0.5 Second). In the elapsed time field, a value indicating the number of images after the start of imaging is stored. Assuming that the target image is the Nth image, the target image can be expressed as N / 2 seconds after the start of imaging. As the imaging position and the imaging direction, information based on a preset position of the subject is stored. As the image file name, the file name of the corresponding image is stored literally. The path to the image may be included in the image file name. In this case, only the management table may be stored in the HDD 104 as the capsule endoscope image information 110, and the actual image file may exist on the network. Although the type of encoding of the image file does not matter, it is assumed that JPEG is used in the embodiment. Note that the configuration of detecting each coordinate position and imaging direction of the capsule endoscope and the configuration of wirelessly receiving an image from the capsule endoscope are well-known, and thus detailed description thereof will be omitted. The data structure shown in FIG. 2 is only an example, and the present invention is not limited to this data format. For example, a plurality of captured images may be stored as a moving image in one file. In this case, each frame is desirably encoded as an intra frame so that each image constituting the moving image can be independently decoded.

  As described above, since the elapsed time is associated with the position and orientation of the capsule endoscope in the body, the image captured by the capsule endoscope is an image captured at which position in the body and in which gaze direction. Can be determined.

  It should be noted that the examination using the capsule endoscope apparatus and the examination using the colon CT are performed independently. In the large intestine CT, gas (generally, carbon dioxide) is injected into the large intestine of a subject, and a scan process is performed by an X-ray CT apparatus in a state where the large intestine (tubular structure) is expanded from the inside. On the other hand, during the examination by the capsule endoscope, gas is not injected into the large intestine.

  In other words, even for the same subject, the shape of the three-dimensional large intestine model included in the large intestine CT image obtained by large intestine CT and the actual shape of the large intestine when the capsule endoscope passes are different. Therefore, the coordinates indicated by the capsule endoscope image information 110 cannot be converted to a three-dimensional large intestine model as it is.

  In Patent Literature 2, a core line indicating a movement trajectory connecting each coordinate of the capsule endoscope is obtained, and a tangent vector of the core line is obtained. Similarly, a core line of the three-dimensional large intestine model is obtained, and a tangent vector of the core line is obtained. Then, the inner product of the tangent vectors at each point of the two core lines is determined, and the combination that maximizes the sum is determined, thereby associating with the two core lines. When focusing only on the large intestine, even if the capsule endoscope moves in the large intestine for one hour, the coordinates during that time are 7,200, and it is easy to require enormous processing for the above processing. I can understand.

  A first feature of the present embodiment is that the mapping of the movement trajectory of the capsule endoscope with respect to the three-dimensional large intestine model obtained by the large intestine CT is realized with simple processing and with sufficient accuracy. Hereinafter, such a point will be described.

  The large intestine is the part that begins at the Bauhin valve (ileocecal valve) located at the border with the small intestine, and extends to the ascending, transverse, descending, sigmoid, rectum, and anus. With the subject facing forward, the connection between the left end of the transverse colon and the ascending colon has a curved shape regardless of the posture of the subject. Further, the connecting portion between the right end of the transverse colon and the descending colon toward the subject also has a curved shape regardless of the posture of the subject. This is because the transverse colon is suspended in the body at both ends. The curved portion from the ascending colon to the transverse colon is called a liver curved portion because its position is close to the liver. Further, the curved portion from the transverse colon to the descending colon is close to the spleen, and is therefore called a spleen curved portion.

  From the above description, in the embodiment, first, a line segment connecting the time-series three-dimensional coordinates in the management table of the capsule endoscope image information 110 is displayed. For a person engaged in medical treatment, the liver curved portion and the spleen curved portion can be specified simply by displaying the movement trajectory of the capsule endoscope, and the position of the Bauhin valve can be easily specified therefrom. The anus is also easily identifiable because it is the final location of valid coordinates.

  Therefore, in the embodiment, when the CPU 101 executes the image diagnosis support application, the CPU 101 uses the capsule endoscope image information 110 to display a moving trajectory connecting time-series three-dimensional coordinates of the capsule endoscope (a certain degree of smoothing processing). Is desirably applied). FIG. 3 is an example of a display window of the movement trajectory curve of the capsule endoscope displayed on the display unit 106. Since the coordinates of the capsule endoscope are represented in three dimensions, the user can freely change the virtual viewpoint position and display a planar image viewed from the virtual viewpoint position. The technique of displaying the object while freely changing the virtual viewpoint position using the data of the three-dimensional coordinates of the object is well known.

  The user (physician or the like) operates the operation unit 107 while watching the movement trajectory curve of the capsule endoscope displayed on the display unit 106, and determines the positions of the liver bending part, the spleen bending part, the Bauhin valve, and the anus. Will be instructed. In response to this, the CPU 101 displays markers 301 to 304 (triangles in the figure) indicating that the position has been designated at the designated position in the display window. Here, the marker 301 indicates the liver curved portion, the marker 302 indicates the spleen curved portion, the marker 303 indicates the Bauhin valve, and the marker 304 indicates the anus. As a result, in the management table in the capsule endoscope image information 110, the correspondence between the designated four points can be determined.

  In order to maintain this relationship, in the embodiment, two fields of “landmark” and “key image” are added to the management table indicated by the capsule endoscope image information 110. FIG. 2 shows a management table to which these two are added. The landmark field is assigned a number of “1” for a Bauhin valve, “2” for a liver curved portion, “3” for a spleen curved portion, and “4” for an anus. Since the management table is data arranged in chronological order, the record of the landmark “1” (the record with the elapsed time “m1” shown) is changed from the record of the landmark “4” (the record with the elapsed time “m4” shown) ) Can be said to be data on the range of the large intestine of the subject. The “key image” field will be described later.

  Further, the CPU 101 generates a three-dimensional large intestine model from the CT image information 120. In order to generate a three-dimensional large intestine model, it is necessary to specify a Bauhin valve and an anus. These can be realized by reconstructing and displaying the three-dimensional built-in model in the scanned range, and allowing the user to specify the Bauhin valve and the anus for the three-dimensional built-in model. When the Bauhin valve and the anus are designated, the space between them is the large intestine, so that a three-dimensional large intestine model can be generated. Then, the CPU 101 displays the generated three-dimensional large intestine model on the display unit 106. The user can freely browse the displayed three-dimensional large intestine model while changing its virtual viewpoint position. Then, similarly to the movement trajectory of the capsule endoscope described above, the CPU 101 causes the user to specify the liver bending portion and the spleen bending portion. FIG. 4 is a display example of a three-dimensional colon model. Markers indicated by reference numerals 401 and 402 in the drawing are the designated liver curved portion and spleen curved portion. The Bauhin valve has already been determined as 403 and the anus as 404.

  As a result, the positions of the Bauhin valve, the liver curved portion, the spleen curved portion, and the anus with respect to the three-dimensional large intestine model are determined.

  Next, the CPU 101 determines a three-dimensional core line (hereinafter, referred to as a colon core line) passing through a center point of a cross section of the three-dimensional colon in a path from an end point (Bauhin valve) of the three-dimensional colon model to another end point (anus). (It is desirable that the colon core line be smoothed to a certain extent to obtain a smooth to curved line). Then, the CPU 101 determines the position on the identified colon colon line closest to the marker designated by the user as the position of the Bauhin valve 403, the liver bending portion 401, the spleen bending portion 402, and the anus 404. The reason why these positions are specified by the positions on the line is to facilitate association with each part on the movement trajectory curve of the capsule endoscope. FIG. 5 shows the calculated core line of the large intestine, and reference numerals 501 to 504 indicate positions of a liver curved portion, a spleen curved portion, a Bauhin valve, and an anus.

  Now, as described above, when the positions of the Bauhin valve 503, the liver bending portion 501, the spleen bending portion 502, and the anus 504 on the colon line in the three-dimensional large intestine model are defined, capsules are provided at those positions. The positions of the Bauhin valve 303, the liver bending portion 301, the spleen bending portion 302, and the anus 304 on the movement trajectory of the endoscope are associated (mapped).

  In other words, it is considered that the section between the Bauhin valve 503 and the liver bending section 501 in the three-dimensional large intestine model corresponds to the section between the Bauhin valve 303 and the liver bending section 301 on the movement trajectory of the capsule endoscope. Hereinafter, this section is referred to as “section I”. In the example of FIG. 4, it is assumed that the length L (I) of the section I of the colon core line in the three-dimensional colon model is 30 cm. From the management table in FIG. 2, the elapsed time of the capsule endoscope corresponding to the section I is [m1 to m2], and it is assumed that 2,000 images have been captured in this section. In the embodiment, in this section I, it is assumed that the capsule endoscope has moved at a constant speed. That is, it is assumed that the capsule endoscope has moved on the core line of the large intestine and has been imaged at an interval of L (I) / 2000 = 30/2000 cm.

  Similarly, it is assumed that a section from the liver bending section 501 to the spleen bending section 502 corresponds to a section from the liver bending section 301 to the spleen bending section 302 (referred to as the section II). Also, a section between the spleen bending section 502 and the anus 504 and a section between the spleen bending section 302 and the anus 304 are regarded as corresponding (indicated as the same section III).

  As described above, mapping of the position of the colon core in the three-dimensional large intestine model by the large intestine CT and the imaging position of the capsule endoscope (the position of the captured image) are performed. Therefore, thereafter, information indicating the three-dimensional coordinates of the “imaging position” field in the management table of the capsule endoscope image information 110 is not referred to.

  Mapping of the position of the core line of the large intestine in the three-dimensional large intestine model by the large intestine CT and the imaging position of the capsule endoscope (the position of the captured image) is not limited to the above, and may be performed as follows.

  First, similarly to the above-described mapping method, the positions of the Bauhin valve 503, the liver bending portion 501, the spleen bending portion 502, and the anus 504 on the colon line in the three-dimensional large intestine model are specified by markers. Also, regarding the movement trajectory of the capsule endoscope, the Bauhin valve 303, the liver bending portion 301, the spleen bending portion 302, and the anus 304 are specified by the markers, as in the above-described mapping method.

  Then, the large intestine core line in the three-dimensional large intestine model is divided into sections in the same manner as the above-described mapping method. For example, it is assumed that the Bauhin valve 503-liver bend 501 is "section I", the liver bend 501-spleen bend 502 is "section II", the spleen bend 502-anus 504 is "section III". Similarly, the movement trajectory of the capsule endoscope is divided into sections. For example, it is assumed that the Bauhin valve 303 and the liver bending portion 301 are “section I”, the liver bending portion 301 and the spleen bending portion 302 are “section II”, and the spleen bending portion 302 and the anus 304 are “section III”. FIG. 14 shows the result of segmentation using landmarks as a delimiter. FIG. 14 is a conceptual diagram showing a case where the colon colon line and the movement trajectory of the capsule endoscope in the three-dimensional colon model are linearly stretched and divided into sections.

  Then, the ratio of the distance of the section of the colon core line in the three-dimensional large intestine model corresponding to the section to the distance of the section of the movement trajectory of the capsule endoscope is calculated. For example, if the distance of the section I of the colon core line in the three-dimensional colon model is Lx (I) and the distance of the section I of the movement trajectory of the capsule endoscope is Ly (I), the ratio is Ly (I) / Lx (I). This is calculated for each section. That is, the ratio of section II is Ly (II) / Lx (II), and the ratio of section III is Ly (III) / Lx (III).

  When the ratio can be calculated for each section, it is specified which section includes the imaging position of the key image. As described above, since the movement trajectory of the capsule endoscope is configured by connecting the imaging positions on three-dimensional coordinates, the imaging position of the key image is also included in this movement trajectory. Therefore, it is possible to specify in which section the key image was captured.

  Then, the imaging position of the specified key image is corrected at a ratio corresponding to the specified section, thereby mapping the imaging position of the key image on the colon line in the three-dimensional large intestine model. For example, if there is a key image captured at a position 10 cm away from the Bauhin valve 303 along the movement trajectory, and if the ratio of the section I is 1.5, the Bauhin valve 503 in the three-dimensional large intestine model moves from the Bauhin valve 503 to the colon core line. Mapping is performed at a position 15 cm away from the camera. That is, the position is estimated by integrating the ratio of 1.5 to 10 cm.

  In this way, the Bauhin valve, the liver curved part, the spleen curved part, and the anus that are landmarks whose positions do not relatively change even when gas or the like is injected into the large intestine in a large intestine CT examination are divided into sections with landmarks. If calculated and used as a mapping correction parameter, mapping (alignment) is more accurate than correction using the ratio of the total length of the colon core line in the three-dimensional large intestine model to the entire length of the capsule endoscope's movement path. ).

[Key Image Settings]
As in the above embodiment, a landmark field is added to the management table of the capsule endoscope image information, and the respective positions of the Bauhin valve, the liver bending portion, the spleen bending portion, and the anus are set. As a result, of all the capsule endoscope image files (including the esophagus, stomach, and small intestine), it is possible to narrow the scope of the large intestine examination. That is, it is only necessary to check the image files in the range from the landmarks “1” to “4”. In addition, the user (doctor) can pay close attention to browsing the captured image of only the large intestine, so that the burden is small.

  In the present embodiment, each image file in the range of the landmarks “1” to “4” is sequentially displayed as a moving image in the same manner as each frame of the moving image. Set as At this stage, the user does not need to know which position in the large intestine the set key image is. The display method for setting the key image is not particularly limited, but the display form shown in FIG. 6 is preferable. As shown in the figure, a plurality of images along a time axis are scrolled to the left at a speed according to an appropriate time and displayed. When an image of interest (for example, an image including a polyp is typical) is displayed, the user sets the key image by clicking the image with, for example, a mouse. By displaying a plurality of adjacent images side by side, it is possible to prevent omission of setting of a key image. The scroll itself may be performed by a user operation.

  Images set as key images need to be managed separately from other images. For this reason, the "key image" field is added to the management table as shown in FIG. Any number of key images may be set. Therefore, every time a key image is set, the number indicating the key image is updated. The initial value is “1”. In the case of FIG. 3, it is indicated that the images captured when the elapsed times are k1 and k2 are set as key images.

  At the stage of setting the key image, it is unknown to the user at which position in the large intestine the set key image is.

[Explanation of virtual colonoscope]
When a three-dimensional large intestine model is obtained and its large intestine core line is calculated, diagnosis and observation with a virtual endoscope simulating an actual large intestine endoscope become possible. Although there is no limitation on the gaze position and direction in the three-dimensional large intestine model, a normal mode imitating an actual large intestine endoscope will be described.

  In the normal mode of the virtual endoscope, the virtual viewpoint (virtual camera) is located on the previously determined colon center line and moves on the colon center line. The direction of movement is the same as that of an actual colonoscope, from the anus to the rectum, sigmoid colon, descending colon, transverse colon, ascending colon, and Bauhin valve. Hereinafter, this direction is referred to as a forward direction. However, there is no limitation on the direction of the line of sight of the actual colonoscope, and the line of sight can be directed in the direction opposite to the forward direction. In some cases, the virtual line of sight may be moved to a position deviating from the center line. In order to move the line of sight to a position away from the center line, the operation unit 107 may be operated to switch from the normal mode to the free line of sight mode. The movement of the virtual viewpoint in the forward and reverse directions and the setting of the direction of the line of sight are performed by operating the keyboard, but may be performed by a pointing device such as a mouse. Absent.

  While moving the virtual viewpoint in the forward direction, a video (virtual image, virtual endoscope image) viewed from the position of the virtual viewpoint is generated and displayed on the display unit 106 in the same manner as a normal endoscope. The second feature of the present embodiment is that, in this process, when the virtual line of sight moves to the vicinity of the key image, the user is notified that the virtual gaze has reached the vicinity of the key image. Then, when the distance between the virtual viewpoint and the photographing position of the key image becomes smaller than or equal to a preset threshold value, the key image captured by the capsule endoscope is superimposed and displayed as a separate window from the virtual image. In order to realize this, the inventor has considered the following processing.

The key image is an image selected by the user (doctor). In the example shown in FIG. 2, images captured at the elapsed times k1 and k2 are set as key images. Here, attention is paid to the image of the elapsed time k2. This image is an image taken when the capsule endoscope is located between the landmarks “2” and “3”, that is, on the transverse colon between the liver curve and the spleen curve. is there. The elapsed time of the image of the landmark “2” is m2, and the elapsed time of the landmark “3” is m3. Therefore, the time required for the capsule endoscope to move from the left end to the right end of the transverse colon can be defined as "m3-m2". On the other hand, the elapsed time k2 is a timing when “k2−m2” has elapsed since the capsule endoscope started the liver bending portion. In the three-dimensional colon model, the length L (II) of the transverse colon along the colon colon line can be calculated from the three-dimensional colon model. Therefore, when it is considered that the capsule endoscope has moved at the constant speed for the distance L (II), the capsule endoscope when the image of the elapsed time k2 is captured is calculated from the position of the liver bending portion by the following equation. Can be derived (calculated) as being at a position on the colon colon line which is separated by a distance D shown by.
D = {(k2-m2) / (m3-m2)} × L (II)
Stated another way, the capsule endoscope when the image of the elapsed time k2 is captured, internally divides the length L (II) of the transverse colon into {k2-m2}: {m3-k2}. It can be said that it can be regarded as being in a position.

  In reality, the moving speed of the capsule endoscope in the large intestine is not constant, and its moving locus does not become a smooth curve. However, as a result of the above processing, it is assumed that, for example, the capsule endoscope captures an image of a polyp while moving on the “transverse colon”, and the user selects an image including the polyp as a key image. According to the present embodiment, it is almost guaranteed that the designated key image is located on the transverse colon in section II, and the error with respect to the position on the three-dimensional large intestine model is about several centimeters, which is sufficient accuracy. Here are some additional points that have been confirmed.

  Now, according to the management table shown in FIG. 2, the vertical (or left and right) direction of the image captured by the camera unit of the capsule endoscope is unknown. This is because there is no information on the rotation angle with respect to the longitudinal axis of the capsule endoscope in the management table. However, the direction to the center position of the image captured by the capsule endoscope image (the line of sight of the camera unit) matches the imaging direction in the management table. Therefore, as shown in FIG. 7, when it is considered that the capsule endoscope is located on the core line of the large intestine, the position of the point 701 where the line of sight intersects with the inner wall of the three large intestine model is calculated. Can be.

  In the present embodiment, if the distance between the virtual viewpoint position and the key image is less than or equal to a predetermined threshold while moving the virtual viewpoint position to the colon core line with the virtual endoscope, the visual field viewed from the virtual viewpoint position Inside the key image, a mark indicating the center position of the key image is further displayed.

  Here, a description will be made so as not to be misunderstood, but the "distance" mentioned here is a distance on the axis when the colon core line is regarded as a one-dimensional axis. For example, the Bauhin valve and the sigmoid colon are physically close, but the distance between them on the axis of the colon core line is substantially separated by the entire length of the large intestine. Therefore, when the user sees the mark while displaying the virtual image using the virtual endoscope, it is possible to confirm that the user has reached near the position specified as the key image. . Further, in the embodiment, when the difference between the virtual viewpoint position on the colon center line and the position of the key image with respect to the colon center line is equal to or less than a preset threshold, the capsule endoscope is used in addition to the virtual image. Key images are now displayed. As a result, the key image is displayed only when the virtual viewpoint position in the process of operating the virtual endoscope approaches the position set as the key image, and much time is devoted to operating the virtual endoscope. You can do it.

  FIG. 8 illustrates a display example of the display unit 106 when the image diagnosis support application according to the embodiment is executed and the device according to the embodiment functions as a virtual endoscope device. On this screen, a first display area 801 for displaying a three-dimensional colon model, a second display area 802 for displaying an image of the inner wall of the large intestine viewed from a virtual viewpoint on a core line of the large intestine in the three-dimensional colon model, And a third display area 803 that displays an image captured by the capsule endoscope positioned before and after the virtual viewpoint position of the center.

  Reference numeral 811 in the first display area 801 is an icon indicating a virtual viewpoint (virtual camera) of the virtual endoscope apparatus. Reference numerals 812 and 813 are estimated positions corresponding to the key image specified by the user. The estimated positions 812 and 813 correspond to the key images of the elapsed times k1 and k2 in FIG. 3, respectively.

  The mark 821 in the second display area 802 indicates that the center position of the key image is included in the video viewed from the virtual viewpoint. This mark 821 is displayed when the difference between the current position of the virtual viewpoint on the colon core line and the position of the key image set by the user on the colon colon line is equal to or less than a first threshold set in advance. Is done.

  The third display area 803 is an area for displaying a thumbnail image of a capsule endoscope image in a predetermined range before and after the current virtual viewpoint on the colon core line. A thick frame 831 at the center indicates a thumbnail image by the virtual endoscope, which is closest to the current position of the virtual viewpoint. A thick broken line frame 832 indicates a key image (corresponding to the estimated position 813 in the first display area).

  When the user operates the operation unit 107 to further advance the position of the virtual viewpoint (icon 811) in the forward direction, the difference between the position of the key image and the current virtual viewpoint position is equal to or less than a preset second threshold value. (The second threshold is smaller than the first threshold). In this case, the CPU 101 displays not the thumbnail image of the key image but the original key image as the fourth display area. FIG. 9 shows a state where the fourth display area 901 is displayed in a superimposed manner. The first and second thresholds may be set by a user. When the second threshold value is set to zero, the fourth display area 901 is displayed when the current virtual viewpoint position matches the position at the time of capturing the key image.

  The operation content and the principle of the virtual colonoscope in the embodiment have been described above. Next, a processing procedure in the image diagnosis application of the embodiment will be described based on the above processing.

[Explanation of processing procedure]
When a user (a doctor or the like) operates the operation unit 107 to instruct activation of the image diagnosis application, the CPU 101 loads the image diagnosis application program from the HDD 104 to the RAM 103 and executes the program. FIG. 10 shows a processing procedure of the CPU 101 when the image diagnosis application program is executed.

  First, in step S1001, the CPU 101 acquires the capsule endoscope image information 110 from the capsule endoscope device via the interface 105 and stores it in the HDD 104 (acquisition unit). In the next step S1002, the CPU 101 analyzes the acquired capsule endoscope image information 110, generates and displays a movement trajectory curve of the capsule endoscope in the body of the subject based on the management table, and displays the curve. Then, the user is caused to set the positions of a plurality of preset landmarks (in the embodiment, the Bauhin valve, the liver bending portion, the spleen bending portion, and the anus) (see FIG. 4).

  As a result, at least the positions of both ends of the large intestine can be determined. In step S1003, the CPU 101 narrows down the captured images captured when the capsule endoscope is positioned in the large intestine (excludes captured images of organs such as the esophagus, stomach, and small intestine), and time-series the captured images. And prompts the user to select a key image. As a result of the processing so far, the management table is updated as shown in FIG. As a display example at the time of selecting a key image, as shown in FIG. 6, a format in which a plurality of images continuous with respect to a time axis is desired is desired. This is because a key image can be selected while comparing with images that are temporally preceding and following.

  Next, in step S1004, the CPU 101 acquires CT image information 120 from the X-ray CT apparatus via the interface 105, and stores the CT image information 120 in the HDD 104. In the next step S1005, CPU 101 generates a three-dimensional large intestine model from acquired CT image information 120. At this time, a Bauhin valve and an anus are designated as landmarks to separate the large intestine from other organs. Next, in S1006, the CPU 101 displays the generated three-dimensional large intestine model and allows the user to set another landmark (liver curved portion, spleen curved portion).

  In the next step S1007, the CPU 101 calculates a three-dimensional colon core line passing through the center along the longitudinal direction of the large intestine indicated by the three-dimensional large intestine model. Thereafter, in step S1008, the imaging position of the key image set by the user on the colon line is determined, and the line-of-sight direction of the capsule endoscope when the key image is imaged is defined by the three-dimensional colon model. Find the position of the intersection with the inner wall of the large intestine.

  The detailed process of step S1008 will be described with reference to FIG. First, in step S1301, the CPU 101 positions the colon core of the three-dimensional large intestine model calculated in step S1007 in accordance with the shape of the large intestine such as the Bauhin valve, the liver curved portion, the spleen curved portion, and the anus set in step S1005 and step S1006. (Second specifying means) according to the landmarks assigned to the sections.

  In step S1302, the CPU 101 also determines the movement path of the capsule endoscope according to landmarks provided at positions corresponding to the shape of the large intestine such as the Bauhin valve, the liver bending portion, the spleen bending portion, and the anus set in step S1002. Divide into sections (first specifying means).

  In step S1303, the CPU 101 determines, for each of the sections divided in steps S1008 and S1009, the distance of the section of the colon core line in the three-dimensional large intestine model corresponding to the section relative to the section of the movement trajectory of the capsule endoscope. Calculate the ratio (parameter). Then, in step S1304, the CPU 101 specifies a section on the movement trajectory of the capsule endoscope including the imaging position of the key image (third specifying unit).

  In step S1305, the CPU 101 performs mapping by correcting the imaging position of the key image with the ratio of the specified section. If there are a plurality of key images, steps S1304 and S1305 are executed for each key image. As described above, by using the ratio calculated for each section, the imaging position can be accurately mapped to the colon colon line. When the processing has been completed for all the key images, the processing proceeds to step S1009.

  As a result, the preparation for the browsing process by the virtual endoscope is completed, and the CPU 101 executes the browsing process by the virtual endoscope in S1009. Hereinafter, this browsing process will be described with reference to the flowchart of FIG. Note that, in addition to the processing described below, the CPU 101 performs display control processing such as updating the position of the icon 811 and the third display area 830, but omits the description.

  In step S1101, the CPU 101 initializes the position of the virtual viewpoint (the icon 811 in FIG. 8) and the line-of-sight direction. The initial position of the virtual viewpoint is typically set at the rectum. However, the virtual viewpoint is located on the colon line. The initial line-of-sight direction is a forward tangent direction at the initial virtual viewpoint position.

  Next, in step S1102, the CPU 101 generates and displays, based on the CT image information 120 (medical image data), a large intestine image within a preset angle of view, which is visible from the current position of the virtual viewpoint and the line of sight. (2nd display area of FIG. 8). Then, in step S1103, the CPU 101 determines whether or not there is an imaging position of the key image within the first threshold Th1 from the current position of the virtual viewpoint on the colonic core line. If the determination result indicates that there is no existence, the process proceeds to step S1107.

  Here, a description will be given of a case where the data exists. In this case, in step S1104, the CPU 101 compositely displays a predetermined mark (reference numeral 821 in FIG. 8) at a position corresponding to the center of the key image in the virtual image being displayed. In the embodiment, a circle is used as a mark shape, but the mark shape is not particularly limited. Further, since the distance from the mark image is known, the display form (size and color) of the mark may be changed according to the distance.

  Next, in step S1105, the CPU 101 determines whether or not there is an imaging position of the key image within the second threshold Th2 from the current position of the virtual viewpoint on the colon colon line. If the determination result indicates that there is no data, the process skips step S1106 and proceeds to step S1107. On the other hand, if the imaging position of the key image exists within the distance indicated by the threshold Th2, the original key image is displayed in step S1106 (the fourth display area 901 in FIG. 9).

  In step S1107, the process waits for a user to input an instruction to change the virtual viewpoint position and the viewing direction. If any of the instructions is given, in step S1108, the CPU 101 changes the current position of the virtual viewpoint or the gaze direction according to the instruction, and returns the process to step S1102.

  As described above, when the user moves the virtual viewpoint or changes the line of sight and diagnoses the inner wall of the large intestine, when the user approaches the position indicated by the actual captured image of the capsule endoscope specified by the user (distance Is less than or equal to the first threshold), the key image is present near the current virtual viewpoint, and its position can be visually confirmed. In addition, when the virtual viewpoint is further approached, the original key image actually photographed is displayed, so that the diagnosis of a lesion becomes easy.

  In the above embodiment, the three-dimensional large intestine model is generated by the image diagnosis application. However, the three-dimensional large intestine model may be generated by another device. In this case, the “other device” may use either CT or MRI. In short, it is only necessary to obtain the three-dimensional colon model and capsule endoscope information of the same subject.

  Further, in the above embodiment, the key image is displayed when the difference between the virtual viewpoint position and the imaging position of the key image is equal to or less than the second threshold Th2. However, when the mouse cursor is moved over the marker, the key image is displayed. An image may be displayed.

[Other embodiments]
In the above embodiment, the imaging position of the key image is regarded as the position of the image on the core line in the three-dimensional colon model. However, as shown in FIG. 7, there is a difference of ΔL between the center position of the captured image and the position of the capsule endoscope on the colon line, so that the ΔL is determined in the distance determination processing. May be added to make the determination.

  In the virtual endoscope, the direction of the line of sight can be freely changed according to a user's instruction. Therefore, there is a possibility that it is not clear whether the current gaze direction is the forward direction. According to the present embodiment, as shown in FIG. 8, the icon 811 representing the virtual viewpoint changes its direction based on the line of sight, so that the possibility can be reduced. Whether the gaze direction is the forward direction or the reverse direction may be determined from the virtual image. For example, the color of the frame of the second display area 802 may be distinguished in the forward direction and the reverse direction. Also, the color of the mark 821 may be distinguished as to whether it is viewed from the forward direction or from the reverse direction.

  In the above embodiment, the virtual endoscope image is generated based on the information from the X-ray CT apparatus. However, the virtual endoscope image is also generated from information of another modality apparatus (medical image diagnostic apparatus) such as MRI. The present invention is not limited to the above embodiment because an image can be generated.

  Further, as is apparent from the above description, the present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program and reads the program. This is the process to be performed.

101 CPU, 102 ROM, 103 RAM, 104 HDD, 105 I / F, 106 display unit, 107 operation unit

Claims (8)

A medical image processing apparatus capable of displaying a virtual endoscopic image of a large intestine included in a plurality of medical images obtained by a medical image diagnostic apparatus,
Acquisition of a plurality of capsule endoscope image information composed of a capsule endoscope image captured by a capsule endoscope and coordinate information indicating an imaging position of the capsule endoscope image, and a plurality of the medical images Acquisition means for
A first specification for specifying a first section corresponding to the shape of the large intestine from a movement trajectory of the capsule endoscope constituted by a plurality of imaging positions included in the capsule endoscope image information acquired by the acquisition means; Means,
A second specifying unit that specifies a second section corresponding to the first section according to a shape of the large intestine from a core line of the large intestine included in the plurality of medical images acquired by the acquiring unit;
Using a parameter based on the way between the end points of the first and journey across points of the identified said first interval by a particular unit, the said identified in the second identifying unit the second section A third specifying unit that specifies a position on a core line of the large intestine corresponding to an imaging position of a capsule endoscope image captured in the first section ;
After the position on the center line of the large intestine corresponding to the imaging position of the capsule endoscope image is specified, when moving the virtual line-of-sight position on the center line of the large intestine with the virtual endoscope, a plurality of If the difference between the imaging position of the key image set as the image of interest to the user and the virtual gaze position of the capsule endoscope image is equal to or less than a threshold, the virtual gaze position is the key image. Notification means for notifying the user that the imaging position has been approached,
A medical image processing apparatus comprising: The distance between both end points of the first section is a distance on the axis when the movement trajectory of the capsule endoscope is regarded as a one-dimensional axis, and is a distance between both end points of the second section. The medical image processing apparatus according to claim 1, wherein the distance is a distance on the axis when the center line of the large intestine is regarded as a one-dimensional axis. The third specific means, between said first relative distance between the end points of the identified said first interval by a specific means, both end points of said identified in the second identifying unit the second section using the ratio of road as the parameter, claim 1, the corresponding to the imaging position of the imaging capsules endoscopic image in the first section, characterized in that for identifying a position on the core line of the colon Or the medical image processing apparatus according to 2. Said first specifying means, the movement locus of the capsule endoscope configured by an imaging position included in the acquired plurality of capsule endoscope image information by the acquisition unit, liver curved from Bauhin valve of the large intestine section up section, a section from the liver curved portion of the large intestine to the spleen bend, and, to identify one of the sections of the section from the spleen bend of the large intestine to the anus as the first section the medical image processing apparatus according to any one of claims 1 to 3, characterized. Said second specifying means, the core of the large intestine that is included in the plurality of medical images acquired by the acquisition unit, a section from Bauhin valve of the colon to the liver bend, spleen curved from the liver curved portion of the large intestine section up parts, and, according to any one of claims 1 to 4, wherein the identifying one of the sections of the section from the spleen bend of the large intestine to the anus as the second section Medical image processing apparatus. Before Symbol medical images medical image processing apparatus according to any one of claims 1 to 5, characterized in that a large intestine CT image generated by performing a colon CT examination by the medical image diagnostic apparatus. A control method of a medical image processing apparatus capable of displaying a virtual endoscopic image of a large intestine included in a plurality of medical images obtained by a medical image diagnostic apparatus,
A plurality of capsule endoscope image information, wherein the acquisition unit of the medical image processing apparatus includes a capsule endoscope image captured by the capsule endoscope and coordinate information indicating an imaging position of the capsule endoscope image; And an acquiring step of acquiring a plurality of the medical images,
The first specifying means of the medical image processing apparatus converts the shape of the large intestine into the shape of the large intestine from the movement trajectory of the capsule endoscope constituted by the plurality of capsule endoscope image information acquired in the acquiring step. A first specifying step of specifying a corresponding first section;
A second identification unit of the medical image processing apparatus, based on a core line of the large intestine included in the plurality of medical images acquired in the acquiring step, corresponding to the first section corresponding to the shape of the large intestine ; A second specifying step of specifying a section;
The third specific means of the medical image processing apparatus, the way between the end points of the first said identified in a particular step of the first section, said by said second specified in the second specifying step A third specification that specifies a position on the center line of the large intestine corresponding to an imaging position of a capsule endoscope image captured in the first section using a parameter based on a distance between both end points of the section. and the step,
After the position on the center line of the large intestine corresponding to the imaging position of the capsule endoscope image is specified, when moving the virtual line-of-sight position on the center line of the large intestine with the virtual endoscope, a plurality of If the difference between the imaging position of the key image set as the image of interest to the user and the virtual gaze position of the capsule endoscope image is equal to or less than a threshold, the virtual gaze position is the key image. A notification step of notifying the user of approaching the imaging position,
A method for controlling a medical image processing apparatus, comprising: A program capable of executing a control method of a medical image processing apparatus capable of displaying a virtual endoscopic image of a large intestine included in a plurality of medical images obtained by a medical image diagnostic apparatus,
The medical image processing apparatus,
Acquisition of a plurality of capsule endoscope image information composed of a capsule endoscope image captured by a capsule endoscope and coordinate information indicating an imaging position of the capsule endoscope image, and a plurality of the medical images Acquisition means for
A first specification for specifying a first section corresponding to the shape of the large intestine from a movement trajectory of the capsule endoscope constituted by a plurality of imaging positions included in the capsule endoscope image information acquired by the acquisition means; Means,
A second specifying unit that specifies a second section corresponding to the first section according to a shape of the large intestine from a core line of the large intestine included in the plurality of medical images acquired by the acquiring unit;
Using a parameter based on the way between the end points of the first and journey across points of the identified said first interval by a particular unit, the said identified in the second identifying unit the second section A third specifying unit that specifies a position on a core line of the large intestine corresponding to an imaging position of a capsule endoscope image captured in the first section ;
After the position on the center line of the large intestine corresponding to the imaging position of the capsule endoscope image is specified, when moving the virtual line-of-sight position on the center line of the large intestine with the virtual endoscope, a plurality of If the difference between the imaging position of the key image set as the image of interest to the user and the virtual gaze position of the capsule endoscope image is equal to or less than a threshold, the virtual gaze position is the key image. A program that functions as a notification unit that notifies the user that the user has approached the imaging position .

Images