JP4160487B2 - Insertion support system - Google Patents

Description

  The present invention relates to an insertion support system that supports insertion of an endoscope.

  In recent years, diagnosis based on images has been widely performed. For example, by taking a tomographic image of a subject with an X-ray CT (Computed Tomography) apparatus or the like, three-dimensional image data is obtained in the subject, and the 3D image data is obtained. Diagnosis of an affected area has been performed using dimensional image data.

  In the CT apparatus, by continuously feeding the subject in the body axis direction while continuously rotating the X-ray irradiation / detection, a helical continuous scan (helical scan) is performed on the three-dimensional region of the subject. A three-dimensional image is created from tomographic images of successive slices of a three-dimensional region.

  One such 3D image is a 3D image of the lung bronchi. The three-dimensional image of the bronchus is used to three-dimensionally grasp the position of an abnormal part suspected of lung cancer, for example. In order to confirm the abnormal portion by biopsy, a bronchoscope is inserted and a biopsy needle, biopsy forceps, or the like is taken out from the distal end portion and a tissue sample is taken.

As in the bronchi, in the ducts in the body with multi-stage branches, when the location of the abnormal part is close to the periphery of the branch, it is difficult to reach the target site correctly in a short time, For example, in Japanese Patent Laid-Open No. 2000-135215, etc., a three-dimensional image of a pipeline in the subject is created based on image data of a three-dimensional region of the subject, and along the pipeline on the three-dimensional image. By obtaining a route to a destination point, creating a virtual endoscopic image of the duct along the route based on the image data, and displaying the virtual endoscopic image, a bronchoscope A device for navigating to a target site has been proposed.
JP 2000-135215 A

  However, if the patient coughs during insertion of the endoscope, the insertion part may move back and forth momentarily, and it will be difficult to continue assistance because the previous navigation position and actual insertion position change. There is a problem you have done

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an insertion support system that can easily return to navigation and can reliably support insertion of an endoscope.

  The insertion support system according to the present invention includes a virtual image generation unit that generates a three-dimensional image of a body cavity in the subject as a continuous virtual image based on image data of a three-dimensional region of the subject, and the subject Navigation comprising an endoscopic image from an endoscope that images an internal body cavity path, the virtual image, and a plurality of reduced images of the virtual image at all branch points where the body cavity path in the subject branches Navigation image generating means for generating an image, and image changing means for changing the virtual image of the navigation image generated by the navigation image generating means to a virtual image that is an original image of a predetermined reduced image.

  According to the present invention, it is possible to easily return to navigation and to reliably support insertion of an endoscope.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIGS. 1 to 22 relate to Embodiment 1 of the present invention, FIG. 1 is a block diagram showing the configuration of a bronchial insertion support system, and FIG. 2 is a flowchart showing the flow of navigation data generation processing by the insertion support device of FIG. 3 is a first diagram showing a route setting screen developed by the processing of FIG. 2, FIG. 4 is a second diagram showing a route setting screen developed by the processing of FIG. 2, and FIG. 5 is a route setting of FIG. FIG. 6 is a first diagram showing a route setting screen developed by the processing of FIG. 5, FIG. 7 is a second diagram showing a route setting screen developed by the processing of FIG. 8 is a third diagram showing a route setting screen developed by the process of FIG. 5, FIG. 9 is a flowchart showing the flow of navigation processing by the bronchial insertion support system of FIG. 1, and FIG. 10 is developed by the process of FIG. The first figure showing the insertion support screen 11 is a second diagram showing the insertion support screen developed by the process of FIG. 9, FIG. 12 is a third diagram showing the insertion support screen developed by the process of FIG. 9, and FIG. 13 is the process of FIG. FIG. 14 shows the insertion support screen expanded by the process of FIG. 9, FIG. 14 shows the insertion support screen expanded by the process of FIG. 9, and FIG. 15 shows the insertion support screen expanded by the process of FIG. FIG. 6, FIG. 16 is a seventh diagram showing the insertion support screen developed by the process of FIG. 9, and FIG. 17 is a diagram showing a first modification of the insertion support screen developed by the process of FIG. 9. 18 is a diagram showing a second modification of the insertion support screen developed in the process of FIG. 9, FIG. 19 is a diagram showing a registered branch point list window superimposed on the insertion support screen of FIG. 18, and FIG. FIG. 21 is a diagram showing a third modification of the insertion support screen developed by the process of FIG. 9, and FIG. 21 shows the insertion support screen developed by the process of FIG. Illustrates a modification of 4, FIG. 22 is a diagram showing the transition of representation of the read button shown in FIG. 21.

  As shown in FIG. 1, the bronchial insertion support system 1 according to the present embodiment includes a bronchial endoscope device 3 and an insertion support device 5.

  The insertion support device 5 generates a virtual endoscopic image (hereinafter referred to as a VBS image) inside the bronchus based on the CT image data, and an endoscopic image (hereinafter referred to as a live image) obtained by the bronchial endoscopic device 3. ) And the VBS image are combined and displayed on the monitor 6 to assist insertion into the bronchus of the bronchoscope apparatus 3.

  Although not shown, the bronchoscope apparatus 3 includes a bronchoscope having an imaging unit, a light source that supplies illumination light to the bronchoscope, a camera control unit that processes an imaging signal from the bronchoscope, and the like. Then, a bronchoscope is inserted into the bronchus in the patient, and the inside of the bronchus is imaged to biopsy the affected tissue at the end of the bronchus, and the live image and the VBS image are synthesized and displayed on the monitor 7.

  The monitor 7 is provided with an input unit 8 formed of a touch panel, and can easily operate the input unit 8 formed of a touch panel while performing an insertion procedure.

  The insertion support device 5 can use 3D image data generated by a known CT device (not shown) for taking an X-ray tomographic image of a patient, such as an MO (Magnetic Optical Disk) device or a DVD (Digital Versatile Disk) device. A CT image data capturing unit 11 that is captured via a portable storage medium, a CT image data storage unit 12 that stores CT image data captured by the CT image data capturing unit 11, and a CT image data storage unit 12. An MPR image generation unit 13 that generates an MPR image (multi-section reconstructed image) based on the CT image data that is present, and a route setting screen that includes an MPR image generated by the MPR image generation unit to generate a bronchoscope device 3 A route to set a support route to the bronchi (hereinafter simply referred to as a route) VBS image as virtual image generation means for generating a continuous VBS image of the route set by the route setting unit 14 based on the CT image data stored in the CT setting data storage unit 12 and the CT image data storage unit 12 A generation unit 15, a VBS image storage unit 16 that stores a VBS image generated by the VBS image generation unit 15, an imaging signal from the bronchoscope device 3 and an input signal from the input unit 8 are input, and a live image, An image processing unit 17 serving as a navigation image generating unit that generates an insertion support screen (described later) composed of a VBS image and a plurality of thumbnail VBS images, and a route setting screen generated by the route setting unit 14 and an insertion support generated by the image processing unit 17 An image display control unit 18 for displaying the screen on the monitor 6 and a keyboard for inputting setting information to the route setting unit 14. And an input device 19 for consisting De and a pointing device.

  The bronchoscope apparatus 3 receives a VBS image and a thumbnail VBS image from the image processing unit 17 of the insertion support apparatus 5 and combines them with a live image, and is equivalent to an insertion support screen displayed on the monitor 6 by the insertion support apparatus 5. Is displayed on the monitor 7, and input information from the input unit 8, which is a touch sensor of the monitor 7, is output to the image processing unit 17 of the insertion support apparatus 5.

  The CT image data storage unit 12 and the VBS image storage unit 16 may be configured by a single hard disk, and the MPR image generation unit 13, the route setting unit 14, the VBS image generation unit 15 and the image processing unit 17 include A single arithmetic processing circuit can be used. The CT image data capturing unit 11 captures CT image data via a portable storage medium such as an MO or a DVD. However, a CT apparatus or a hospital server storing the CT image data is connected to the hospital LAN. In such a case, the CT image data capturing unit 11 may be configured by an interface circuit that can be connected to the in-hospital LAN, and CT image data may be captured through the in-hospital LAN.

  The operation of this embodiment configured as described above will be described.

  As shown in FIG. 2, prior to the observation / treatment by the bronchoscope apparatus 3, the insertion support apparatus 5 captures the CT image data of the patient generated by the CT apparatus by the CT image data capturing unit 11 in step S1, The CT image data captured in step S2 is stored in the CT image data storage unit 12.

  In step S <b> 3, the route setting unit 14 displays a route setting screen 21 as shown in FIG. 3 on the monitor 6, and selects patient information on the patient information tag screen 22 on the route setting screen 21. By this selection, an MPR image composed of, for example, three different multi-sectional images of the patient selected in step S4 is generated, and this MPR image 23 is displayed on the route setting screen 21 in step S5.

  The selection of patient information on the patient information tag screen 22 is performed by inputting a patient ID for identifying a patient by the input device 19.

  Next, when the route setting tag 24 (see FIG. 3) on the route setting screen 21 is selected by the input device 19 in step S6, a route setting tag screen 25 as shown in FIG. 4 is displayed on the route setting screen 21, which will be described later. The route setting process is performed, and the bronchoscope insertion support route in the bronchus is set.

  When the insertion support route is set, continuous VBS images of all routes set by the VBS image generation unit 15 are generated in units of frames in step S7, and the VBS image generated in step S8 is stored in the VBS image storage unit 16. To do.

  With the processing of steps S1 to S8 described above, preparation for insertion support by the insertion support device 4 during observation / treatment with a bronchoscope is completed.

  Here, the route setting process in step S6 will be described with reference to FIG.

  As shown in FIG. 5, in the route setting process in step S6, when the route search button on the route setting tag screen 25 shown in FIG. 4 is clicked by operating the input device 19, the step shown in FIG. A start point input instruction window 31 for prompting the input of the start point of the route is displayed on the route setting screen 21, and the start point is set on one tomographic image of the MPR image 23 using the cursor 30 on the route setting screen 21. To do. When the start point is set, the start point is set at a corresponding position on the two tomographic images of the other MPR images 23, and an end point input instruction window 32 for prompting the input of the end point of the route as shown in FIG. 7 is displayed on the route setting screen. 21 is displayed.

  Therefore, in step S12, the end point is set on one tomographic image of the MPR image 23 using the cursor 30 on the route setting screen 21 in the same manner as the setting of the start point. When the end point is set, the end point is set at a position corresponding to the two tomographic images of the other MPR images 23.

  When the start point and end point are set, the route setting unit 14 searches for a route in the bronchus from the start point to the end point in step S13. Since the bronchus has a complicated route, the route in the bronchus from the start point to the end point is not necessarily uniquely determined. Therefore, in the route setting unit 14, the route in the bronchus from the start point to the end point is determined in step S 13. The first candidate is searched.

  Then, on the route setting screen 21, the route setting unit 14 superimposes and displays the route searched in step S14 on the MPR image 23 as shown in FIG. A window 33 is displayed.

  The route confirmation window 33 includes a route confirmation button 41 for instructing the confirmation of the searched route, a next candidate search button 42 for instructing a search for the next candidate route, and a route reset button 43 for resetting the start point and the end point. And a cancel button 44 for canceling the route search process.

  In step S15, it is determined whether or not the next candidate search button 42 has been clicked. If clicked, the next candidate route is automatically searched in step S16 and the process proceeds to step S17. If not clicked, the process proceeds to step S18. In step S17, it is determined whether or not the next candidate exists as a result of searching for the next candidate. If there is no next candidate, a warning that the next candidate route does not exist is displayed but not shown, and the process returns to step S13. Return to step S14.

  In step S18, it is determined whether or not the route reset button 43 has been clicked. If clicked, the process returns to step S11, and if not clicked, the process proceeds to step S19.

  In step S19, it is determined whether or not the route confirmation button 41 is clicked. If not clicked, the process returns to step S15. If clicked, the process proceeds to step S20. In step S20, the position information of the route and each branch point in the route is obtained. And return to step S7 in FIG.

  The insertion support at the time of observation and treatment by the insertion support device 5 and the bronchoscope device 3 in which the route is set in this way will be described. In the following, a case where there are 10 branch points on the route will be described as an example.

  As shown in FIG. 9, when bronchoscopy under insertion assistance by the insertion assistance device 5 is started in step S <b> 41, an insertion assistance screen 51 as shown in FIG. 10 is displayed on the monitor 7. Note that the same insertion support screen 51 as the monitor 7 is displayed on the monitor 6.

  The insertion support screen 51 includes an endoscope live image display area 52 that displays a live image 52a from the bronchoscope apparatus 3, a VBS image display area 53 that displays a VBS image image 53a, and all branches of the route. A virtual image corresponding to the branch point where the live image 52a is located, which is composed of a branch thumbnail VBS image area 54 which is displayed as branch thumbnail VBS images 54 (a) to 54 (j) by reducing the VBS image image 53a at the point. A VBS image 53a is displayed in the VBS image display area 53.

  Here, the same branch thumbnail VBS image frame as the VBS image 53a displayed in the VBS image display area 53 is displayed in a thick frame or in color, so that it can be distinguished from other branch thumbnail VBS images. The branch of the VBS image displayed in the display area 53 can be easily recognized.

  In addition, in the insertion support screen 51 of the present embodiment, the re-navigation position marker 100 indicating the return branch point at which re-navigation is started to return when the branch position becomes unknown during the insertion of the corresponding branch thumbnail VBS image. Marked at the bottom.

  In the insertion support screen 51, as a reduced image designation unit for performing position setting (return branch point setting) of the re-navigation button 101 and the re-navi position marker 100 as an image changing unit for instructing the start of re-navigation. A setting button 102 is provided. In addition, the insertion support screen 51 is provided with a front button 104 and a next button 105 for moving the VBS image 53a back and forth in units of frames. By selecting at 103, the VBS image 53a can be moved forward and backward in units of frames.

  For example, the default re-navigation position marker 100 is set in the branch thumbnail VBS image 54 (a). In FIG. 10, the re-navigation position marker 100 by default and the VBS image image 53a which is a virtual image corresponding to the branch point where the live image 52a is located are the original images of the branch thumbnail VBS image 54 (f). Show.

  Returning to FIG. 9, in step S42, the input unit 8 is operated, the setting button 102 is clicked by the cursor 103 on the insertion support screen 51, and it is determined whether there is an instruction to reset the return branch point. If there is a setting instruction, a return branch point is set in step S43, re-marking is performed below the branch thumbnail VBS image corresponding to the return branch point where the re-navigation position marker 100 is set in step S44, and the process proceeds to step S45. If there is no return branch point resetting instruction, the process directly proceeds to step S45, where the bronchoscope insertion destination navigation at the branch point by the VBS image is executed. Specific contents of the processing of steps S42 to S44 will be described later.

  During the execution of the navigation in step S45, the navigation continues to be hindered, such as the patient coughing and the insertion position is not known. In step S46, the input unit 8 is operated to re-navigate with the cursor 103 on the insertion support screen 51. It is determined whether or not the button 101 is clicked to instruct to start re-navigation. If there is an instruction to start re-navigation, the VBS image 53a is returned to the branch point in step S47, and the process proceeds to step S48. If NO, the process proceeds to step S48, and it is determined whether or not the end of bronchoscopy has been instructed in step S48. If there is no end instruction, the process returns to step S42. Specific contents of the processing in steps S46 and S47 will be described later.

  In the process of steps S46 and S47, when the patient coughs up, the live image 52a in FIG. 10 changes like the live image 52a in FIG. 11, and the correspondence between the live image 52a and the VBS image 53a becomes impossible. 11, when the re-navigation button 101 is clicked with the cursor 103 on the insertion support screen 51, as shown in FIG. 12, the original of the branch thumbnail VBS image having the re-navigation position marker 100 indicating the return branch point. A VBS image 53a of the image is displayed.

  In this state, by pressing the next button 105 with the cursor 103, the VBS image 53a is advanced in units of frames to the VBS image 53a that can be associated with the live image 52a as shown in FIG. 13, and navigation is resumed.

  Next, for example, in FIG. 14, the position of the re-navigation position marker 100 is set to the default branch thumbnail VBS image 54 (a), but the surgeon desires navigation only after the branch thumbnail VBS image 54 (c). (When the surgeon knows the route from the branch thumbnail VBS image 54 (a) to the branch thumbnail VBS image 54 (c)), the branch thumbnail VBS image 54 (a) with the re-navigation position marker 100 is used. Based on the example of resetting to the branch thumbnail VBS image 54 (c), the processing of steps S42 to S44 will be described.

  As shown in FIG. 15, first, a branch thumbnail VBS image to be set as a return branch point is selected with the cursor 103 on the insertion support screen 51, and then the setting button 102 is clicked with the cursor 103 as shown in FIG. Thus, the setting is completed, and the re-navigation position marker 100 moves to the lower portion of the desired branch thumbnail VBS image 54 (c) in this example, and marking is performed.

  As described above, in this embodiment, even if the patient coughs, the correspondence between the live image and the VBS image cannot be taken, the insertion position is unknown and the navigation is disturbed, the return branch point can be simply clicked on the re-navigation button. Since the VBS image can be returned to the position, the VBS image can be advanced in frame units, and the VBS image can be returned to the position corresponding to the live image, the navigation process can be easily continued.

  Further, since the return position can be arbitrarily set, the navigation can be returned from the position desired by the operator.

  As shown in FIG. 17, the registration button 200 and the readout button 201 are displayed on the insertion support screen 51, and the position of the return branch point set by the setting button 102 when the registration button 200 is clicked with the cursor 103 is displayed. The return branch point registered in the image processing unit 17 can be instantaneously read by clicking the read button 201 with the cursor 103, and the branch thumbnail VBS of the read return branch point can be registered and stored in the image processing unit 17. The re-navigation position marker 100 can be moved and displayed at the bottom of the image.

  In addition, the number of return branch points registered / stored in the image processing unit 17 is not limited to one, and a plurality of return branch point positions can be registered / stored in the image processing unit 17, and the number of registrations can be increased. As the number increases, the icon notation of the registration button 200 changes from “registration 1” to “registration 2”, “registration 2” to “registration 3”, etc., for example, the positions of three return branch points When registered and stored in 17, the icon notation of the registration button 200 becomes “registration 3” as shown in FIG. 18, and the number of registrations can be confirmed by this icon notation.

  When the read button 201 is clicked with the cursor 103 as an action of the read button 201 when the positions of a plurality of return branch points are registered and stored in the image processing unit 17, for example, as shown in FIG. Can be superimposed on the insertion support screen 51, and the desired branch point can be double-clicked with the cursor 103 to read the return branch point instantaneously. At this time, the registered branch point list is displayed in one-to-one correspondence with the return branch points registered with the registration button 200.

  If, for example, the positions of three return branch points are registered without displaying the registered branch point list window 211, the icon notation of the read button 201 is changed from simple “read” to the icon notation shown in FIG. If a change is made and the “→” icon 220 is clicked, the registered branch point position number may be incremented, and if the “←” icon 221 is clicked, the registered branch point position number may be decremented. In the icon notation shown in FIG. 20, a branch point number notation area 222 indicating the number of the position of the currently selected branch point is provided between the “→” icon 220 and the “←” icon 221, and this branch point number is provided. The number of the position of the currently selected branch point can be easily confirmed in the notation area 222.

  If, for example, the positions of three return branch points are registered without displaying the registered branch point list window 211, the icon notation of the read button 200 is changed from “Read” to “Read 2” as shown in FIG. When the readout button 200 is clicked, for example, if the position number of the currently selected branch point is “1”, as shown in FIG. 22, “read 1” → “ The notation is changed as “read 2” → “read 3” → “read 1” →. The number indicated by this notation has a one-to-one correspondence with the position of the branch point in the order registered by the registration button 200, and “Read 1” is a state in which the position of the first registered branch point is currently being read. “Read 2” indicates a state where the position of the next registered branch point is currently read, and “Read 3” indicates a state where the position of the last registered branch point is currently read. The re-navigation position marker 100 can be moved and displayed below the branch thumbnail VBS image at the read return branch point.

  The present invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the scope of the present invention.

The block diagram which shows the structure of the bronchial insertion assistance system which concerns on Example 1 of this invention. The flowchart which shows the flow of the production | generation process of the navigation data by the insertion assistance apparatus of FIG. The 1st figure which shows the route setting screen expand | deployed by the process of FIG. 2nd figure which shows the route setting screen expand | deployed by the process of FIG. The flowchart which shows the flow of the route setting process of FIG. 1st figure which shows the route setting screen expand | deployed by the process of FIG. 2nd figure which shows the route setting screen expand | deployed by the process of FIG. 3rd figure which shows the route setting screen expand | deployed by the process of FIG. The flowchart which shows the flow of the navigation process by the bronchial insertion assistance system of FIG. The 1st figure which shows the insertion assistance screen expand | deployed by the process of FIG. The 2nd figure which shows the insertion assistance screen expand | deployed by the process of FIG. The 3rd figure which shows the insertion assistance screen expand | deployed by the process of FIG. The 4th figure which shows the insertion assistance screen expand | deployed by the process of FIG. The 5th figure which shows the insertion assistance screen expand | deployed by the process of FIG. The 6th figure which shows the insertion assistance screen expand | deployed by the process of FIG. The 7th figure which shows the insertion assistance screen expand | deployed by the process of FIG. The figure which shows the 1st modification of the insertion assistance screen expand | deployed by the process of FIG. The figure which shows the 2nd modification of the insertion assistance screen expand | deployed by the process of FIG. The figure which shows the registration branch point list window superimposed on the insertion assistance screen of FIG. The figure which shows the 3rd modification of the insertion assistance screen expand | deployed by the process of FIG. The figure which shows the 4th modification of the insertion assistance screen expand | deployed by the process of FIG. The figure which shows the transition of the description of the read-out button of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Bronchial insertion support system 3 ... Bronchoscope apparatus 5 ... Insertion support apparatus 6, 7 ... Monitor 8 ... Input part 11 ... CT image data acquisition part 12 ... CT image data storage part 13 ... MPR image generation part 14 ... Route Setting unit 15 ... VBS image generation unit 16 ... VBS image storage unit 17 ... Image processing unit 18 ... Image display control unit 19 ... Input device 51 ... Insertion support screen 100 ... Re-navigation position marker 101 ... Re-navi button 102 ... Setting button 103 ... Cursor 104 ... Previous button 105 ... Next button Agent Patent attorney Susumu Ito

Claims (3)

Virtual image generating means for generating a three-dimensional image of a body cavity in the subject as a continuous virtual image based on image data of a three-dimensional region of the subject;
An endoscopic image from an endoscope that images a body cavity path in the subject, the virtual image, and a plurality of reduced images of the virtual image at all branch points where the body cavity path in the subject branches. Navigation image generating means for generating a navigation image comprising:
An insertion support system comprising: an image changing unit that changes the virtual image of the navigation image generated by the navigation image generating unit to a virtual image that is an original image of a predetermined reduced image. The insertion support system according to claim 1, further comprising navigation start position setting means for setting a navigation start position during navigation using the virtual image. The insertion support system according to claim 2, wherein the navigation start position setting unit sets a re-navigation start position at the time of re-navigation using the virtual image.

Images