JP4575143B2 - Insertion support system - Google Patents

Description

  The present invention relates to an insertion support system that supports insertion of an endoscope.

  In recent years, diagnosis based on images has been widely performed. For example, by taking a tomographic image of a subject with an X-ray CT (Computed Tomography) apparatus or the like, three-dimensional image data is obtained in the subject, and the 3D image data is obtained. Diagnosis of an affected area has been performed using dimensional image data.

  In the CT apparatus, by continuously feeding the subject in the body axis direction while continuously rotating the X-ray irradiation / detection, a helical continuous scan (helical scan) is performed on the three-dimensional region of the subject. A three-dimensional image is created from tomographic images of successive slices of a three-dimensional region.

  One such 3D image is a 3D image of the lung bronchi. The three-dimensional image of the bronchus is used to three-dimensionally grasp the position of an abnormal part suspected of lung cancer, for example. In order to confirm the abnormal portion by biopsy, a bronchoscope is inserted and a biopsy needle, biopsy forceps, or the like is taken out from the distal end portion and a tissue sample is taken.

In a body duct having a multi-stage branch such as the bronchi, even if the entire bronchial screen is displayed on the display unit, it is difficult to confirm and correct the insertion direction. In JP-A-135215 and the like, a three-dimensional image of a pipeline in the subject is created based on image data of a three-dimensional region of the subject, and a route along the pipeline to a target point on the three-dimensional image. And generating a virtual endoscopic image of the duct along the route based on the image data, and displaying the virtual endoscopic image, thereby navigating the bronchoscope to a target site An apparatus has been proposed.
JP 2000-135215 A

  However, in navigation to a target site using the device disclosed in Japanese Patent Laid-Open No. 2000-135215, a live endoscopic image captured by a bronchial endoscope is displayed and a virtual endoscopic image at a bronchial branch is displayed. However, as described above, the bronchus not only has multi-stage branching, but also finds a branch destination route from a plurality of branch destination routes to a route that reaches the target site by branching. Because it is difficult to create a branch image that requires more attention than other branches at the time of insertion, simply displaying a virtual endoscopic image only at a branch point makes it possible to determine whether a branch requires attention or a normal branch. This is not possible, and there is a possibility that the approach to the branch requiring attention may be hindered.

  The present invention has been made in view of the above circumstances, and provides an insertion support system that can reliably and easily notify a branch position requiring attention and reliably support insertion of an endoscope into a target site. It is an object.

The insertion support system of the present invention is
In an insertion support system for guiding an insertion path of an endoscope into a body cavity path branched into a plurality of branch paths in a subject,
An insertion path setting means for setting the insertion path based on position information of a start point and an end point;
Virtual cavity path image generation means for generating a plurality of virtual cavity path images at a plurality of path points on the insertion path set by the insertion path setting section based on image data of a three-dimensional region in the subject; ,
Attention point extraction means for extracting a plurality of attention points on the insertion path from the virtual cavity image at the start point to the virtual cavity image at the end point;
A quasi-point that designates, as a quasi-point of interest, the path point between a desired point of interest among the plurality of points of interest and a point of interest adjacent to at least the base end side of the insertion path with respect to the desired point of interest. Attention point designation means ,
Thumbnail image generating means for generating thumbnail images of the virtual cavity image of the plurality of points of interest and the quasi-point of interest;
A synthesized image generating means for generating a synthesized image including the virtual cavity image and the thumbnail images arranged from the proximal end side to the distal end side of the insertion path;
The display mode of each thumbnail image belonging to the first thumbnail image group corresponding to the plurality of attention points and the display mode of each thumbnail image belonging to the second thumbnail image group corresponding to the semi-interest points are combined. Display control means for differentiating each other on the image;
It is configured with.

  According to the present invention, there is an effect that it is possible to reliably and easily notify a branch position requiring attention and to reliably support insertion of an endoscope into a target site.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIGS. 1 to 32 relate to the first embodiment of the present invention, FIG. 1 is a block diagram showing the configuration of the bronchial insertion support system, and FIG. 2 is a flowchart showing the flow of insertion support data generation processing by the insertion support device of FIG. 3 is a first diagram showing a route setting screen developed by the processing of FIG. 2, FIG. 4 is a second diagram showing a route setting screen developed by the processing of FIG. 2, and FIG. 5 is a route of FIG. 6 is a flowchart showing the flow of setting processing, FIG. 6 is a first diagram showing a route setting screen developed by the processing of FIG. 5, and FIG. 7 is a second diagram showing a route setting screen developed by the processing of FIG. 8 is a third diagram showing the route setting screen developed by the process of FIG. 5, FIG. 9 is a first diagram for explaining the quasi-attention point addition process of FIG. 5, and FIG. 10 is the quasi-attention point addition of FIG. FIG. 11 is a second diagram for explaining the processing, FIG. 11 is a third diagram for explaining the quasi-attention point addition processing of FIG. 12 is a fourth diagram illustrating the quasi-interest point addition process of FIG. 5, FIG. 13 is a fourth diagram showing a route setting screen developed by the process of FIG. 5, and FIG. 14 is a quasi-interest point addition process of FIG. FIG. 15 illustrates a second example of a modification of the semi-interest point addition process of FIG. 5, and FIG. 16 illustrates a modification of the semi-interest point addition process of FIG. FIGS. 3 and 17 are fourth diagrams for explaining a modification of the quasi-interest point addition process of FIG. 5, and FIG. 18 is a thumbnail registration / deletion process when the root VBS image is reproduced by the insertion support apparatus of FIG. FIG. 19 is a diagram showing a patient information input image developed by the processing of FIG. 18, FIG. 20 is a first diagram showing a thumbnail image setting image developed by the processing of FIG. 18, and FIG. FIG. 22 is a second diagram showing thumbnail image setting images developed by the processing of FIG. 18, and FIG. FIG. 23 shows a thumbnail image setting image developed by the processing of FIG. 18, FIG. 23 shows a thumbnail image setting image developed by the processing of FIG. 18, and FIG. 24 shows a thumbnail developed by the processing of FIG. FIG. 25 shows the image setting image, FIG. 25 shows the thumbnail image setting image developed by the process of FIG. 18, and FIG. 26 shows the thumbnail image setting image developed by the process of FIG. 7 and FIG. 27 are eighth diagrams showing thumbnail image setting images developed by the processing of FIG. 18, and FIG. 28 is a first diagram showing an insertion support image developed during bronchial insertion by the insertion support device of FIG. FIG. 29 is a second view showing an insertion support image developed when a bronchus is inserted by the insertion support apparatus of FIG. 1, and FIG. 30 is a first view showing an insertion support image developed when a bronchus is inserted by the insertion support apparatus of FIG. Fig. 3 and Fig. 31 are diagrams. FIG. 32 is a diagram showing a modification of the thumbnail image setting image of FIG. 19 and FIG. 32 is a fourth diagram showing an insertion support image developed at the time of bronchial insertion by one insertion support device.

  As shown in FIG. 1, the bronchial insertion support system 1 according to the present embodiment includes a bronchial endoscope device 3 and an insertion support device 5.

  The insertion support device 5 generates a VBS image based on the CT image data, displays the endoscopic image obtained by the bronchial endoscopic device 3 on the monitor 7, and also displays a virtual endoscopic image (from the VBS generating device 2 ( (Hereinafter referred to as “VBS image”) is displayed on the monitor 6 and is inserted into the bronchus of the bronchial endoscopic device 3.

  Although not shown, the bronchial endoscope device 3 performs signal processing on a bronchial endoscope having an imaging unit, a light source that supplies illumination light to the bronchial endoscope, and an imaging signal from the bronchial endoscope. A bronchoscope is inserted into the bronchus in the patient's body, the inside of the bronchus is imaged to biopsy the affected tissue at the end of the bronchus, and a live image is displayed on the monitor 7. Note that the live image and the VBS image can be combined and displayed on the monitor 7.

  The monitor 7 is provided with an input unit 8 including a pointing device such as a touch panel, and a nurse such as an assistant easily operates the input unit 8 including a touch panel in accordance with the operator's instructions while the operator performs an insertion procedure. It is possible.

  The insertion support device 5 can use 3D image data generated by a known CT device (not shown) for taking an X-ray tomographic image of a patient, such as an MO (Magnetic Optical Disk) device or a DVD (Digital Versatile Disk) device. A CT image data capturing unit 21 that is captured via a portable storage medium, a CT image data storage unit 22 that stores three-dimensional image data captured by the CT image data capturing unit 21, and a CT image data storage unit 22. An MPR image generation unit 13 that generates an MPR image based on the three-dimensional image data that is generated, and a route setting screen that includes an MPR image generated by the MPR image generation unit 13 to generate a bronchus endoscopic device 3 Insertion to set a support route to the bronchus of the mirror (hereinafter simply referred to as the route) A virtual unit that generates a continuous VBS image of a route set by the route setting unit 14 based on the three-dimensional image data stored in the route setting unit 14 that is an input route setting unit and the CT image data storage unit 22 in units of frames. A VBS image generation unit 23 which is a cavity image generation unit; a VBS image storage unit 24 which stores a VBS image generated by the VBS image generation unit 23; an imaging signal from the bronchoscope of the bronchoscope device 3; An input signal from the input unit 8 is input, and an image processing unit 55 that is an attention point extracting unit that generates an insertion support image described later including a live image, a VBS image, and a plurality of thumbnail VBS images, and a route setting unit 14 generate The route setting screen and the image display control unit 56 for displaying the insertion support image generated by the image processing unit 55 on the monitor 6 and the route setting unit 14 Composed of the setting information input unit 52 is quasi-target point specifying means including a keyboard and a pointing device for inputting the setting information.

  The CT image data storage unit 22 and the VBS image storage unit 24 may be configured by a single hard disk, and the MPR image generation unit 13, the route setting unit 14, the VBS image generation unit 23, and the image processing unit 55 are A single arithmetic processing circuit can be used. The CT image data capturing unit 21 captures CT image data via a portable storage medium such as MO or DVD. However, a CT apparatus or a hospital server storing CT image data is connected to the hospital LAN. In such a case, the CT image data capturing unit 21 may be configured by an interface circuit that can be connected to the in-hospital LAN, and CT image data may be captured through the in-hospital LAN.

  As shown in FIG. 2, prior to observation / treatment by the bronchoscope apparatus 3, the insertion support apparatus 5 captures the three-dimensional image data of the patient generated by the CT apparatus by the CT image data capture unit 21 in step S1. The 3D image data captured in step S2 is stored in the CT image data storage unit 22.

  In step S <b> 3, the route setting unit 14 displays a route setting screen 521 as shown in FIG. 3 on the monitor 6, and selects patient information on the patient information tag screen 522 on the route setting screen 521. By this selection, an MPR image 523 including, for example, three different multi-sectional images of the patient selected in step S4 is generated, and this MPR image 523 is displayed on the route setting screen 521 in step S5.

  The selection of patient information on the patient information tag screen 522 is performed by inputting a patient ID for identifying a patient by the setting information input unit 52.

  Next, when a route setting tag 524 (see FIG. 3) on the route setting screen 521 is selected by the setting information input unit 52 in step S6, a route setting tag screen 525 as shown in FIG. 4 is displayed on the route setting screen 521. Then, a route setting process, which will be described later, is performed to set a bronchoscope insertion support route of the bronchial endoscope device 3 in the bronchi.

  When the insertion support route is set, continuous VBS images of all routes set by the VBS image generation unit 23 are generated in units of frames in step S7, and the VBS image generated in step S8 is stored in the VBS image storage unit 24. To do.

  With the processing of steps S1 to S9 described above, preparation for insertion support by the insertion support device 5 during observation / treatment by the bronchoscope of the bronchoscope device 3 is completed.

  Here, the route setting process in step S6 will be described with reference to FIGS.

  As shown in FIG. 5, in the route setting process of step S6, when the route search button on the route setting tag screen 525 shown in FIG. 6 is displayed on the route setting screen 521, and a cursor 530 is used on the route setting screen 521 on one tomographic image of the MPR image 523. Set the start point. When the start point is set, the start point is set at a corresponding position on the two tomographic images of the other MPR images 523, and an end point input instruction window 532 for prompting input of the end point of the route as shown in FIG. 7 is displayed on the route setting screen. 521 is displayed.

  In step S12, the end point is set on one tomographic image of the MPR image 523 by using the cursor 530 on the route setting screen 521 in the same manner as the setting of the start point. When the end point is set, the end point is set at a position corresponding to the two tomographic images of the other MPR images 523.

  When the start point and the end point are set, the route setting unit 14 searches for a route in the bronchus from the start point to the end point in step S13. Since the bronchus has a complicated route, the route in the bronchus from the start point to the end point is not necessarily uniquely determined. Therefore, in the route setting unit 14, the route in the bronchus from the start point to the end point is determined in step S 13. The first candidate is searched.

  Then, on the route setting screen 521, the route setting unit 14 superimposes and displays the route searched in step S14 on the MPR image 523 on the route setting screen 521, and in step S15, the VBS image generating unit 23 displays the route. A VBS image of a plurality of frames forming a moving image from the start point to the end point of the searched route is generated, and branch points are extracted from the VBS images of the plurality of frames generated by the image processing unit 55 in step S16. In addition to marking the branch point position of the bronchus on the route searched in S17, a route confirmation window 533 that prompts input such as route confirmation is displayed.

  As a branch point extraction method in step S16, for example, a frame in which a bronchus branch hole has reached a predetermined size on a VBS image is detected by the image processing unit 55 as a branch point VBS image by image processing. The frame position of the obtained VBS image is extracted as a branch point.

  The route confirmation window 533 includes a route confirmation button 541 for instructing confirmation of the searched route, a next candidate search button 542 for instructing a search for the next candidate route, and a route reset button 543 for resetting the start point and the end point. A cancel button 544 for canceling the route search process, and a quasi-branch point designation button 545 for inputting a quasi-interesting point designated between the branch points in order to emphasize a branch point requiring attention at the time of insertion at the branch point of the bronchus. I have.

  In step S18, it is determined whether or not the next candidate search button 542 has been clicked. If clicked, the next candidate route is automatically searched in step S19 and the process proceeds to step S20. If not clicked, the process proceeds to step S21. In step S20, as a result of searching for the next candidate, it is determined whether or not the next candidate exists. If it does not exist, a warning that there is no next candidate route is displayed, but the process returns to step S13. Return to step S14.

  In step S21, it is determined whether or not the quasi-branch point designation button 545 has been clicked. If clicked, the process proceeds to step S22. If not clicked, the process proceeds to step S23.

  In step S22, a semi-interesting point addition process is executed. Specifically, by operating the setting information input unit 52, when the quasi-branch point designation button 545 is clicked, a quasi-interest point addition window 546 as shown in FIG. 9 is developed on the route setting screen 521. . In the quasi-interest point addition window 546, a route model 548a in which the searched route is modeled, a start / end marker 548b indicating a start point and an end point, and a branch point marker 548c indicating a branch point are displayed.

  Then, as shown in FIG. 10, by operating the setting information input unit 52, the quasi attention point is specified before and after the branch point requiring attention at the time of insertion using the pointer 547 in the quasi attention point addition window 546. A point-of-interest marker 548d is displayed, and an OK button 549 is clicked to add a quasi-point of interest.

  Specifically, as shown in FIG. 11, a quasi-interesting point is specified near a branching point that requires attention at the time of insertion with respect to the frame VBS image 600 of a plurality of consecutive branching points extracted in step S16 using a pointer 547. Then, as shown in FIG. 12, the frame VBS image 601 of the quasi attention point is set between the branch points.

  When the quasi attention point is added in the quasi attention point addition window 546 in this way, the position of the quasi attention point is marked on the route of the MPR image 523 on the route setting screen 521 as shown in FIG. .

  In step S23, it is determined whether or not the route reset button 543 has been clicked. If clicked, the process returns to step S11, and if not clicked, the process proceeds to step S24.

  In step S24, it is determined whether or not the route confirmation button 541 has been clicked. If not clicked, the process returns to step S18, and if clicked, the process proceeds to step S25. And return to step S7 of FIG.

  As shown in FIG. 14, a semi-interest point number input unit 550 for setting the number of semi-interest points to be added before and after the branch point may be provided in the semi-interest point addition window 546.

  In the quasi-interest point number input unit 550, the number of quasi-interest points to be added before and after the branch point is set in the quasi-interest point input unit 550, and when a desired branch point is clicked with the pointer 547, as shown in FIG. A semi-interest point marker 548d is displayed before and after the clicked branch point marker 548c on the route model 548a.

  Specifically, as shown in FIG. 16, when a branch point requiring attention at the time of insertion is clicked using a pointer 547, frame VBS images corresponding to the number of semi-interest points set for the frame VBS image 600 at the branch point are displayed. It is specified at a predetermined frame interval. In addition, as shown in FIG. 17, the frame VBS image of the number of the quasi attention points set with respect to the frame VBS image 600 of the branch point may be a frame that equally divides between the branch points.

  In this way, the quasi-interest point can be designated in the route setting, but in the present embodiment, the quasi-interest point can be additionally designated at the time of moving image reproduction of the VBS image of the set route. The moving image reproduction of the VBS image is used as a simulation of observation / treatment of the actual bronchoscope apparatus 3.

  Next, the addition designation of the quasi-attention point at the time of moving image reproduction of the root VBS image will be described. In the following description, a case where there are seven branch points (including the insertion support start point and insertion support end point) and two semi-interest points will be described as an example.

  As shown in FIG. 18, CT image data is input via the CT image data capturing unit 21 in step S31, and the CT image data is stored in the CT image data storage unit 22 in step S32. In step S33, a VBS image for each frame of the set route in the bronchial lumen of the patient is generated based on the CT image data stored in the CT image data storage unit 22 by the VBS image generation 23, and in step S34. The VBS image generated by the VBS image generation 23 is stored in the VBS image storage unit 24.

  In step S35, the image processing unit 55 displays a patient information input image 101 as shown in FIG. 19 on the monitor 6, and each patient information (patient ID, patient name, sex, comment) of each input item designated by the pointer 100. The setting information input unit 52 waits for input.

  When patient information is input on the patient information input image 101, the frame image of the VBS image of the route set in step S36 is fetched from the VBS image storage unit 24, and the thumbnail image setting as shown in FIG. 20 is performed in step S37. The image 110 is displayed on the monitor 6.

  As shown in FIG. 20, the thumbnail image setting image 110 includes a VBS image display area 111 that displays the VBS image 120 as a full screen, and a thumbnail image display area that displays thumbnail images of the VBS image 120 at the branch point and the semi-interesting point. 112 and an error and comment display area 99. By default, the VBS image 120 at the insertion support starting point is displayed in the VBS image display area 111.

  In the error and comment display area 99, an error message is displayed when an error occurs, so that the user can be notified of the occurrence of the error or a comment can be displayed.

  In the case of FIG. 20, in the thumbnail image display area 112, the thumbnail image 112 (a) of the VBS image 120 at the insertion support start point, the thumbnail image 112 (g) of the VBS image 120 at the insertion support end point, the insertion support start point, and the insertion support. Thumbnail images 112 (b) to 112 (f) of the VBS image 120 at the branch point between the end points, and thumbnail images 125a and 125b of the quasi-attention points added before and after the branch point of the branch point name 1aα are displayed. Yes.

  When a thumbnail image that matches the VBS image 120 displayed in the VBS image display area 111 exists in the thumbnail image display area 112, the matching thumbnail image frame is set to a thick frame and the VBS in the VBS image display area 111 is displayed. The relationship between the image 120 and the thumbnail image in the thumbnail image display area 112 can be easily understood. In FIG. 20, since the VBS image 120 displayed in the VBS image display area 111 is the VBS image of the insertion support start point 103, the frame of the thumbnail image 112 (a) in the thumbnail image display area 112 is displayed in a thick frame. .

  Further, in the thumbnail image setting image 110 of the present embodiment, the semi-interest point identifying means is realized by displaying the thumbnail image of the semi-interest point with a different size from the thumbnail image of the branch point (including the start point and the end point). Yes.

  The thumbnail image setting image 110 includes a registration button 113 for newly registering a VBS frame image as a thumbnail image in the thumbnail image display area 112, a delete button 114 for deleting the registration of the thumbnail image, and before and after the thumbnail image in the thumbnail image display area 112. The previous button 115 and the next button 116 for skipping to the next image, the playback / stop button 117 for instructing to stop the playback of the moving image in the VBS image display area 111, and the speed designation bar for specifying the playback speed of the moving image in the VBS image display area 111 118, and a confirmation button 119 for confirming the thumbnail image in the thumbnail image display area 112 and storing the thumbnail image in the VBS image storage unit 24.

  Returning to FIG. 18, in step S38, as shown in FIG. 21, the VBS image of the moving image starting from the VBS image at the insertion support start point is reproduced in the VBS image display area 111 by pressing the play / stop button 117 with the pointer 100.

  Then, in step S39, as shown in FIG. 22, when the playback of the VBS image is stopped by pressing the playback / stop button 117 with the pointer 100, the process proceeds to the registration mode, and the instruction for registering the thumbnail image is received. to decide.

  In step S40, upon receiving a selection instruction from the branch hole pointer 100 into which the bronchoscope is inserted on the VBS image stopped in the VBS image display area 111, the pointer 100 is selected as shown in FIG. The insertion destination marker 131 is superimposed on the instructed VBS image.

  Further, when the registration button 113 is pressed by the pointer 100, the VBS image of the frame that is being registered that has been stopped in the VBS image display area 111 in step S41 is temporarily registered in the predetermined area of the VBS image storage unit 24 as a registered thumbnail. Store and proceed to step S42. At this time, the position information of the insertion destination marker 131 is stored as branch information in the predetermined area of the VBS image storage unit 24 together with the registered thumbnail.

  For example, as shown in FIG. 23, the VBS image stopped in the VBS image display area 111 between the branch point 1aαx1 and the branch point 1aαx1 is temporarily stored as a registered thumbnail in a predetermined area of the VBS image storage unit 24. Then, as the thumbnail image 125c of the semi-attention point newly added as the thumbnail image of the registered thumbnail, as shown in FIG. 24, the thumbnail image 112 (d) of the branch point 1aαx and the branch point 1aαx1 of the thumbnail image display area 112 are displayed. The thumbnail image 112 (e) is displayed.

  If it is determined in step S39 that the play / stop button 117 is not pressed by the pointer 100, the process directly proceeds to step S42.

  In steps S42 and S43, the deletion process is executed by selecting a thumbnail image to be deleted and operating the pointer 100 of the delete button 114. Details of this deletion will be described later.

  Then, steps S38 to S43 are repeated until a registered thumbnail of a semi-interesting point newly added at a desired position reaching the insertion support end point is obtained in step S44.

  As described above, for example, the thumbnail image 112 (a) of the insertion support start point VBS image 120 and the thumbnail image 112 (g) of the insertion support end point VBS image 120 set at the time of route setting, the insertion support start point, and the insertion support end point. Thumbnail images 112 (b) to 112 (f) of the VBS image 120 at the branch point between them, and thumbnail images 125a and 125b of the quasi-attention points added before and after the branch point of the branch point name 1aα, are newly registered In a state where the thumbnail image 125c of the quasi-attention point is displayed, it is determined that the thumbnail image has been determined when the confirm button 119 is pressed by the pointer 100 in step S45, and a predetermined area of the VBS image storage unit 24 is determined in step S46. The desired number of registered thumbnails and all VBS images on the route as video files of VBS images It ends the storing processing in the image storage unit 24. If the thumbnail image is not determined, steps S38 to S45 are repeated. Here, in the VBS image storage unit 2454, branch information is stored by being linked to each frame image of the thumbnail image.

  Next, the deletion process in steps S42 and S43 in FIG. 18 will be described. As shown in FIG. 25, when a predetermined number of registered thumbnails are stored and registered in a predetermined area of the VBS image storage unit 24, for example, when the registered frame navigation VBS image of the thumbnail image 112 (f) is deleted, step S42 is performed. The thumbnail image 112 (f) is selected with the pointer 100, so that the frame of the thumbnail image 112 (f) is displayed in a thick frame and the VBS image of the thumbnail image 112 (f) is displayed in the VBS image display area 111. The When the delete button 114 is pressed with the pointer 100 in step S43, the registered thumbnail of the thumbnail image 112 (f) is deleted as shown in FIG. 26, and the thumbnail image 112 (f) is deleted from the thumbnail image display area 112. Thus, for example, the frame of the next thumbnail image 112 (g) is displayed as a thick frame, and the VBS image of the thumbnail image 112 (g) is displayed in the VBS image display area 111. In this way, the deletion process is executed.

  The insertion support apparatus 5 uses the VBS image to store the bronchial endoscopy of the bronchial endoscope apparatus 3 in a state in which the moving image file of the VBS image having the information of the branch point and the quasi-attention point in the route set as described above is stored. Provide insertion support during the actual insertion of the mirror into the bronchus.

  Specifically, an endoscopic image (hereinafter referred to as a live image) obtained by the bronchial endoscopic device 3 is displayed on the monitor 7, and an insertion support image 210 as shown in FIG. 27 is displayed on the monitor 6. This insertion support image 210 has a VBS image display area 111 and a thumbnail image display area 112.

  In the case of FIG. 27, in the thumbnail image display area 112, the thumbnail image 112 (a) of the VBS image 120 at the insertion support start point, the thumbnail image 112 (g) of the VBS image 120 at the insertion support end point, the insertion support start point, and the insertion support. Thumbnail images 112 (b) to 112 (f) of the VBS image 120 at the branch point between the end points, and thumbnail images 125a and 125b of the quasi-attention points added before and after the branch point of the branch point name 1aα are displayed. Yes.

  In the insertion support image 210 of the present embodiment, the semi-interest point identifying means is realized by displaying the thumbnail image of the semi-interest point in a different size from the thumbnail image of the branch point (including the start point and the end point). .

  The insertion support image 210 is provided with a left button 121, a right button 122, a previous button 115, and a next button 116. By selecting the next button 116 with the cursor 100 using the input unit 8, a VBS image display area is displayed. The VBS image 120 displayed in 111 can be skipped to the VBS image of the next thumbnail image, and the VBS image displayed in the VBS image display area 111 can be selected by selecting the previous button 115 with the cursor 100. The image 120 can be skipped to the VBS image of the previous thumbnail image.

  Further, the VBS image 120 can be continuously rotated to the left by selecting the left button 121 with the cursor 100, and the VBS image 120 can be continuously rotated to the right by selecting the right button 122 with the cursor 100. Can be rotated.

  Further, in the present embodiment, it is possible to additionally specify a quasi attention point even during actual insertion of the bronchoscope into the bronchus using the insertion support image 210. For this reason, the insertion support image 210 includes a quasi attention point addition button. 141 and a semi-interest point deletion button 140 are provided.

  As shown in FIG. 27, when the bronchoscope of the bronchoscope device 3 is inserted into the bronchus, the insertion support image 210 selects a thumbnail image in the thumbnail image display area 112 and the VBS image display area 111. Since the VBS frame image of the thumbnail image is displayed on the VBS frame image and the insertion destination marker 131 on the VBS frame image can be referred to, the insertion hole indicated by the insertion destination marker 131 is found on the live image of the monitor 7 and the insertion operation is performed. Thus, in the bronchial insertion technique, it is possible to reliably and easily insert the bronchoscope into the appropriate route to the site of interest such as the affected part at the insertion support end point.

  When the VBS image displayed in the VBS image display area 111 is added to the quasi-attention point during the insertion support at the time of actual insertion by the insertion support image 210, as shown in FIG. If the displayed VBS image is a frame image between the thumbnail image 112 (e) and the thumbnail image 112 (f), when the semi-interest point addition button 141 is selected with the pointer 100 in this state, as shown in FIG. The VBS image displayed in the VBS image display area 111 is added to the quasi attention point, and the quasi attention point added between the thumbnail image 112 (e) and the thumbnail image 112 (f) in the thumbnail image display area 112 is added. A thumbnail image 125x of the VBS image is displayed.

  When it is not necessary to display the thumbnail image of the semi-interest point in the thumbnail image display area 112, the semi-interest point deletion button 140 is selected with the pointer 100, so that the thumbnail image display area is displayed as shown in FIG. From 112, it is possible to delete the thumbnail image of the semi-attention point. Note that, when deleted, the space 180 remains in the portion where the thumbnail image of the quasi-attention point was displayed, so that it is possible to visually recognize that the branch point where the space 180 is before and after is a branch point requiring attention. .

  As described above, in this embodiment, at the time of route setting (route setting screen), at the time of VBS image playback after route setting (simulation: thumbnail image setting image), and further at the time of actual insertion support (during navigation: insertion support image) Regardless of the timing, a quasi-attention point for emphasizing the bifurcation point that requires attention for bronchial insertion support is registered, and a thumbnail image of the quasi-attention point is displayed. And it can be easily notified and visually recognized.

  In the thumbnail image setting image 110 and the insertion support image 210, as a semi-interest point identification unit, the thumbnail image of the semi-interest point is displayed in a different size from the thumbnail image of the branch point (including the start point and the end point). Although the thumbnail image at the semi-interest point in the thumbnail image display area 112 can be identified from the thumbnail image at the branch point, the present invention is not limited to this. For example, a thumbnail image at the semi-interest point as in the thumbnail image setting image 110 shown in FIG. A semi-interest point identifying unit 190 may be realized by displaying a semi-interest point marker 190 indicating a semi-interest point on the upper part of the image (the same applies to the insertion support image 210: not shown).

  In addition, the thumbnail image display area 112 always displays all thumbnail images of the branch point (including the start point and the end point) and the semi-interesting point. However, the present invention is not limited to this. For example, as shown in FIG. When the VBS image displayed in the display area 111 reaches a quasi-attention point, a thumbnail image of a branch point in which the quasi-attention point is a cautionary branch point, and thumbnail images of quasi-attention points before and after the thumbnail image Only the thumbnail image display area 112 may be displayed.

  In this embodiment, at the time of route setting (route setting screen), VBS image playback after route setting (simulation: thumbnail image setting image), and further actual insertion support (navigation: insertion support image) Regardless of the timing, a quasi-attention point for emphasizing a branch point that requires attention for bronchial insertion support is registered and a thumbnail image of the quasi-attention point is displayed. Pay attention to support for insertion into the bronchi at least at the timing of setting screen), VBS image playback after route setting (simulation: thumbnail image setting image), and actual insertion support (navigation: insertion support image) This is also possible even if a semi-interest point for emphasizing a bifurcation point that requires is registered and a thumbnail image of the semi-interest point is displayed. Needless to say it is possible to obtain the same advantages as the.

  The present invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the scope of the present invention.

The block diagram which shows the structure of the bronchial insertion assistance system which concerns on Example 1 of this invention, The flowchart which shows the flow of the production | generation process of the insertion assistance data by the insertion assistance apparatus of FIG. The 1st figure which shows the route setting screen expand | deployed by the process of FIG. 2nd figure which shows the route setting screen expand | deployed by the process of FIG. The flowchart which shows the flow of the route setting process of FIG. 1st figure which shows the route setting screen expand | deployed by the process of FIG. 2nd figure which shows the route setting screen expand | deployed by the process of FIG. 3rd figure which shows the route setting screen expand | deployed by the process of FIG. FIG. 5 is a first diagram for explaining the quasi-attention point addition process of FIG. FIG. 5 is a second diagram illustrating the quasi-attention point addition process of FIG. The 3rd figure explaining the quasi attention point addition process of FIG. FIG. 4 is a fourth diagram for explaining the quasi-attention point addition process of FIG. 4th figure which shows the route setting screen expand | deployed by the process of FIG. The 1st figure explaining the modification of the quasi attention point addition process of FIG. FIG. 5 is a second diagram for explaining a modified example of the semi-interest point addition processing of FIG. FIG. 5 is a third diagram for explaining a modified example of the semi-interest point addition processing of FIG. FIG. 4 is a fourth diagram for explaining a modified example of the semi-interest point addition processing of FIG. 5; The flowchart which shows the flow of the thumbnail registration / deletion process at the time of reproduction | regeneration of the root | route VBS image by the insertion assistance apparatus of FIG. The figure which shows the patient information input image developed by the process of FIG. FIG. 18 is a first diagram showing thumbnail image setting images developed by the processing of FIG. The 2nd figure which shows the thumbnail image setting image developed by the process of FIG. The third figure which shows the thumbnail picture setting picture which is developed with the processing of Figure 18 FIG. 18 is a fourth diagram showing thumbnail image setting images developed by the processing of FIG. FIG. 5 shows a thumbnail image setting image developed by the processing of FIG. FIG. 6 shows a thumbnail image setting image developed by the processing of FIG. FIG. 7 shows a thumbnail image setting image developed by the processing of FIG. FIG. 18 is an eighth diagram showing thumbnail image setting images developed by the processing of FIG. The 1st figure which shows the insertion assistance image developed at the time of bronchial insertion by the insertion assistance apparatus of FIG. The 2nd figure which shows the insertion assistance image developed at the time of bronchial insertion by the insertion assistance apparatus of FIG. The 3rd figure which shows the insertion assistance image developed at the time of bronchial insertion by the insertion assistance apparatus of FIG. 4th figure which shows the insertion assistance image developed at the time of bronchial insertion by the insertion assistance apparatus of FIG. The figure which shows the modification of the thumbnail image setting image of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Bronchial insertion assistance system 2 ... VBS production | generation apparatus 3 ... Bronchoscope apparatus 5 ... Insertion assistance apparatus 6, 7 ... Monitor 8 ... Input part 21 ... CT image data acquisition part 22 ... CT image data storage part 23 ... VBS image Generation 24 ... VBS image storage unit 51 ... VBS image capture unit 52 ... Setting information input unit 55 ... Image processing unit 56 ... Image display control unit Agent Patent attorney Susumu Ito

Claims (6)

In an insertion support system for guiding an insertion path of an endoscope into a body cavity path branched into a plurality of branch paths in a subject,
An insertion path setting means for setting the insertion path based on position information of a start point and an end point;
Virtual cavity path image generation means for generating a plurality of virtual cavity path images at a plurality of path points on the insertion path set by the insertion path setting section based on image data of a three-dimensional region in the subject; ,
Attention point extraction means for extracting a plurality of attention points on the insertion path from the virtual cavity image at the start point to the virtual cavity image at the end point;
A quasi-point that designates, as a quasi-point of interest, the path point between a desired point of interest among the plurality of points of interest and a point of interest adjacent to at least the base end side of the insertion path with respect to the desired point of interest. Attention point designation means ,
Thumbnail image generating means for generating thumbnail images of the virtual cavity image of the plurality of points of interest and the quasi-point of interest;
A synthesized image generating means for generating a synthesized image including the virtual cavity image and the thumbnail images arranged from the proximal end side to the distal end side of the insertion path;
The display mode of each thumbnail image belonging to the first thumbnail image group corresponding to the plurality of attention points and the display mode of each thumbnail image belonging to the second thumbnail image group corresponding to the quasi-point of interest are combined. Display control means for differentiating each other on the image;
An insertion support system characterized by comprising: The display control means makes the size of each thumbnail image belonging to the first thumbnail image group different from the size of each thumbnail image belonging to the second thumbnail image group on the composite image. The insertion support system according to claim 1, wherein: The insertion support system according to claim 1, wherein the display control unit adds a marker to each thumbnail image belonging to the second thumbnail image group . The composite image generating means, when an instruction to hide each thumbnail image belonging to the second thumbnail image group is given, there is a blank in the arrangement position of the second thumbnail image group on the composite image. Delete the second thumbnail image group from the composite image so that it remains
The insertion support system according to claim 1 or 2, characterized in that. The attention point is a branch point on the insertion path
The insertion support system according to any one of claims 1 to 4, wherein The semi-interest point designating unit designates a plurality of the semi-interest points.
The insertion support system according to any one of claims 1 to 5, wherein

Images