JP2018166932A - Endoscope system - Google Patents

Abstract

To display a specific part, such as a special light observation image, in a three-dimensional image.SOLUTION: In an endoscope system which can display an insertion part shape of an insertion part 10M as a three-dimensional insertion part image IA by a sensor coil 16 in a video scope 10 and an antenna unit 50 for shape detection, when a special light observation mode is set, a three-dimensional organ image IB to be an appearance shape of an organ to be photographed is displayed at the same time. For the three-dimensional organ image IB, a position of an emphasis part EI of the special light observation image is identified in the three-dimensional organ image IB, and an emphasis part FI is displayed overlapped.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an endoscope system that observes a subject such as an organ using a scope (endoscope), and more particularly to display of an observation image.

  In the endoscope system, it is possible to irradiate an observation site with light in a specific wavelength region (special light) and extract a portion estimated to be a lesion from the observation image. Then, for example, an observation image in which the portion is emphasized with a color component different from the surroundings is displayed so that the extracted region can be identified (see, for example, Patent Document 1).

JP 2012-139482 A

  An observation image based on special light or the like is a two-dimensional image, and even if a portion that is assumed to be a lesion is identified and emphasized in the observation image, only a small part in the organ is specified. It is impossible to grasp which part the identified and emphasized part corresponds to in an organ having a target shape.

  Therefore, it is required to grasp a place where inflammation or the like is estimated or the whole organ from a two-dimensional observation image based on special light or the like.

  The present invention provides an observation image processing unit that identifies a predetermined portion in an observation image obtained by endoscopic imaging, an insertion unit shape detection unit that detects the shape of an endoscope insertion unit, and a detected insertion unit shape. And a three-dimensional image processing unit that displays the external shape of the subject to be imaged as a three-dimensional image. For example, the three-dimensional image processing unit can simultaneously display the three-dimensional shape image of the endoscope insertion unit and the three-dimensional image of the subject.

  Then, the three-dimensional image processing unit of the present invention identifies a part corresponding to the identified part in the three-dimensional image based on the position of the distal end of the endoscope and the orientation of the observation image, and identifies the part. indicate. For example, an observation image obtained by an endoscope is recorded in accordance with the insertion amount of the endoscope insertion unit or a predetermined time interval, and the three-dimensional image processing unit is configured to generate a three-dimensional image based on the recorded observation image. A location corresponding to the extracted portion can be identified.

  For example, the endoscope system corrects the orientation of the observation image based on the orientation of the observation image that serves as a reference, and an image direction detection unit that detects a change in the orientation of the observation image according to a change in the posture of the endoscope tip. An image correction processing unit.

  The image direction detection unit can detect a change in the orientation of the observation image due to the rotation of the endoscope tip portion around the axis. For example, the image direction detection unit detects a change in the direction of the observation image based on an acceleration sensor provided at the distal end portion of the endoscope. The image correction processing unit may be based on the orientation of the observation image at the start of insertion of the endoscope insertion unit.

  The three-dimensional image processing unit can display an index representing the quantitative value of the identified part together with the three-dimensional image.

  For example, an endoscope system includes an illumination unit that can irradiate a plurality of special lights having different wavelength ranges. In the special light observation mode, the observation image processing unit can extract a portion satisfying a predetermined condition from the special light observation image obtained by the selected special light.

  In addition, the illumination unit can illuminate the subject while switching a plurality of special lights, and the three-dimensional image processing unit generates a plurality of three-dimensional images corresponding to the plurality of special lights and extracts the three-dimensional images. The portions can be displayed so that they can be identified in their respective three-dimensional images.

  An image processing apparatus according to another aspect of the present invention includes a three-dimensional image processing unit that displays a three-dimensional image of an appearance shape of a subject to be photographed based on a detected endoscope insertion unit shape, and an endoscope. An image direction detection unit that adjusts the direction of an observation image that identifies a predetermined portion in an observation image obtained by shooting, and a three-dimensional image processing unit that performs identification based on the position of the endoscope tip and the direction of the observation image A location corresponding to the converted portion is specified in the three-dimensional image, and the location is identified and displayed.

  Thus, according to the present invention, a specific portion such as a special light observation image can be displayed in the three-dimensional image.

It is a block diagram of the endoscope system in a 1st embodiment. It is the figure which showed special observation image IM with which the specific pixel part EI was emphasized. It is the figure which showed the display screen of the monitor for three-dimensional displays. It is the figure which showed the three-dimensional insertion part image and three-dimensional organ image which are displayed on a monitor with progress of the insertion operation of a video scope. It is the figure which showed the change of direction of the special light observation image by the attitude | position change of a scope front-end | tip part. It is the figure which showed the correspondence of the emphasis part of a special light observation image, and the emphasis part of a three-dimensional organ image. It is the figure which showed acquisition of the lesioned part when the scope insertion part 10M is pulled back. It is a flowchart of the three-dimensional image processing performed with the image processing apparatus 70 at the time of special light observation mode. It is the figure which showed three-dimensional organ image IA, IB, and IC when mode A, mode B, and mode C are set, respectively. It is the figure which showed the three-dimensional organ image in 2nd Embodiment. 6 is a timing chart showing switching of special light in a special light observation mode.

  Below, the endoscope system which is this embodiment is demonstrated with reference to drawings.

  FIG. 1 is a block diagram of an endoscope system according to the first embodiment.

  The endoscope system 100 includes an endoscopic measure including a video scope 10 and a processor 30 to which the video scope 10 is detachably connected. When the affected part is observed, treated, etc., the video scope 10 is inserted. Part 10M is inserted into the body. The processor 30 is connected to a monitor 40A that displays an observation image. The operator can bend the scope tip 10T vertically and horizontally by operating the scope operating unit 10P.

  The light emitted from the light source 42 provided in the processor 30 is guided to the distal end portion 10T of the video scope 10 by the light guide 11 provided in the video scope 10, and is directed from the distal end portion 10T toward the subject (organ inner wall). Is irradiated. The reflected light from the subject forms an image on the imaging device 12 provided at the distal end portion 10T, thereby forming a subject image. The imaging device 12 such as a CCD or CMOS image sensor is an on-chip color filter type imaging device here, and a color filter array composed of R, G, B, etc. is provided on the light receiving surface of the imaging device 12. Yes.

  In the imaging device 12, pixel signals for one field or one frame are read at a predetermined time interval. The read pixel signal is sent to the image processing circuit 36 via the drive / process circuit 13, the drive / process circuit 32 of the processor 30 and the system control circuit 40. In the image processing circuit 36, color conversion processing, white balance adjustment processing, and the like are performed on the pixel signal to generate a color image signal. By outputting the color image signal to the monitor 40A, the color observation image is displayed on the monitor 40A.

  The system control circuit 40 controls the operation of the processor 30 and outputs a control signal to the image processing circuit 36 and the like. A timing generator (not shown) outputs a clock pulse signal that adjusts the output timing of each circuit. The system control circuit 40 performs data mutual communication with the system control circuit 76 of the image processing circuit 70.

  In the processor 30, a filter mechanism 44 is provided between the light source 42 and the incident end of the light guide 11. The filter mechanism 44 includes a disk-shaped rotary filter (not shown), and an opening is formed along the circumferential direction, and a plurality of color filters that transmit light in different wavelength ranges are arranged along the circumferential direction. They are arranged at predetermined intervals.

  Here, three color filters that respectively transmit narrow band light corresponding to blue, narrow band light corresponding to green, and narrow band light corresponding to infrared light are arranged at predetermined intervals. A filter driving unit (not shown) of the filter mechanism 44 can rotate the rotary filter in accordance with a predetermined amount or one field / frame period.

  The front panel 37 of the processor 30 has a mode switching button 38 for switching between the normal observation mode and the special light observation mode, a button 38A for selecting the type of special light in the special light observation mode, and a specific portion by image editing processing. An image enhancement button 39 for displaying the enhanced observation image is provided. In the normal observation mode, white light emitted from the light source 42 passes through the opening of the rotary filter and enters the incident end of the light guide 11.

  On the other hand, when the special light observation mode is set, the rotary filter rotates by a predetermined angle, and the white light from the light source 42 passes through one of the color filters. Thereby, the subject is irradiated with narrowband light. The pixel signal read from the imaging device 12 is sent to the characteristic light image processing circuit 34 of the processor 30. The special light image processing circuit 34 generates a color image signal having a hue different from that of the normal observation image, and extracts a predetermined portion from the pixel signal for one field / frame by arithmetic processing.

  Specifically, a pixel portion having a pixel value satisfying a predetermined condition is extracted from pixel signals for one field / frame. Here, pixels whose pixel values of a specific color component are higher or lower than a threshold are extracted. For example, a portion where the value of Red Density is higher than a threshold is specified in the observation image. When a pixel portion satisfying the predetermined condition is extracted, an image enhancement process is performed so that the pixel portion is identified in the observation image. Thereby, the special light observation image in which the specific portion is emphasized is displayed on the monitor 40A.

  FIG. 2 is a diagram showing the special observation image IM in which the specific pixel portion EI is emphasized. Note that the filter mechanism 44 may be rotationally controlled so that the normal observation image and the special light observation image using white light are simultaneously displayed on the monitor 40A.

  On the other hand, the endoscope system includes a shape detection antenna unit 50 that generates a magnetic field and an image processing device 70 that generates a support three-dimensional image as an endoscope work support system. The image processing device 70 detects the insertion portion shape of the video scope 10 and displays the insertion portion shape on the monitor 60 as a three-dimensional image.

  The insertion part of the video scope 10 is provided with a plurality of sensor coils 16 and outputs a magnetic detection signal when a magnetic field is generated by the shape detection antenna unit 50. The system control circuit 76 detects the relative position of the sensor coil in the insertion portion of the video scope 10 based on the detected magnetism, and thereby detects the shape of the portion inserted into the body. Then, the image processing circuit 74 displays the external shape of the insertion portion of the video scope 10 on the monitor 60 as a three-dimensional image based on the detected insertion portion shape.

  FIG. 3 is a diagram showing a display screen of the monitor 60. In the present embodiment, when the special observation mode is set, the appearance shape of the organ to be observed together with the three-dimensional image (hereinafter referred to as the three-dimensional insertion portion image) IA indicating the appearance shape of the insertion unit 10M and the patient model IC. A three-dimensional image IB (hereinafter referred to as a three-dimensional organ image) IB is simultaneously displayed on the monitor 60.

  The three-dimensional organ image IB is displayed by a known computer graphic (CG) process based on the three-dimensional insertion portion image IA. Here, the three-dimensional organ image IB is an image when the large intestine is observed, and a tube-like image having a diameter corresponding to the organ is created based on the insertion portion shape data, Then, the image is corrected so that a graphic image in which the appearance shape is shaded is generated.

  In the three-dimensional organ image IB, a portion FI (hereinafter referred to as an emphasized portion) FI corresponding to the specific pixel portion EI identified and enhanced by the special light image processing circuit 34 of the processor 30 is identified from other portions. Thus, the image is displayed on the three-dimensional organ image IB. Here, the emphasized portion FI is displayed by making the color different from that of the other organ outer surface (monochrome or the like).

  Hereinafter, the process of displaying the emphasized portion FI of the special light observation image at the corresponding portion of the three-dimensional organ image will be described with reference to FIGS.

  FIG. 4 is a diagram showing a three-dimensional insertion portion image and a three-dimensional organ image displayed on the monitor as the video scope insertion operation progresses.

  The operator inserts the scope insertion unit 10M into the target organ (here, the large intestine) with the start of the endoscopic work. The image processing device 70 detects the position of the scope distal end portion 10T based on the magnetic signal from the sensor coil 16 disposed in the scope insertion portion 10M, calculates the insertion portion shape, and displays the three-dimensional insertion portion image IA. . The three-dimensional insertion portion image IA is displayed in a range from a position where insertion of the scope distal end portion 10T is started to a position where the current scope distal end portion 10T is advanced.

  The three-dimensional organ image IB is displayed with an appearance shape that matches the shape change of the three-dimensional insertion portion image IA, that is, the progress of the scope distal end portion 10T. As the scope tip 10T advances into the symmetric organ, the total length of the three-dimensional organ image IB becomes longer. Then, simultaneously with the display of the three-dimensional organ image IB with the passage of time, the emphasized portion FI is displayed for the corresponding portion of the emphasized portion EI of the special light observation image displayed on the monitor 40A.

  FIG. 5 is a diagram showing a change in the direction of the special light observation image due to a change in the attitude of the scope tip. FIG. 6 is a diagram showing a correspondence relationship between the enhanced portion of the special light observation image and the enhanced portion of the three-dimensional organ image.

  The operator inserts the scope insertion portion 10M into the target organ while curving the scope distal end portion 10T by operating the scope operation portion 10M. Meanwhile, the data of the special light observation image generated by the special light image processing circuit 34 of the processor 30 is transmitted to the system control circuit 76 of the image processing apparatus 70 and recorded in a memory (not shown) at predetermined time intervals. However, it is also possible to record for every fixed insertion length of the scope tip 10T.

  Since the scope insertion unit 10M moves along the shape of the target organ while the operator performs an endoscope operation, the shape of the insertion unit is complicatedly bent. Further, in the process of operating the scope insertion portion 10M, there is a case where an operation of moving the scope distal end portion 10T forward is performed while rotating the scope insertion portion 10M around the axis. Therefore, the posture of the scope distal end portion 10T changes depending on the operation method, and the orientation of the observation image changes accordingly. That is, the up / down / left / right directions of the displayed observation image change.

  When the scope insertion unit 10M is inserted into the target organ GA having the lesion GB as shown in FIG. 5, the display position of the emphasized portion EI in the special light observation image changes depending on the posture of the scope tip 10T. More specifically, if the posture around the central axis with respect to the visual field direction (perpendicular to the paper surface) of the scope distal end portion 10T does not change, the vertical and horizontal directions of the observation image are the same as at the start of insertion. On the other hand, when the scope distal end portion 10T is rotated 180 degrees from the start of insertion due to rotation or twisting of the scope insertion portion 10M, the orientation of the observation image is reversed vertically and horizontally.

  The acceleration sensor 14 provided at the scope tip 10T can detect angular acceleration with respect to a plane along the scope tip 10T, and outputs a detection signal when the scope tip 10T rotates about its axis. The special light image processing circuit 72 of the image processing device 70 corrects the orientation of the observation image based on the angular acceleration information sent from the processor 30. Here, the posture of the scope distal end portion 10T at the start of insertion is defined as a reference position (angle), and the orientation of the observation image is adjusted to the reference position in accordance with a change in angle from the reference position.

  For example, as shown in FIG. 5, when the scope distal end 10T is rotated 180 degrees with respect to the reference position, the vertical direction is corrected (adjustment adjustment) to a special light observation image converted by 180 degrees. As a result, the display position of the emphasized portion EI becomes a display position that matches the orientation of the observation image at the reference position. The image processing circuit 74 specifies and displays the position of the emphasized portion FI in the three-dimensional organ image IB based on the special observation image data in which the direction of the observation image is matched with the reference position. The three-dimensional organ image IB is stored in a memory or the like in the image processing device 70.

  FIG. 6 is a diagram showing the correspondence between the three-dimensional organ image and the special light observation image. As described above, special light observation images are recorded at predetermined time intervals. The objective optical system provided at the scope distal end 10T has an angle of view corresponding to a fisheye lens, and if the position of the scope distal end 10T is considered to be near the center of the target organ, the objective optical system is displayed at the corner of the screen of the observation image. It is possible to determine the distance interval between the position to be recorded and the scope tip portion 10T.

  Therefore, when the emphasis portion EI is projected near the screen corner of the special light observation image, the emphasis portion EI in the three-dimensional organ image IB is determined based on the position of the scope tip 10T detected by the sensor coil 16. Applicable locations can be defined. At this time, the special light observation image IM uses an image whose vertical direction is aligned with the vertical direction of the reference position.

  The enhancement portion FI of the three-dimensional organ image IB is defined by the overall length of the enhancement portion EI displayed in the special light observation image IM and the number of fields / frames n in which the enhancement portion EI is continuously observed. It is done. Since the emphasized portion EI appears in the special light observation image IM in the insertion process of the scope insertion unit 10M, the emphasized portion FI is displayed so as to be superimposed on the three-dimensional organ image IB in accordance with the appearance location.

  FIG. 7 is a diagram illustrating acquisition of a lesion when the scope insertion unit 10M is pulled back. The operator not only always advances the scope tip 10T, but also pulls it back once in some cases and observes the same portion again. In this case, an image region that is not detected when the vehicle is moving forward may be extracted.

  FIG. 7 shows that after the scope insertion unit 10M moves forward in the target organ GA, an area corresponding to the lesion GB is extracted, and then a different area GB is acquired by pulling back. By detecting the position of the scope tip 10T, the newly extracted emphasized portion is overwritten or overlaid with respect to the data of the special observation image corresponding to the position. As a result, the emphasized portion is displayed in the three-dimensional observation image IB.

  FIG. 8 is a flowchart of three-dimensional image processing executed by the image processing device 70 in the special light observation mode.

  When the normal observation mode is switched to the special light observation mode (S101, S102), three-dimensional image processing is executed based on the special light selected from the three special lights. Here, special light observation for irradiating special light (narrowband light) according to blue is mode A (S103, S105), and special light observation for irradiating special light (narrowband light) according to green is mode B ( S104, S106), special light observation for irradiating special light (narrowband light) corresponding to infrared light is defined as mode C (S107). An operator can select a mode by operating a touch panel button.

  FIG. 9 is a diagram showing three-dimensional organ images IA, IB, and IC when mode A, mode B, and mode C are set. By irradiating special light having different wavelength ranges, the displayed positions of the highlighted portions KA, KB, and KC are different.

  As described above, according to the present embodiment, in the endoscope system capable of displaying the insertion portion shape of the insertion portion 10M as the three-dimensional insertion portion image IA by the sensor coil 16 and the shape detection antenna unit 50 in the video scope 10. When the special light observation mode is set, the three-dimensional organ image IB that is the external shape of the organ to be imaged is displayed simultaneously. Then, with respect to the three-dimensional organ image IB, the position of the emphasized portion EI of the special light observation image is specified in the three-dimensional organ image IB, and the emphasized portion FI is superimposed and displayed.

  By such a three-dimensional image display, it is possible to grasp the position and size of the highlighted and identified part in the whole organ. Further, when inspecting again, the operator can promptly move the distal end of the scope to the position while confirming the position of the emphasized part by reproducing and displaying the three-dimensional organ image in advance.

  For the orientation of the observation image, a sensor (angular velocity sensor) other than the acceleration sensor may be used. Further, instead of the special light observation image, the normal observation image using white light can be applied to a case where a specific portion is identified and emphasized by predetermined image processing. Further, it is possible to detect the position of the distal end portion of the scope using an acceleration sensor or the like without using a sensor coil, and a configuration for displaying only a three-dimensional organ image is also possible.

  Next, an endoscope system according to the second embodiment will be described with reference to FIG. In the second embodiment, quantitative data of the emphasized portion of the three-dimensional organ image is displayed as an index.

  FIG. 10 is a view showing a three-dimensional organ image in the second embodiment. When a portion where the pixel signal data satisfies a predetermined condition is extracted from the special light observation image, a quantitative value of the extracted portion is simultaneously displayed as an index. Here, the numerical value IK of Red Density is displayed at the same time. By displaying eigenvalues that differ depending on the place as an index, a doctor or the like can determine the degree of disease state, progress, etc. of the emphasized portion.

  Next, an endoscope system according to a third embodiment will be described with reference to FIG. In the third embodiment, the subject is illuminated while sequentially switching the three special lights.

  FIG. 11 is a timing chart showing switching of special light in the special light observation mode. By driving the rotary filter, the special light mode A, mode B, and mode C are switched at predetermined time intervals. Accordingly, the special light corresponding to blue, the special light corresponding to green, and the special light corresponding to infrared light are switched and irradiated.

  The special light image processing circuit 72 corrects the orientation of the observation image in each mode according to the detected angle of the scope tip 10T. Then, the image processing circuit 74 generates and records three-dimensional organ images corresponding to the three special lights. At the same time, the emphasized portions respectively extracted according to the three special lights are displayed superimposed on the three-dimensional organ image. Three three-dimensional organ images may be displayed simultaneously on the screen, or only the selected three-dimensional organ image may be displayed.

10 Videoscope 14 Acceleration sensor (image direction detection unit)
30 processor 70 image processing device 72 special light image processing circuit 74 image processing circuit

Claims (11)

An observation image processing unit for identifying a predetermined portion in an observation image obtained by endoscopic imaging;
An insertion portion shape detection unit for detecting the shape of the endoscope insertion unit;
A three-dimensional image processing unit that displays a three-dimensional image of the appearance of a subject to be imaged based on the detected endoscope insertion unit shape;
The three-dimensional image processing unit identifies a portion corresponding to the identified portion in the three-dimensional image based on the position of the endoscope tip and the orientation of the observation image, and identifies and displays the portion. Endoscope system characterized by. An image direction detection unit that detects a change in the orientation of the observation image in accordance with a change in posture of the endoscope tip,
The endoscope system according to claim 1, further comprising an image correction processing unit that corrects the orientation of the observation image based on the orientation of the reference observation image.   The endoscope system according to claim 2, wherein the image direction detection unit detects a change in the orientation of the observation image accompanying the rotation of the endoscope tip portion around the axis.   The endoscope system according to claim 2 or 3, wherein the image direction detection unit detects a change in the direction of the observation image based on an acceleration sensor provided at the distal end portion of the endoscope.   The endoscope system according to any one of claims 2 to 4, wherein the image correction processing unit is based on a direction of an observation image at the start of insertion of the endoscope insertion unit.   The endoscope system according to claim 1, wherein the three-dimensional image processing unit displays an index representing a quantitative value of the identified portion together with the three-dimensional image.   7. The three-dimensional image processing unit simultaneously displays a three-dimensional shape image of the endoscope insertion unit and a three-dimensional image of a subject. The endoscope system described. A recording processing unit for recording an observation image by the endoscope according to an insertion amount of the endoscope insertion unit or a predetermined time interval;
The said three-dimensional image processing part specifies the location applicable to the part extracted in the three-dimensional image based on the recorded observation image, The one in any one of Claim 1 thru | or 7 characterized by the above-mentioned. Endoscope system. Equipped with a lighting unit that can irradiate multiple special lights with different wavelength ranges,
The endoscope system according to any one of claims 1 to 8, wherein the observation image processing unit extracts a portion satisfying a predetermined condition from the special light observation image obtained by the selected special light. . The illumination unit illuminates the subject while switching a plurality of special lights,
The three-dimensional image processing unit generates a plurality of three-dimensional images corresponding to a plurality of special lights, and displays the extracted portions so as to be identified into the respective three-dimensional images. The endoscope system according to 9. A three-dimensional image processing unit that displays a three-dimensional image of the appearance of a subject to be imaged based on the detected endoscope insertion unit shape;
An image direction detection unit that adjusts the orientation of the observation image in which a predetermined portion is identified in the observation image obtained by endoscopic imaging; and
The three-dimensional image processing unit identifies a portion corresponding to the identified portion in the three-dimensional image based on the position of the endoscope tip and the orientation of the observation image, and identifies and displays the portion. An image processing apparatus.

Images