JP5763893B2 - Image processing system and program, and operation method of endoscope system - Google Patents

Description

The present invention relates to an image processing method for operating a system及beauty program and endoscope system for correcting the image quality of a fluorescent video obtained when receiving fluorescence generated by irradiation of the excitation light to the subject.

  In the medical field in recent years, many diagnoses and treatments using an electronic endoscope have been performed. The electronic endoscope includes an elongated insertion portion that is inserted into the body cavity of a subject, and an imaging device such as a CCD is built in the distal end of the insertion portion. The electronic endoscope is connected to a light source device that emits broadband light such as white light, and the broadband light emitted from the light source device is irradiated from the distal end of the insertion portion to the inside of the body cavity. In this state where the light is irradiated inside the body cavity, the subject tissue in the body cavity is imaged by the imaging element provided at the distal end portion of the insertion portion. An image obtained by imaging is displayed on a monitor after various processing is performed by a processor device connected to the electronic endoscope.

  A white light source such as a xenon lamp capable of emitting white broadband light having a wavelength ranging from a blue region to a red region is used for the light source device. The entire subject tissue can be grasped from the broadband optical image obtained by irradiating the body cavity with white broadband light. In addition, by rotating a rotary filter having an RGB color filter at a constant period on an optical path of a white light source, light of three colors of RGB is irradiated in a field sequential manner. In this way, the entire subject tissue can also be grasped from a broadband light image obtained by irradiating light of three colors of RGB in a frame sequential manner.

  On the other hand, a tumor affected part or the like formed on the inner wall of a patient's body cavity may be difficult to grasp from a broadband optical image. In order to cope with such a case, in recent years, by using a diagnostic method called PDD (Photodynamic Diagnosis: photodynamic diagnosis), fluorescent light is generated from the tumor affected part, and by imaging the generated fluorescent light, A fluorescent image different from the broadband image is generated. In this fluorescent image, the tumor affected part is emitting fluorescence, so that the position, size, and range of the tumor affected part can be easily grasped.

  However, since the fluorescence light intensity is weak, it is necessary to ensure that the fluorescence can be imaged by making the exposure time longer than usual and lowering the frame rate. However, by increasing the exposure time or lowering the frame rate in this way, the image quality of the fluorescent image is degraded. As an example of image quality degradation, blurring may occur in the movement of the subject. In order to solve such a problem, Patent Document 1 prevents the deterioration of the image quality of the fluorescent image by detecting the motion vector of the fluorescent image between the fluorescent images to compensate for the motion of the fluorescent image.

Japanese Patent Application Laid-Open No. 07-250804

  However, since the method of Patent Document 1 originally calculates a motion vector based on a fluorescent image with poor image quality, the calculated motion vector often has poor accuracy. Thus, when motion compensation is performed using a motion vector with poor accuracy, the image quality of the fluorescent image cannot be sufficiently improved.

The present invention is, when correcting the image quality of the fluorescent image with the movement of the object, the motion of an object accurately determined, the image processing system及beauty programs and inner quality of the fluorescent image can be sufficiently improved It is an object of the present invention to provide a method for operating a endoscope system .

The image processing system according to the present invention includes an irradiation unit that irradiates a subject with surface sequential light obtained by time-dividing three colors of RGB light or excitation light that generates fluorescence, and irradiation of RGB surface sequential light or excitation light reflected by the subject. A half mirror that divides the fluorescence generated by the light into two lights, and one of the lights divided by the half mirror is received by the first imaging unit, the other is received by the second imaging unit, and RGB frame sequential light When the excitation light is irradiated, the image obtained by the first imaging unit and the image obtained by the second imaging unit are combined to obtain a reflected moving image of RGB frame sequential light . An endoscope having an imaging unit that acquires a fluorescent moving image based on fluorescence from an image obtained by the first imaging unit, and a motion for calculating a subject motion in the reflected light moving image from a reflected light moving image having a higher resolution than the fluorescent moving image Calculated by the calculation unit and the motion calculation unit And an image correction unit that corrects the image quality of the fluorescent moving image based on the movement.

The imaging unit acquires the reflected light moving image having an exposure time shorter than that of the fluorescent moving image, and the image correcting unit detects a blur in a moving image constituent image included in the fluorescent moving image from the motion calculated by the motion calculating unit. It is preferable to correct. The imaging unit acquires the reflected light moving image having a frame rate higher than that of the fluorescent moving image, and the image correcting unit complements the moving image constituent image included in the fluorescent moving image from the motion calculated by the motion calculating unit. Is preferred. It is preferable that the pixel area of the first imaging element included in the first imaging unit is larger than the pixel area of the second imaging element included in the second imaging unit .

  The irradiation unit includes a light source that emits white light, an R color filter that transmits light in a red band of white light from the light source, a G color filter that transmits light in a green band of the white light, and It is preferable to include a B-color filter that transmits blue light of white light and a rotary filter that includes an excitation light transmission filter that transmits the wavelength component of excitation light of the white light. The rotation filter preferably includes a first filter in which an R color filter, a G color filter, and a B color filter are continuously arranged, and a second filter in which an excitation light transmission filter is arranged. In the rotation filter, an R color filter, a G color filter, and a B color filter are arranged at regular intervals along the circumferential direction, and the excitation light transmission filter is a color of each color. It is preferable to be provided between the filters.

The operation method of the endoscope system according to the present invention is such that the illumination unit emits the surface sequential light obtained by time-dividing the three colors of RGB and the excitation light that generates fluorescence, and the half mirror is reflected by the subject. The step of dividing the fluorescence generated by the irradiation of RGB surface sequential light or excitation light into two lights, and the imaging unit receives one of the lights divided by the half mirror with the first imaging unit and the other with the first 2 When the light is received by the image pickup unit and is irradiated with the RGB frame sequential light, the image obtained by the first image pickup unit and the image obtained by the second image pickup unit are synthesized to reflect the RGB frame sequential light. When the moving image is acquired and the excitation light is irradiated, the step of acquiring the fluorescent moving image based on the fluorescence from the image obtained by the first imaging unit, and the motion calculating unit from the reflected light moving image having a higher resolution than the fluorescent moving image , To calculate the movement of the subject in the reflected light video The image correction unit includes a step of correcting the image quality of the fluorescent moving image from the movement of the subject.

The image processing program according to the present invention includes an irradiation unit that irradiates a subject with surface sequential light obtained by time-dividing three colors of RGB light or excitation light that generates fluorescence, and irradiation of RGB surface sequential light or excitation light reflected by the subject. A half mirror that divides the fluorescence generated by the light into two lights, and one of the lights divided by the half mirror is received by the first imaging unit, the other is received by the second imaging unit, and RGB frame sequential light When the excitation light is irradiated, the image obtained by the first imaging unit and the image obtained by the second imaging unit are combined to obtain a reflected moving image of RGB frame sequential light . In an image processing program installed in an image processing system having an endoscope having an imaging unit that acquires a fluorescent moving image based on fluorescence from an image obtained by the first imaging unit, from a reflected light moving image having a higher resolution than the fluorescent moving image , Reflection The computer is caused to function as a motion calculation unit that calculates the motion of a subject in an optical moving image and an image correction unit that corrects the image quality of a fluorescent moving image from the motion of the subject.

  According to the present invention, since the motion of the subject in the reflected light moving image is calculated from the reflected light moving image generated based on the three colors of RGB light reflected by the subject or the reflected light of the excitation light, Can be obtained with high accuracy. Since the image quality of the fluorescent moving image is corrected based on the movement of the subject thus obtained with high accuracy, the image quality of the fluorescent image can be sufficiently improved.

1 is a schematic diagram illustrating an image processing system according to a first embodiment of the present invention. It is the schematic which shows the specific structure of an image process part. It is the schematic of a light source side rotation filter. It is the schematic which shows the specific structure of an irradiation part. It is the schematic of the light source side rotation filter from which the arrangement | sequence of each filter differs from the light source side rotation filter of FIG. It is the schematic which shows the specific structure of an imaging part. It is the schematic of an imaging part side rotation filter. It is a time chart which shows the exposure time of the imaging part in 1st Embodiment. It is a flowchart which shows the effect | action of this invention. It is the schematic which shows the specific structure of the imaging part in 2nd Embodiment of this invention. It is a time chart which shows the exposure time of the imaging part in 2nd Embodiment. It is the schematic of the specific structure of the irradiation part corresponding to normal moving image acquisition mode, fluorescence moving image acquisition mode, and 2 mode. It is the schematic of the light source side rotation filter corresponding to normal moving image acquisition mode, fluorescence moving image acquisition mode, and 2 mode.

  As shown in FIG. 1, the image processing system 100 according to the first embodiment of the present invention includes an endoscope 101, an image processing unit 102, an image recording unit 103, an image display unit 104, an irradiation unit 105, and forceps 106. ing. 1 is an enlarged view of the distal end portion 109 of the endoscope 101. FIG.

  The endoscope 101 includes a forceps port 111, an imaging unit 112, and a light guide 113. The distal end portion 109 of the endoscope 101 has a lens 131 as a part of the imaging unit 112 on the distal end surface 130 thereof. Further, the distal end surface 130 has an emission port 132 as a part of the light guide 113. The forceps port 111 guides the inserted forceps 106 to the distal end portion 130. The nozzle 133 delivers water or air.

  The irradiation unit 105 includes a light source 107 and a light source side rotation filter 108. The light source is a xenon lamp or the like, and emits broadband white light ranging from the blue band to the red band. The light source side rotation filter 108 time-divides and transmits light of three colors of RGB among the white light emitted from the light source 107, and the wavelength component of excitation light that excites fluorescence in the subject out of the light from the light source 107. Permeate only. Hereinafter, the light that is sequentially irradiated with the three colors of RGB in such a manner is referred to as RGB surface sequential light. Then, the RGB surface sequential light and excitation light transmitted through the light source side rotation filter 108 are incident on the incident end of the light guide 113.

  The light guide 113 guides excitation light and RGB frame sequential light to the front end portion 109. The guided excitation light and RGB frame sequential light are emitted from the emission port 132 of the distal end surface 130 and are irradiated onto the subject. In the irradiating unit 105, the excitation light and the RGB surface sequential light are alternately irradiated, so that either the excitation light or the RGB surface sequential light is irradiated into the subject.

  The imaging unit 112 acquires a reflected light moving image by imaging the RGB surface sequential light and the reflected light of the excitation light irradiated on the subject. In addition, the imaging unit 112 acquires a fluorescent moving image by imaging fluorescence excited by irradiation of excitation light. Here, since the fluorescence has a weak intensity, the imaging unit 112 may capture an image with a longer exposure time when capturing the fluorescent light than when capturing the reflected light. Further, when imaging a fluorescent moving image, the imaging unit 112 may capture an image at a frame rate lower than the frame rate of the reflected light moving image. The imaging unit 112 may capture a fluorescent image with higher sensitivity than the reflected light image. Further, when the fluorescence is received by the imaging device 153 (see FIG. 6) in the imaging unit 112, the pixel charge of the imaging device 153 may be added more than a certain amount.

  As described above, when the fluorescent moving image is captured, the exposure time is set longer than that of the reflected light moving image, so that the moving image of the fluorescent light causes the subject to be blurred in the image. Also, if the fluorescent moving image is lower than the frame rate of the reflected light moving image, the fluorescent moving image cannot reproduce the movement of the subject in the image in detail.

  The image processing unit 102 includes a CPU, an electronic circuit, an electric circuit, or the like, and performs image processing on the fluorescent moving image based on the reflected light moving image captured by the imaging unit 112. The specific configuration of this image processing unit will be described later in detail. The image recording unit 103 includes a storage medium such as a flash memory, and records a reflected light moving image and also records a fluorescent moving image that has been subjected to image processing by the image processing unit 102.

  The image display unit 104 includes a liquid crystal display, an organic EL display, a plasma display, and the like, and displays a moving image recorded by the image recording unit. The image display unit 104 includes a reflected light moving image display unit 104a that displays a reflected light moving image, and a fluorescent moving image display unit 104b that displays a fluorescent moving image.

  As illustrated in FIG. 2, the image processing unit 102 includes an image acquisition unit 121, a motion calculation unit 124, and an image correction unit 125. The image acquisition unit 121 includes a reflected light moving image acquisition unit 122 that acquires a reflected light moving image captured by the imaging unit 112 and a fluorescent moving image acquisition unit 123 that acquires a fluorescent moving image captured by the imaging unit 112. The reflected light moving image acquiring unit 122 outputs the acquired reflected light moving image to the motion calculating unit 124 and outputs the reflected light moving image to the image recording unit 103. The fluorescent moving image acquisition unit outputs the acquired fluorescent moving image to the image correction unit 125.

  The motion calculation unit 124 calculates the motion of the subject in the reflected light moving image. Here, the motion is calculated by a block matching method or the like. The calculated subject movement is output to the image correction unit 125. The image correction unit 125 corrects the movement of the subject in the fluorescent video from the movement of the subject calculated by the movement calculation unit 124. The image correction unit 125 outputs the corrected fluorescent moving image to the image recording unit 103.

  When the exposure time of the reflected light moving image is shorter than that of the fluorescent moving image, the image correcting unit 125 may correct the blur in the moving image constituent image included in the fluorescent moving image from the calculated movement. In other words, if the exposure time is long, blurring is likely to occur in the image. Therefore, blurring is corrected or reduced by correcting the blurring. Here, the moving image constituent image refers to an image constituting a moving image such as a frame image or a field image.

  When the reflected light moving image has a higher frame rate than the fluorescent moving image, the image correction unit 125 may generate a complementary image that complements the moving image constituent image included in the fluorescent moving image from the calculated movement. In other words, the frame rate of the fluorescent moving image is increased by generating a complementary image that complements the moving image constituting image between the frames of the fluorescent moving image. When the reflected light moving image has a higher resolution than the fluorescent moving image, the image correcting unit 125 may increase the resolution of the moving image constituent image included in the fluorescent moving image based on the calculated movement. That is, the resolution of the fluorescent moving image is increased.

  As shown in FIG. 3, the light source side rotation filter 108 is provided with a first filter 141 and a second filter 142 on the same circumference, and a shaft 143 serving as a rotation center is provided at the center thereof. ing. The first filter 141 includes an R color filter 141 a that transmits red light in the white light from the light source 107, and a G color filter 141 b that transmits green light in the white light from the light source 107. A B color filter 141c that transmits blue light out of the white light from the light source 107 is provided. The second filter 142 is provided with an excitation light transmission filter 142 a that transmits only the excitation light of the white light from the light source 107.

  As shown in FIG. 4, the first filter 141 and the second filter 142 are alternately arranged on the optical path of the light emitted from the light source 107 by rotating the light source side rotation filter 108 about the axis 143. . The irradiation unit 105 rotates the light source side rotation filter 108 to irradiate the subject with R-color light, G-color light, and B-color light as time-divided surface sequential light. After the sequential light, the subject is irradiated with excitation light for a certain period of time. Since the length of the first filter 141 in the circumferential direction is shorter than the length of the second filter 142 in the circumferential direction, the irradiation unit 105 irradiates the excitation light longer than the irradiation time of RGB frame sequential light. The irradiation unit 105 includes a control unit that controls the light source 107 and the light source side rotation filter 108. This control unit may be realized by an information processing device such as a CPU.

  As shown in FIG. 5, a light source side rotation filter 208 in which the arrangement of the filters is different from the arrangement of the filters of the light source side rotation filter 108 shown in FIG. 3 may be used. The light source side rotation filter 208 is between the R color filter 141a and the G color filter 141b, between the G color filter 141b and the B color filter 141c, and between the B color filter 141c and the R color filter 141a. An excitation light transmission filter 142a is provided in each of them. When this light source side rotation filter 208 is used, each light is irradiated to the subject while switching in the order of R light → excitation light → G color light → excitation light → B color light → excitation light. Can do.

  When the light source side rotation filter 108 is used, light of three colors of RGB is continuously irradiated. Therefore, when the imaging unit 112 reads out an imaging signal corresponding to each color, a sufficient reading rate cannot be secured. There is. On the other hand, when the light source side rotation filter 208 is used, the irradiation of the three colors of RGB is intermittently performed by the excitation light irradiation in between. Even when reading out, it is possible to secure a sufficient reading rate.

  As illustrated in FIG. 6, the imaging unit 112 includes a lens 131, an imaging unit side rotation filter 151, and an imaging element 153. The image sensor 153 is composed of a monochrome CCD or the like, and receives light transmitted through the imaging unit side rotation filter 151 out of light transmitted through the lens 131. The imaging unit-side rotary filter 151 includes an opening 161 and an excitation light cut filter 162 that cuts light in the wavelength band of excitation light and transmits light in other wavelength bands. The imaging unit 112 alternately arranges the excitation light cut filters 162 and the apertures 161 on the optical path between the lens 131 and the imaging element 153 by rotating the imaging unit-side rotation filter 151 about the axis 163. .

  The imaging unit 112 arranges the excitation light cut filter 162 on the optical path when the irradiation unit 105 emits excitation light. Further, the imaging unit 112 arranges the opening 161 on the optical path when the irradiating unit 105 emits RGB surface sequential light. Thereby, the imaging device 153 can capture a subject image in a fluorescent state when the irradiation unit 105 is irradiating excitation light. That is, when excitation light is irradiated, reflected light and fluorescence of the excitation light from the subject enter the imaging unit 112, but the excitation light cut filter 162 cuts the excitation light, so that the image sensor 153 emits only fluorescence. An image can be taken. On the other hand, when the irradiating unit emits RGB frame sequential light, RGB plane sequential light is incident on the image sensor 153 through the opening 161, so that the entire subject is colored. be able to.

  Note that the exposure time, frame rate, and the like of the image sensor 153 are controlled by an information processing apparatus such as a CPU. Further, the rotation of the imaging unit side rotation filter 151 is controlled by the information processing apparatus. Note that the imaging unit 112 includes an imaging element driving driver that drives the imaging element 153. This image sensor drive driver is controlled by an information processing device such as a CPU. The information processing apparatus may be provided in the imaging unit 112 or may be provided in the image processing system 100.

  As shown in FIG. 7, the imaging unit side rotation filter 151 is provided with an opening 161 and an excitation light cut filter 162 on the same circumference. Note that the imaging unit-side rotary filter 151 may be provided with a filter that transmits the incident white light as it is, instead of the opening 161. Further, the length of the opening 161 in the circumferential direction is shorter than the length of the excitation light cut filter 162 in the circumferential direction.

  Further, during the period in which the first filter 141 of the light source side rotation filter 108 is located on the optical path of the light source 107, the opening 161 of the imaging unit side rotation filter 151 is located on the optical path between the lens 131 and the imaging element 153. Match the period. The period in which the second filter 142 of the light source side rotation filter 108 is on the optical path of the light source 107 is the period in which the excitation light cut filter 162 of the imaging unit side rotation filter 151 is on the optical path between the lens 131 and the image sensor 153. Match. Therefore, the RGB frame sequential light irradiation start timing of the light source side rotary filter 108 and the RGB frame sequential light transmission start timing of the imaging unit side rotary filter 151 are substantially synchronized. The excitation light irradiation start timing of the light source side rotation filter 108 and the excitation light cut start timing of the imaging unit side rotation filter 151 are substantially synchronized.

  In the time chart of the exposure time of the imaging unit 112 in FIG. 8, a short bidirectional arrow indicates the exposure time of the imaging unit 112 that exposes RGB frame sequential light, and a long bidirectional arrow indicates the exposure time of the imaging unit 112 that exposes fluorescence. Represents. As shown in this time chart, when the image capturing unit 112 is irradiated with RGB frame sequential light, the image capturing unit 112 exposes reflected light of the RGB frame sequential light to capture a color image 301. Next, when the excitation light is irradiated, the imaging unit 112 exposes the fluorescence to capture the fluorescence image 311. After that, when the three colors of RGB light are irradiated, the imaging unit 112 captures a color image 302 by exposing the reflected light of the RGB surface sequential light. Note that the exposure time of reflected light of RGB surface sequential light is shorter than the exposure time of fluorescence. Accordingly, the irradiation time of the RGB surface sequential light is shorter than the irradiation time of the excitation light. Since the fluorescent light is weak, the fluorescent light is exposed longer than the exposure time of the reflected light.

  By such an operation, a color image 301 by plane sequential light → fluorescence image 311 → color image 302 by plane sequential light → fluorescence image 312 → color image 303 by plane sequential light → fluorescence image 313 and so on. The color image and the fluorescence image by the frame sequential light are captured alternately. Therefore, a color reflected light moving image is constituted by the color images of these surface sequential lights arranged in time series. In addition, a fluorescent moving image is constituted by these fluorescent images arranged in time series.

  Next, the operation of the image processing system 100 will be described with reference to the flowchart of FIG. The irradiating unit 105 irradiates excitation light and RGB frame sequential light by rotating the light source side rotation filter 108. In accordance with this, the imaging unit 112 rotates the imaging unit side rotation filter 151. Thereby, the imaging part 112 can image a fluorescence moving image and a reflected light moving image in parallel. Here, since the fluorescence is weak, the imaging unit 112 receives the fluorescence with an exposure time longer than the exposure time for receiving the reflected light. Along with this, the irradiation unit 105 irradiates the excitation light with an irradiation time longer than the irradiation time for irradiating light of three colors of RGB.

  The reflected light moving image acquiring unit 122 sequentially acquires the reflected light moving image captured by the image capturing unit 112, and the fluorescent moving image acquiring unit 123 sequentially acquires the fluorescent moving image captured by the image capturing unit 112. Then, the motion calculating unit 124 calculates the motion of the subject from the reflected light moving image acquired by the reflected light moving image acquiring unit. Then, the image correction unit 125 corrects the image quality of the fluorescent moving image acquired by the fluorescent moving image acquisition unit 123 from the motion calculated by the motion calculating unit 124.

  Further, when the frame rate of the reflected light moving image is higher than that of the fluorescent moving image, the image correcting unit 125 complements the moving image constituent image included in the fluorescent moving image from the motion calculated by the motion calculating unit 124. That is, since the moving picture with a higher frame rate reproduces the movement of the subject in detail, the moving can be obtained with high accuracy, and the image quality of the fluorescent moving picture can be improved. Further, when the resolution of the reflected light moving image is higher than that of the fluorescent moving image, the image correcting unit 125 may increase the resolution of the fluorescent moving image based on the motion calculated by the motion calculating unit 124. In other words, since the subject having a higher resolution can reproduce the subject in detail, the movement can be obtained with high accuracy, and the image quality of the fluorescent moving image can be improved.

  The image recording unit 103 records a fluorescent moving image whose image quality is corrected and also records a reflected light moving image. The image display unit 104 displays the reflected light moving image recorded in the image recording unit 103 on the reflected light moving image display unit 104a, and displays the fluorescent moving image recorded in the image recording unit 103 on the fluorescent moving image display unit 104b. Note that the image display unit 104 may display either a reflected light moving image or a fluorescent moving image. Further, the reflected light moving image or the fluorescent moving image may be switched and displayed according to a display switching instruction from the user.

  In the second embodiment of the present invention, unlike the first embodiment in which only one imaging device is provided in the imaging unit, the imaging unit includes a first imaging device for fluorescent moving image imaging and a second for reflected light moving image imaging. Since imaging is performed using two imaging elements of the imaging element, the imaging unit side rotation filter as in the first embodiment is not required. Since the parts other than the imaging unit are the same as those in the first embodiment, description thereof is omitted.

  As illustrated in FIG. 10, the imaging unit 212 of the second embodiment includes a lens 231, a first imaging unit 271, a second imaging unit 272, and a half mirror 273. The half mirror 273 divides the light transmitted through the lens 231 into two lights by transmitting a part of the light and reflecting the rest. In the half mirror 273, the amount of reflected light and the amount of transmitted light are substantially the same. Note that a beam splitter may be used instead of the half mirror 273. When a beam splitter is used, the ratio between the amount of reflected light and the amount of transmitted light may be varied.

  The first imaging unit 271 includes an excitation light cut filter 162 and a first imaging element 275. The excitation light cut filter 262 cuts the light in the wavelength band of the excitation light out of the light transmitted through the half mirror 173 and transmits the light in the other wavelength bands. The first image sensor 275 receives light that has passed through the excitation light cut filter. Therefore, the first image sensor 275 can receive only fluorescence.

  The second imaging unit 272 includes a second imaging element 276. The second image sensor 276 receives the light reflected by the half mirror. Therefore, the second image sensor 276 can capture a color image of the subject when the irradiation unit 105 irradiates the subject with RGB frame sequential light, and captures a background image when the excitation light is irradiated. can do.

  Here, since the first image sensor 275 receives fluorescence with weak light, the pixel area of the first image sensor 275 may be larger than the pixel area of the second image sensor 276. When the effective pixel areas of the first image sensor 275 and the second image sensor 276 are the same, the resolution of the first image sensor 275 is lower than that of the second image sensor 276.

  The first imaging unit 271 includes a first imaging element driving driver that drives the first imaging element 275. In addition, the second imaging unit 272 includes a second imaging element driving driver that drives the second imaging element 276. The first image sensor drive driver and the second image sensor drive driver are controlled by an information processing apparatus such as a CPU. The information processing apparatus may be provided in the imaging unit 212 or in the image processing system 100.

  FIG. 11A shows a time chart of exposure times of the first imaging unit 271 and the second imaging unit 272 when only excitation light is irradiated. In this time chart, the short bidirectional arrow indicates the exposure time of the second imaging unit 272 that exposes the reflected light of the excitation light, and the long bidirectional arrow indicates the exposure time of the first imaging unit 271 that exposes the fluorescence. Show.

  As shown in this time chart, the light from the subject is divided into two lights by the half mirror 273, and each of the divided lights is imaged by the first imaging unit 271 and the second imaging unit 272. The subject can be imaged. Since the second imaging unit 172 exposes the reflected light of the excitation light, the second imaging unit 172 exposes the exposure time shorter than the exposure time of the first imaging unit 271 that exposes only fluorescence. In addition, since the second imaging unit 172 has a short exposure time, it can also capture images at a frame rate higher than the frame rate of the first imaging unit 271.

  By performing imaging according to this time chart, the second imaging unit 272 causes the excitation light image 321 → the excitation light image 322 → the excitation light image 323 → the excitation light image 324 → the excitation light image 325 → the excitation light image 325. A moving image of excitation light such as an image 326 is captured at a constant frame rate. On the other hand, the first imaging unit 271 captures a fluorescent image simultaneously with the imaging of the second imaging unit 272. That is, the first imaging unit 171 captures the fluorescence image 341 by exposing the fluorescence. Therefore, a moving image of fluorescence is captured at a frame rate lower than the constant frame rate of the moving image of excitation light, such as fluorescent image 341 → fluorescent image 342 → fluorescent image 343. As a result, the background moving image, which is one of the reflected light moving images, is formed from the excitation light images arranged in time series, and the fluorescent moving images are arranged in time series. Is configured.

  FIG. 11B shows a time chart of the exposure time of the first imaging unit 271 and the second imaging unit 272 when the surface sequential light of RGB and the excitation light are irradiated. In this time chart, a short bidirectional arrow indicates an exposure time of the second imaging unit 272 that exposes reflected light of excitation light or exposes light of three colors of RGB. The long bidirectional arrow is the same as in FIG.

  First, when RGB frame sequential light is irradiated, the second imaging unit 272 exposes RGB plane sequential light to capture a color image 351. Next, when the excitation light is irradiated, the second imaging unit 272 exposes the reflected light of the excitation light to capture an image 361 of the excitation light. At this time, the second imaging unit 272 captures the fluorescence image 371 by exposing the fluorescence for an exposure time longer than the exposure time for capturing the excitation light image 361. By performing the same operation as this, a color image 352 and an excitation light image 362 are obtained after a predetermined time of acquisition of the color image 351 and the excitation light image 361. In addition, a fluorescence image 372 is obtained after a certain time of acquisition of the fluorescence image 371. By repeating such an operation, it is possible to obtain a moving image of reflected color light, a moving image of excitation light, and a moving image of fluorescence by RGB frame sequential light.

  In addition, when irradiating RGB frame sequential light, not only the 2nd imaging part 272 but the 1st imaging part 271 may image the reflected light of RGB frame sequential light. Alternatively, the images of the reflected light of the RGB frame sequential light captured by the first imaging unit 271 and the second imaging unit 272 at the same timing may be combined into a single image. The synthesized image constitutes a moving image of reflected light of RGB frame sequential light.

  When the irradiating unit 105 irradiates excitation light and RGB surface sequential light, the motion is obtained from the moving image of the reflected light of RGB surface sequential light captured by the second imaging unit 272 when determining the motion of the subject. Alternatively, the movement may be obtained from the moving image of the reflected light of the excitation light imaged by the second imaging unit 272. Furthermore, the movement of the subject may be obtained from two moving images, a moving image of reflected light of RGB frame sequential light captured by the second imaging unit 272 and a moving image of reflected light of excitation light.

  In the first and second embodiments, there is only one mode of the fluorescent moving image acquisition mode in which the reflected light moving image and the fluorescent moving image are acquired simultaneously. However, in addition to this fluorescent moving image acquisition mode, only the reflected light moving image is acquired. You may implement this invention in 2 modes, normal moving image acquisition mode. In order to implement the present invention in these two modes, it is necessary to change the irradiation unit 105 of the image processing system 100 to the irradiation unit 403 shown in FIG.

  In the irradiation part 403, it replaces with the light source side rotation filter 108 of 1st Embodiment, and the light source side rotation filter 408 corresponding to two modes, fluorescence moving image acquisition mode and normal moving image acquisition mode, is used. The light source side rotation filter 408 transmits a first filter 405 that transmits white light from the light source 107 that is used in the normal moving image acquisition mode, and a second filter that transmits white light that is used in the fluorescent moving image acquisition mode. And the first filter 405 is outside the second filter 406. Further, the light source side rotary filter 408 can rotate around the shaft 410 and can be moved in two radial directions by a filter switching unit 411 attached to the shaft 410. Therefore, when switching modes and processing, the filter switching unit 411 moves the light source side rotary filter 408 in the radial direction so that the filter corresponding to the mode to be switched is positioned on the optical path of the light source 107.

  As shown in FIG. 13, the first filter 405 of the light source side rotation filter 408 includes an R color filter 405 a that transmits red band light of white light from the light source 107, and green band light of white light. A G color filter 405b that transmits light and a B color filter 405c that transmits blue light of white light are provided in this order along the circumferential direction. On the other hand, the second filter 406 is provided with an opening 406a that transmits white light as it is and an excitation light transmission filter 406b that transmits only excitation light of the white light from the light source 107 along the circumferential direction. Yes.

  In the second filter 406, an RGB color filter may be provided instead of the opening 406a so that RGB surface sequential light can be emitted, as in the first embodiment, and a G color filter. You may provide only.

  In the above embodiment, when acquiring a color moving image, the color moving image is acquired by irradiating RGB frame sequential light and imaging with a monochrome CCD. A color moving image may be acquired by irradiating a subject and imaging the reflected light of white light reflected by the subject with a primary or complementary color CCD (simultaneous acquisition of a color moving image).

  Here, the primary color CCD includes a B pixel provided with a B color filter, a G pixel provided with a G color filter, and an R pixel provided with an R color filter. By receiving white light with these color pixels, imaging signals corresponding to the B, G, and R colors are acquired. A color moving image can be acquired from the imaging signals corresponding to these three colors.

  The complementary color CCD is provided with a C pixel provided with a color filter that transmits C (cyan) color of white light from the light source 107 and a color filter that transmits M (magenta) color of white light. M pixels and Y pixels provided with a color filter that transmits Y (yellow) color of white light are provided. A color moving image can be acquired from the three-color imaging signals obtained by receiving white light with the pixels of each color. In addition to the C pixel, the M pixel, and the Y pixel, a G pixel provided with a G color filter may be added to the pixels of the complementary color CCD.

DESCRIPTION OF SYMBOLS 100 Image processing system 102 Image processing part 105,403 Irradiation part 107 Light source 108,208,408 Light source side rotation filter 112,212 Imaging part 124 Motion calculation part 125 Image correction part 141a R color filter 141b G color filter 141c B color filter 142a Excitation light transmission filter 271 First imaging unit 272 Second imaging unit

Claims (9)

An irradiation unit that irradiates an object with surface-sequential light obtained by time-dividing RGB three-color light or excitation light that generates fluorescence, and fluorescence generated by irradiation of RGB surface-sequential light or excitation light reflected by the object. One of the half mirror divided into two lights and the light divided by the half mirror is received by the first imaging unit, the other is received by the second imaging unit, and the RGB surface sequential light is irradiated. Sometimes, the image obtained by the first imaging unit and the image obtained by the second imaging unit are synthesized to obtain a reflected light moving image of the RGB frame sequential light, and when the excitation light is irradiated, An endoscope having an imaging unit for acquiring a fluorescent moving image based on the fluorescence from an image obtained by the first imaging unit ;
A motion calculation unit for calculating a motion of a subject in the reflected light moving image from the reflected light moving image having a higher resolution than the fluorescent moving image;
An image processing system comprising: an image correction unit that corrects an image quality of the fluorescent moving image based on the motion calculated by the motion calculation unit. The imaging unit
Obtaining the reflected light moving image having a shorter exposure time than the fluorescent moving image;
The image correction unit
The image processing system according to claim 1, wherein blurring in a moving image constituent image included in the fluorescent moving image is corrected based on the movement calculated by the motion calculating unit. The imaging unit
Obtaining the reflected light moving image having a higher frame rate than the fluorescent moving image;
The image correction unit
The image processing system according to claim 1, wherein a complementary image is generated by complementing a moving image constituent image included in the fluorescent moving image from the movement calculated by the movement calculating unit. 4. The image according to claim 1, wherein a pixel area of the first imaging element included in the first imaging unit is larger than a pixel area of the second imaging element included in the second imaging unit. 5. Processing system. The irradiation unit is
A light source that emits white light;
An R color filter that transmits red light in the white light from the light source, a G color filter that transmits green light in the white light, and B that transmits blue light in the white light. 5. The image processing system according to claim 1, further comprising a color filter and a rotation filter including a pumping light transmission filter that transmits a wavelength component of the pumping light in the white light. The rotary filter is
6. A first filter in which an R color filter, a G color filter, and a B color filter are continuously arranged, and a second filter in which an excitation light transmission filter is arranged. Image processing system. The rotary filter is
The R color filter, the G color filter, and the B color filter are arranged at regular intervals along the circumferential direction.
6. The image processing system according to claim 5, wherein the excitation light transmission filter is provided between the color filters of the respective colors. An illuminating unit that emits surface-sequential light obtained by time-dividing light of three colors of RGB and excitation light that generates fluorescence;
A half mirror that divides the fluorescence generated by the irradiation of the RGB surface sequential light or the excitation light reflected by the subject into two lights;
When the imaging unit receives one of the lights divided by the half mirror by the first imaging unit, receives the other by the second imaging unit, and is irradiated with the RGB surface sequential light, the first imaging unit receives the first light. The image obtained by the imaging unit and the image obtained by the second imaging unit are combined to obtain a reflected light moving image of the RGB frame sequential light, and when the excitation light is irradiated, the first imaging unit Obtaining a fluorescence animation based on the fluorescence from the image obtained in
A step of calculating a movement of a subject in the reflected light moving image from the reflected light moving image having a higher resolution than the fluorescent moving image;
And a step of correcting the image quality of the fluorescent moving image from the movement of the subject. An irradiation unit that irradiates an object with surface-sequential light obtained by time-dividing RGB three-color light or excitation light that generates fluorescence, and fluorescence generated by irradiation of RGB surface-sequential light or excitation light reflected by the object. One of the half mirror divided into two lights and the light divided by the half mirror is received by the first imaging unit, the other is received by the second imaging unit, and the RGB surface sequential light is irradiated. Sometimes, the image obtained by the first imaging unit and the image obtained by the second imaging unit are synthesized to obtain a reflected light moving image of the RGB frame sequential light, and when the excitation light is irradiated, In an image processing program installed in an image processing system having an endoscope having an imaging unit that acquires a fluorescence moving image based on the fluorescence from an image obtained by the first imaging unit ,
A motion calculation unit for calculating a motion of a subject in the reflected light moving image from the reflected light moving image having a higher resolution than the fluorescent moving image;
An image processing program that causes a computer to function as an image correction unit that corrects the image quality of the fluorescent moving image from the movement of the subject.

Images