JP6047467B2 - Endoscope system and operating method thereof - Google Patents

Description

  The present invention relates to an endoscope system capable of switching between a simultaneous method and a frame sequential method, and an operation method thereof.

  In recent medical treatments, diagnosis and the like using an endoscope system including a light source device, an electronic endoscope, and a processor device are widely performed. The light source device generates illumination light and irradiates the specimen. The electronic endoscope is irradiated with illumination light, and the inside of the specimen is imaged by an image sensor to generate an image signal. The processor device performs image processing on the imaging signal generated by the electronic endoscope and generates an observation image for display on the monitor.

  As an illumination method of this endoscope system, there are a frame sequential (time division) method and a simultaneous method. The frame sequential method is a method in which, for example, red (R), green (G), and blue (B) illumination lights are sequentially irradiated onto a specimen, and a specimen image illuminated with each illumination light is captured in monochrome. Images are taken individually by the element. The simultaneous method is a method of simultaneously irradiating a specimen with R, G, and B illumination light (that is, irradiating white light), and a specimen image irradiated with white light is imaged by a simultaneous imaging device having a color filter. Is done.

  The frame sequential method is superior in terms of color reproducibility and resolution compared to the simultaneous method. On the other hand, since the simultaneous method has a higher frame rate than the frame sequential method, image blurring hardly occurs. In the case of the frame sequential method, image blurring is likely to occur due to a low frame rate, and in addition, each of R, G, and B images is acquired at different timings. , B will cause color misregistration.

  As described above, since the frame sequential method and the simultaneous method have advantages and disadvantages, an endoscope system that uses a simultaneous image sensor and can switch the illumination method between the simultaneous method and the surface sequential method has been proposed. (Patent Document 1). In Patent Document 1, the amount of blur is always detected while moving image shooting is performed in the frame sequential method, and the method is switched to the simultaneous method when the amount of blur exceeds a certain level.

  Further, in Patent Document 2, in a field sequential endoscope system, a plurality of moving images are detected when a freeze instruction for constantly detecting a blur amount and capturing a still image is input during moving image shooting. It has been proposed to select an image with the smallest blur amount as a still image from among the images.

JP 2009-284959 A Japanese Patent No. 4847250

  In the endoscope systems described in Patent Documents 1 and 2, moving image shooting is performed in a frame sequential manner. However, when a moving image is observed during endoscopy, there is movement in the specimen or the endoscope. Since it tends to occur, a simultaneous method with a high frame rate is generally suitable. In addition, when acquiring still images, skilled doctors are good at capturing images with less blurring by suppressing the movement of the endoscope.Therefore, a frame sequential method with better color reproducibility rather than a simultaneous method is used. Liked.

  Therefore, it is possible not only to switch from the frame sequential method to the simultaneous method but also to switch from the simultaneous method to the frame sequential method according to the request of the doctor who uses the endoscope system, and to acquire still images with less blurring. Is desired.

  An object of the present invention is to provide an endoscope system that can switch between a simultaneous method and a frame sequential method, and that can acquire a still image with less blurring, and an operating method thereof.

In order to achieve the above object, an endoscope system according to the present invention includes a light source device that generates illumination light of multiple colors, a simultaneous image sensor having a color separation filter, and a pixel signal read from the simultaneous image sensor. An image signal processing unit for generating image data based on the image signal processing unit, a simultaneous method for simultaneously illuminating a plurality of colors of illumination light in response to an input of a freeze instruction signal for freezing a moving image into a still image, and a plurality of colors of illumination A controller that switches the illumination method of the light source device between the surface sequential method that irradiates light in a time-division manner, and calculates a first blur amount from each image data of the first image data group generated after the illumination method is switched, A blur amount calculation unit that calculates a second blur amount from each image data of the second image data group generated before the switching of the illumination method, and the first and second image data based on the first and second blur amounts. From the group, stationary And a still image selecting section for selecting an image data to be. When the minimum value of the first blur amount is smaller than the reference value, the still image selection unit selects image data having the minimum value from the first image data group as a still image, and the minimum value of the first blur amount is If it is greater than or equal to the reference value, it is determined whether or not the minimum value of the second blur amount is smaller than the reference value. If the minimum value is smaller than the reference value, the minimum value is extracted from the second image data group. Is selected as a still image, and when the minimum value of the second blur amount is equal to or greater than the reference value, the smaller one of the first and second minimum blur values is selected as the first and second image data. Select from the second image data group.

  In the case of the frame sequential method, the image processing apparatus includes a synchronization processing unit that generates image data obtained by synthesizing image data obtained at the time of irradiation of illumination lights of a plurality of colors. When switching from the simultaneous method to the frame sequential method and the image data of the frame sequential method is selected by the still image selection unit, a plurality of image data to be combined are respectively color-corresponding to the image data obtained by the simultaneous method. It is preferable that the amount of positional deviation is calculated by comparing with an image, and the positions are aligned based on the calculated amount of positional deviation and then combined.

  Preferably, the light source device has a normal light observation mode that generates red light, green light, and blue light, and a light source device that has a narrow band light observation mode that generates first and second narrow band light.

  In the case of the narrow-band light observation mode, the synchronization processing unit synthesizes one color separation image corresponding to the first narrow-band light or the second narrow-band light of the image data generated by the simultaneous method after alignment. It is preferable to add to one of the plurality of image data.

  In the narrow-band light observation mode and the simultaneous method, the still image selection unit sets image data to be a still image based on the color separation amount related to the contrast width in addition to the first blur amount or the second blur amount. Is preferably selected.

  The blur amount calculation unit is based on one color signal included in the image data generated by the simultaneous method and the color signal of the color included in the image data when the illumination light of the color is irradiated by the frame sequential method. It is preferable to calculate the first and second blur amounts.

  It is preferable that the shake amount calculation unit calculates the first and second shake amounts based on an integrated value obtained by integrating the high frequency components of the spatial frequency in the image data.

  The blur amount calculation unit may calculate the first and second blur amounts by detecting a difference between two temporally adjacent image data.

  The blur amount calculation unit calculates an image blur amount based on each color signal included in the image data, calculates the first and second blur amounts by multiplying each image blur amount by a weighting coefficient, and adds the result. Also good.

  The color separation filter is preferably a complementary color system.

  It is preferable to have a dual mode in which image data selected as a still image by the still image selection unit and the image data subjected to special image processing are simultaneously displayed on the image display device.

  The blur amount calculation unit may calculate the second blur amount each time image data is generated by the image signal processing unit before the freeze instruction signal is input.

The operation method of the endoscope system according to the present invention includes a first step in which the light source device generates illumination light of a plurality of colors, and a pixel signal read out from the simultaneous image sensor having a color separation filter by the image signal processing unit. A second step of generating image data based on the same; a simultaneous method in which the control unit simultaneously irradiates a plurality of colors of illumination light in response to an input of a freeze instruction signal for freezing the moving image into a still image; A third step of switching the illumination method of the light source device between the surface sequential method of irradiating colored illumination light in a time-sharing manner, and each image of the first image data group generated by the blur amount calculation unit after the illumination method is switched A fourth step of calculating a first blur amount from the data and calculating a second blur amount from each image data of the second image data group generated before the switching of the illumination method; Second blur amount Based on provided from the first and second image data group, and a fifth step of selecting the image data to be a still image, a. In the fifth step, when the minimum value of the first blur amount is smaller than the reference value, the still image selection unit selects image data having the minimum value from the first image data group as a still image, and the first blur amount is selected. If the minimum value of the amount is greater than or equal to the reference value, it is determined whether or not the minimum value of the second blur amount is smaller than the reference value. If the minimum value is smaller than the reference value, the second image data When the image data having the minimum value is selected as a still image from the group and the minimum value of the second blur amount is equal to or greater than the reference value, the smaller one of the minimum values of the first and second blur amounts Are selected from the first and second image data groups.

  According to the present invention, the illumination method of the light source device is switched between the simultaneous method and the frame sequential method in response to the input of the freeze instruction signal, and the first image data of the first image data group generated after the switching is changed to the first. A blur amount is calculated, a second blur amount is calculated from each image data of the second image data group generated before switching, and the first and second blurs are calculated based on at least one of the first and second blur amounts. Since image data to be a still image is selected from the image data group, a still image with less blur can be acquired.

It is an external view of an endoscope system. It is a block diagram which shows the internal structure of an endoscope system. It is a figure explaining the structure of a multiplexing part. It is a schematic diagram which shows a complementary color system color separation filter. It is a figure which shows the output signal from a complementary color type image pick-up element. It is a figure which shows the drive timing of the light source and complementary color type image pick-up element in the case of a simultaneous system in normal light observation mode. It is a figure which shows the drive timing of the light source and complementary color type image pick-up element in the case of a field sequential system in normal light observation mode. It is a block diagram which shows the structure regarding the input control of the illumination system of a control part. It is a block diagram which shows the structure of an image signal processing part. It is a figure which shows the image data before and behind reception of the freeze instruction | indication signal in normal light observation mode, (A) shows the case of a simultaneous system, (B) shows the case of a frame sequential system. It is a flowchart which shows the still image selection processing procedure by a still image selection part. It is a figure which shows the image data in the case of switching from a simultaneous system to a frame sequential system in normal light observation mode. It is a flowchart which shows the effect | action of an endoscope system. It is a flowchart which shows the calculation procedure of total blur amount. It is a figure explaining the position shift correction | amendment at the time of the synchronization process in normal light observation mode. It is a figure explaining the position shift correction | amendment at the time of the synchronization process in narrow band light observation mode. It is a figure explaining an effective area | region. It is a figure explaining a dual mode. It is a block diagram which shows the example which provided the blue narrow-band filter in the light source device.

  In FIG. 1, an endoscope system 10 includes a light source device 11, a processor device 12, and an electronic endoscope (hereinafter simply referred to as an endoscope) 13 that can be detachably connected to the light source device 11 and the processor device 12. It is configured. The light source device 11 generates illumination light and supplies it to the endoscope 13. The endoscope 13 is inserted into the body cavity or the like of the specimen to image the inside of the body cavity. The processor device 12 controls the imaging of the endoscope 13 and performs signal processing on the imaging signal acquired by the endoscope 13.

  An image display device 14 and an input device 15 are connected to the processor device 12. The image display device 14 is a liquid crystal monitor or the like, and displays a sample image representing an image in the sample generated by the processor device 12. The input device 15 includes a keyboard and a mouse, and inputs various information to the processor device 12.

  The endoscope 13 includes an insertion portion 16, an operation portion 17, a universal cable 18, a light guide connector 19a, and a signal connector 19b. The insertion portion 16 is elongated and inserted into the body cavity of the specimen. The operation unit 17 is connected to the rear end of the insertion unit 16 and is provided with a scope switch, a bending operation dial, and the like. The scope switch includes a mode switch 17a for switching the observation mode and a freeze switch 17b for freezing a moving image displayed on the image display device 14 to display a still image.

  The universal cable 18 extends from the operation unit 17. The light guide connector 19 a and the signal connector 19 b are provided at the end of the universal cable 18. The light guide connector 19a is detachably connected to the light source device 11. The signal connector 19b is detachably connected to the processor device 12.

  The endoscope system 10 has a normal light observation mode and a narrow-band light observation mode as observation modes. In the normal light observation mode, imaging is performed by irradiating the specimen with normal light (white light) whose wavelength range extends from the blue band to the red band, and a normal image is generated. In the narrowband light observation mode, imaging is performed by irradiating the specimen with narrowband light (blue narrowband light Bn and green narrowband light Gn, which will be described later) having a narrow wavelength range, and a special image is generated.

  The normal light observation mode and the narrowband light observation mode can be switched by the mode switch 17a described above, but are provided on a foot switch (not shown) that can be connected to the processor device 12 or on the front panel of the processor device 12. It may be possible to switch between the buttons, the input device 15 and the like.

  In FIG. 2, the light source device 11 includes a plurality of LED (Light Emitting Diode) light sources 20, a light source control unit 21, a green narrow band filter 22, a filter insertion / removal unit 23, and a multiplexing unit 24. Yes. The LED light source 20 includes B (Blue) -LED 20a, G (Green) -LED 20b, and R (Red) -LED 20c.

  The B-LED 20a generates blue light BL having a wavelength range of 420 to 500 nm. The G-LED 20b generates green light GL having a wavelength range of 500 to 600 nm. The R-LED 20c generates red light RL having a wavelength range of 600 to 650 nm.

  The light source control part 21 performs lighting control of each LED 20a-20c. Specifically, in the normal light observation mode, the light source control unit 21 drives all the LEDs 20a to 20c simultaneously or sequentially to generate illumination light, and in the narrow band light observation mode, the B-LED 20a and the G-LED 20b. Only the lights are driven simultaneously or sequentially to generate illumination light.

  The light source control unit 21 enables the simultaneous method and the frame sequential method as the illumination method by the LED light source 20 in each of the normal light observation mode and the narrow-band light observation mode. In the case of the simultaneous method in the normal light observation mode, all the LEDs 20a to 20c are turned on simultaneously. In the normal light observation mode, the LEDs 20a to 20c are turned on in a time-sharing manner in order. In the case of the simultaneous mode in the narrow-band light observation mode, the B-LED 20a and the G-LED 20b are turned on simultaneously. In the case of the field sequential method in the narrow-band light observation mode, the B-LED 20a and the G-LED 20b are turned on in a time-sharing manner in order.

  The green narrow band filter 22 is inserted into and removed from the optical path of the green light GL emitted from the G-LED 20 b by the filter insertion / removal unit 23. Specifically, the green narrow band filter 22 is inserted on the optical path of the green light GL in the narrow band light observation mode, and is removed from the optical path of the green light GL in the normal light observation mode. The green narrow band filter 22 transmits light in a wavelength range of 530 to 550 nm.

  Since the blue light BL emitted from the B-LED 20a has a narrow half width of about 50 nm, the blue light BL is directly used as the blue narrowband light Bn in the narrowband light observation mode. On the other hand, since the green light GL emitted from the G-LED 20b has a wide wavelength band, the wavelength band is limited to about 20 nm by transmitting the green narrow band filter 22. In the narrow-band light observation mode, the wavelength-limited green light GL is used as the green narrow-band light Gn. The blue narrow band light Bn has a center wavelength in the vicinity of 445 nm. The green narrow band light Gn has a center wavelength in the vicinity of 540 nm. These central wavelengths are generally included in a wavelength range that is easily absorbed by hemoglobin in blood used in narrow-band light observation.

  As shown in FIG. 3, the multiplexing unit 24 includes first and second dichroic mirrors (DM) 25 a and 25 b and first to fourth lenses 26 a to 26 d. The 1st-3rd lenses 26a-26c are each arrange | positioned corresponding to LED20a-20c, The light inject | emitted from each LED20a-20c is condensed, and it is set as parallel light.

  The G-LED 20b and the R-LED 20c are arranged so that the optical paths of the green light GL and the red light RL, which are made parallel by the second and third lenses 26b and 26c, are orthogonal to each other, and the first DM 25a is located at this intersection. Is arranged. The green light GL is incident on one surface of the first DM 25a at an angle of 45 °, and the red light RL is incident on the other surface at an angle of 45 °. The first DM 25a has an optical characteristic of transmitting the green light GL and reflecting the red light RL. Thereby, when the G-LED 20b and the R-LED 20c are simultaneously turned on, the green light GL transmitted through the first DM 25a and the red light RL reflected by the first DM 25a are combined.

  The blue light BL made parallel by the first lens 26a, and the combined light of the green light GL and the red light RL (hereinafter referred to as the first combined light) have orthogonal optical paths, and the second DM 25b is disposed at this intersection. Has been. The blue light BL is incident on one surface of the second DM 25b at an angle of 45 °, and the first combined wave is incident on the other surface at an angle of 45 °. The second DM 25b has an optical characteristic of reflecting the blue light BL and transmitting the first combined light. As a result, the blue light BL reflected by the second DM 25b and the first combined light transmitted through the second DM 25b are combined. The combined light is collected by the fourth lens 26 d and enters the light guide 27 of the endoscope 13.

  In the case of the simultaneous method in the normal light observation mode, the light combining unit 24 combines the blue light BL, the green light GL, and the red light RL into normal light (white light) and enters the light guide 27. On the other hand, in the normal light observation mode, in the case of the frame sequential method, the blue light BL, the green light GL, and the red light RL are individually generated and enter the light guide 27.

  In the case of the simultaneous mode in the narrow band light observation mode, the above-described green narrow band filter 22 is inserted between the second lens 26b and the first DM 25a, and the blue narrow band light Bn emitted from the B-LED 20a The green narrow band light Gn generated by the green narrow band filter 22 is combined and enters the light guide 27. In the case of the field sequential method in the narrow band light observation mode, the blue narrow band light Bn and the green narrow band light Gn are individually generated and enter the light guide 27.

  An illumination window and an observation window are provided adjacent to each other at the distal end of the insertion portion 16 of the endoscope 13, an illumination lens 25 is attached to the illumination window, and an objective lens 26 is attached to the observation window. Yes. A light guide 27 is inserted into the endoscope 13, and one end of the light guide 27 faces the illumination lens 25. The other end of the light guide 27 is disposed on the light guide connector 19 a and is inserted into the light source device 11.

  The illumination lens 25 is supplied from the light source device 11 to the light guide 27, collects the light emitted from the light guide 27, and irradiates the sample. The objective lens 26 collects the reflected light from the biological tissue of the specimen and forms an optical image. A complementary color image pickup device 28 that picks up an optical image and generates an image pickup signal is disposed at the image forming position of the objective lens 26. The complementary color image sensor 28 is a CCD (Charge Coupled Device) image sensor.

  A complementary color system color separation filter 28 a that optically separates an optical image for each pixel is provided on the imaging surface of the complementary color image sensor 28. As shown in FIG. 4, the complementary color separation filter 28a has four color filter segments of magenta (Mg), green (G), cyan (Cy), and yellow (Ye). Are attached in pixel units. Therefore, the complementary color image sensor 28 has four kinds of pixels of Mg, G, Cy, and Ye, and odd-numbered columns are arranged in the order of Mg pixels, Cy pixels, Mg pixels, Ye pixels,. , G pixel, Ye pixel, G pixel, Cy pixel,..., Mg pixels and G pixels are alternately arranged in odd rows, and Cy pixels and Ye pixels are alternately arranged in even rows. Has been placed. This color filter array is called a complementary color checkered color difference line sequential method.

  The endoscope 13 is provided with an information storage unit 29 composed of a nonvolatile memory such as a flash memory. The information storage unit 29 stores unique information of the endoscope 13 (color filter array of the image sensor and the number of pixels) and the like.

  The processor device 12 includes a control unit 30, an imaging control unit 31, a correlated double sampling (CDS) circuit 32, an A / D conversion circuit 33, a brightness detection circuit 34, a dimming circuit 35, an image signal, The processing unit 36, the shake amount calculation unit 37, the still image selection unit 38, the still image memory 39, and the display control unit 40 are included.

  The control unit 30 controls each unit in the processor device 12 and the light source device 11. The control unit 30 reads specific information of the endoscope 13 from the information storage unit 29 when the endoscope 13 is connected to the light source device 11 and the processor device 12. The imaging control unit 31 drives the complementary color imaging element 28 based on the read unique information.

  The imaging control unit 31 drives the complementary color imaging device 28 by the field readout method in accordance with the light emission timing of the light source device 11. Specifically, in the field readout method, at the time of each readout of an odd field and an even field, two pixels adjacent in the column direction are mixed (added) and read out (see FIG. 4). . This mixing of pixel signals is performed in a horizontal transfer path (not shown) of the CCD image sensor.

  With this field readout method, the complementary color image sensor 28 has mixed pixel signals (hereinafter referred to as first mixed pixel signals) of Mg pixels and Cy pixels, as shown in FIG. ) M1, a mixed pixel signal of G pixel and Ye pixel (hereinafter referred to as a second mixed pixel signal) M2, a mixed pixel signal of Mg pixel and Ye pixel (hereinafter referred to as a third mixed pixel signal) M3, A mixed pixel signal (hereinafter referred to as a fourth mixed pixel signal) M4 of the G pixel and the Cy pixel is output.

  In the case of the simultaneous method in the normal light observation mode, as shown in FIG. 6, the blue light BL, the green light GL, and the red light RL are simultaneously irradiated, and the odd and even fields are read during this irradiation period. . An image for one frame is generated by the read odd field and even field.

  In the normal light observation mode and the field sequential method, as shown in FIG. 7, the blue light BL, the green light GL, and the red light RL are sequentially irradiated, and the odd field and the even field are read out during each irradiation period. Is done. An image for one frame is generated for each irradiation period by the read odd field and even field. Note that the order of emission of the blue light BL, the green light GL, and the red light RL is not limited to this order, and may be changed as appropriate.

  In the narrow-band light observation mode, the illumination light is the same as the normal light observation mode except that the illumination light becomes two types of blue narrow-band light Bn and green narrow-band light Gn.

  The signal output from the complementary color image sensor 28 is input to the CDS circuit 32. The CDS circuit 32 performs correlated double sampling on the input signal to remove noise components generated by the CCD image sensor. The signal from which the noise component has been removed by the CDS circuit 32 is input to the A / D conversion circuit 33 and also to the brightness detection circuit 34. The A / D conversion circuit 33 converts the signal input from the CDS circuit 32 into a digital signal and inputs the digital signal to the image signal processing unit 36.

  The brightness detection circuit 34 detects brightness (average luminance of the signal) based on the signal input from the CDS circuit 32. The dimming circuit 35 generates a dimming signal that is a difference between the brightness signal detected by the brightness detection circuit 34 and the reference brightness (target dimming value). This dimming signal is input to the light source control unit 21. The light source control unit 21 adjusts the light emission amounts of the plurality of LED light sources 20 so that the reference brightness is obtained.

  The control unit 30 receives a mode switching signal that is generated when the mode switching switch 17a of the endoscope 13 is operated, and an observation mode (normal light observation mode or narrowband light observation mode) indicated by the received mode switching signal. Based on the above, the light source control unit 21 is controlled.

  In addition, at the start of inspection, the control unit 30 controls the imaging control unit 31 and the light source control unit 21 to repeatedly perform the complementary color imaging device 28 and the LED light source 20 to execute moving image shooting. When receiving a freeze instruction signal from the freeze switch 17b of the endoscope 13 during moving image shooting, the control unit 30 causes the still image selection unit 38 to select a still image.

  The illumination method (simultaneous method or frame sequential method) for moving image shooting and still image shooting can be set from the input device 15. Specifically, as shown in FIG. 8, the control unit 30 includes a moving image illumination method input unit 41, a moving image illumination method storage unit 42, a still image illumination method input unit 43, and a still image illumination. A system storage unit 44, a batch input unit 45, and a still image shooting instruction unit 46 are provided. The illumination method can be individually input from the input device 15 to the moving image illumination method input unit 41 and the still image illumination method input unit 43.

  The collective input unit 45 stores a plurality of patterns so that the illumination method for moving image shooting and still image shooting can be set by a single input operation. This pattern can be selected from the input device 15, and the batch input unit 45 selects a lighting method corresponding to the pattern selected by the input device 15 as a moving image lighting method input unit 41 and a still image lighting method input unit. Input to 43 respectively.

  The illumination methods input to the moving image illumination method input unit 41 and the still image illumination method input unit 43 are stored in the moving image illumination method storage unit 42 and the still image illumination method storage unit 44, respectively. Is input.

  A freeze instruction signal is input to the still image shooting instruction unit 46 from the freeze switch 17b. When receiving the freeze instruction signal, the still image shooting instruction unit 46 transfers the freeze instruction signal to the still image selection unit 38 and the light source control unit 21. The light source control unit 21 receives an input of a freeze instruction signal and illuminates when the moving image illumination method and the still image illumination method input from the moving image illumination method storage unit 42 and the still image illumination method storage unit 44 are different. Switch the method. In addition, when the still image shooting is completed, the illumination method is returned to the moving image illumination method.

  As shown in FIG. 9, the image signal processing unit 36 includes a Y / C conversion unit 50, a matrix calculation unit 51, a frame generation unit 52, an image memory 53, and a synchronization processing unit 54. Yes. First to fourth mixed pixel signals M1 to M4 (see FIG. 5) are input to the Y / C conversion unit 50 from the complementary color imaging device 28 via the CDS circuit 32 and the A / D conversion circuit 33. .

  The Y / C conversion unit 50 performs Y / C conversion by a known calculation used in the complementary color checkered color difference line sequential method to generate the luminance signal Y and the color difference signals Cr and Cb. Specifically, the luminance signal Y and the color difference signals Cr and Cb are added and subtracted between the first pixel signal M1 and the second pixel signal M2 adjacent in the row direction, and the third pixel signal M3 and the fourth pixel adjacent in the row direction. It is calculated by addition / subtraction with the pixel signal M4.

  The matrix calculation unit 51 generates RGB signals by performing a predetermined matrix calculation on the luminance signal Y and the color difference signals Cr and Cb generated by the Y / C conversion unit 50. The Y / C conversion unit 50 and the matrix calculation unit 51 perform Y / C conversion and matrix calculation for the odd field and the even field, respectively.

  The frame generation unit 52 generates one frame of image data by synthesizing the RGB signals obtained for each of the odd field and the even field. The image memory 53 can store a plurality of image data, and sequentially stores the image data generated by the frame generation unit 52.

  When the illumination method is the simultaneous method, each frame of image data includes each color component (R, G, B components in the normal light observation mode, and in the narrow band light observation mode). (B, G components) are all included. On the other hand, in the case of the frame sequential method, one frame of image data includes only one color component of illumination light. Therefore, in the case of the simultaneous method, the image data stored in the image memory 53 is output from the image signal processing unit 36. In the case of the frame sequential method, the image data stored in the image memory 53 is input to the synchronization processing unit 54.

  The synchronization processing unit 54 selects color components corresponding to illumination light from image data of a plurality of consecutive frames (three frames in the normal light observation mode and two frames in the band light observation mode). Is extracted, and the extracted image data of each color component is combined to generate synchronized image data for one frame. In the case of the frame sequential method, the synchronized image data is output from the image signal processing unit 36.

  The blur amount calculation unit 37 calculates an image blur amount based on the image data stored in the image memory 53. In the present embodiment, in the case of the simultaneous method, the blur amount calculation unit 37 calculates an image blur amount for each image data based on the G signal included in the image data. On the other hand, in the case of the frame sequential method, the image blur amount is calculated based on the G signal included in the image data at the time of green light GL irradiation. Since the G signal has the highest correlation with the luminance signal among the RGB signals, the blur amount calculation unit 37 calculates the image blur amount using the G signal.

  The amount of image blur is expressed, for example, as a reciprocal of an integrated value obtained by extracting high frequency components of a predetermined value or more from the spatial frequency of the G signal and integrating the high frequency components. This is based on the fact that when blurring occurs, the image is blurred and the high-frequency component is reduced, and the integrated value of the high-frequency component is reduced.

  When the illumination mode cannot be switched with the input of the freeze instruction signal, the still image selection unit 38, for example, a plurality of frames before and after receiving the freeze instruction signal among the plurality of image data stored in the image memory 53. The image data with the smallest image blur amount is selected from the image data for the minute.

For example, in the case of concurrency in the normal light observation mode, FIG. 10 (A), the image data _P 1 _{to P 6,} the image blur amount from the P -1 _{to P -6} after reception of the freeze instruction signal Is selected. The selected image data is stored in the still image memory 39 as a still image. On the other hand, in the case of the frame sequential method in the normal light observation mode, the image data P ₁ at the time of green light GL irradiation among the image data P _{1 to} P ₆ and P _{−1 to} P ₋₆ shown in FIG. , P ₄ , P _−3, P ₋₆ , the image data with the smallest image blur amount is selected. When image data is selected by this frame sequential method, the synchronization processing unit 54 performs synchronization processing together with the image data before and after the image data. The synchronized image data for one frame is stored in the still image memory 39 as a still image.

When the illumination mode is switched in response to the input of the freeze instruction signal, the still image selection unit 38 selects a still image according to the procedure shown in the flowchart shown in FIG. When moving image shooting is started in the simultaneous method or the frame sequential method, a freeze instruction signal is waited for. When the freeze instruction signal is received, the illumination method is switched by the light source control unit 21. For example, as shown in FIG. 12, the simultaneous method is switched to the frame sequential method in response to reception of the freeze instruction signal, and image data P _{1 to} P ₆ and P _{−1 to} P _{−6 of} a plurality of frames before and after the switching are changed. Stored in the image memory 53.

First, the blur amount calculation unit 37 calculates the image blur amount (hereinafter referred to as the first image blur amount of the image data after switching) from each image data of the switched image data P _{1 to} P ₆ (first image data group). Is calculated). The still image selection unit 38 specifies the minimum value LV1 among the calculated first blur amounts. Then, the still image selection unit 38 determines whether or not the minimum value LV1 is smaller than the reference value. If the minimum value LV1 is smaller than the reference value, the image for which the first blur amount of the minimum value LV1 is calculated. Select data as a still image.

On the other hand, when the minimum value LV1 is equal to or larger than the reference value, the blur amount calculation unit 37 calculates the image blur amount (from the image data P _{−1 to} P ₋₆ (second image data group) before switching (the image blur amount (second image data group)). Hereinafter, the image blur amount of the image data before switching is referred to as “second blur amount”). The still image selection unit 38 specifies the minimum value LV2 among the calculated second blur amounts. Then, the still image selection unit 38 determines whether or not the minimum value LV2 is smaller than the reference value. If the minimum value LV2 is smaller than the reference value, the image for which the second shake amount of the minimum value LV2 is calculated. Select data as a still image.

  Further, when the minimum value LV2 is greater than or equal to the reference value, the still image selection unit 38 compares the minimum value LV1 with the minimum value LV2, and the smaller minimum value between the minimum value LV1 and the minimum value LV2 The image data for which the amount of blur is calculated is selected as a still image. The image data selected by the still image selection unit 38 is stored in the still image memory 39.

  In the present embodiment, the reference value for comparing the minimum value LV1 of the first blur amount and the reference value for comparing the minimum value LV2 of the second blur amount are the same, but they may be different.

  The still image selection operation by the still image selection unit 38 described above is the same when the frame sequential method is switched to the simultaneous method. The same applies to the case of the narrow-band light observation mode except that there are two types of illumination light.

  At the time of moving image shooting, the display control unit 40 causes the image display device 14 to display (moving image display) an image based on the image data sequentially output from the image signal processing unit 36. When image data selected as a still image is stored in the still image memory 39 after receiving the freeze instruction signal, the display control unit 40 displays an image based on this image data on the image display device 14 (still image). Display). Thereafter, when the freeze switch 17b is operated again and a freeze release signal is input, the display control unit 40 returns from the still image display to the moving image display.

  Next, the operation of the endoscope system 10 will be described along the flowchart shown in FIG. First, the surgeon sets the illumination method at the time of moving image shooting and still image shooting using the input device 15. This illumination method can be set differently between the normal light observation mode and the narrow-band light observation mode.

  The surgeon performs an endoscopic examination by inserting the insertion portion 16 of the endoscope 13 into the body cavity of the patient. When the endoscope 13 is connected to the light source device 11 and the processor device 12, the light source device 11 and the processor device 12 are set to the normal light observation mode, and the light source device 11 is based on the illumination method set by the input device 15. Illumination light is supplied to the endoscope 13 and emitted toward the specimen, and imaging is performed by the complementary color imaging element 28 of the endoscope 13. This imaging is repeatedly performed, and the image data sequentially generated by the image signal processing unit 36 is displayed as a moving image on the image display device 14.

  When the freeze switch 17b is operated by the surgeon, a freeze instruction signal is input to the control unit 30, and when the illumination method set by the input device 15 differs between moving image shooting and still image shooting, the light source device 11 illumination methods are switched. Then, an image blur amount is calculated for each image data of a plurality of frames before and after the input of the freeze instruction signal by the blur amount calculation unit 37, and based on this image blur amount, image data to be a still image is obtained by the still image selection unit 38. Selected. The selected image data is stored in the still image memory 39 and a still image is displayed on the image display device 14. Thereafter, when the freeze switch 17 b is operated again by the operator and a freeze release signal is input to the control unit 30, the moving image display is resumed on the image display device 14.

  When the surgeon wants to observe in more detail the running state of the surface blood vessels of the tissue to be examined such as the affected part in the body cavity, the mode switch 17a is operated. When the mode switch 17a is operated, the operation signal is detected by the control unit 30, and the light source device 11 and the processor device 12 are switched to the narrowband light observation mode. In the narrow-band light observation mode, the blue narrow-band light Bn and the green narrow-band light Gn are supplied from the light source device 11 to the endoscope 13, and moving image display and still image display are similarly performed.

  As described above, in the endoscope system 10 of the present invention, image data with less image blur is selected as a still image regardless of the switching of the illumination method. In the frame sequential method, when an image blur occurs, a color shift is caused accordingly, so that the color shift is also reduced at the same time.

  In the above-described embodiment, the blur amount calculation unit 37 calculates the image blur amount based on the G signal of the RGB signals, but one color signal, two color signals of the RGB signals, Alternatively, the image blur amount may be calculated using signals of all three colors.

  In the normal light observation mode, as shown in FIG. 14, first, the image data is color-separated into an R image, a G image, and a B image. In the case of the simultaneous method, one frame of image data is color-separated into an R image, a G image, and a B image. In the case of the frame sequential method, the image data at the time of red light RL irradiation is color-separated to generate an R image, the image data at the time of green light GL irradiation is color-separated to generate a G image, and the blue light BL is irradiated. The B image is generated by color-separating the current image data.

  When the R image, the G image, and the B image are obtained, the image blur amount is calculated according to the procedure described above based on the color signal of each image, and the image blur amount of each calculated color signal (referred to as blur amounts Er, Eg, and Eb, respectively). ) Is memorized. Then, the total blur amount E (= Wr × Er + Wg × Eg + Wb × Eb) is calculated by multiplying each blur amount Er, Eg, Eb by weighting coefficients Wr, Wg, Wb and adding them. This total blur amount E is used as the aforementioned image blur amount. Here, the weighting coefficients Wr, Wg, and Wb satisfy the relationship of Wr + Wg + Wb = 1. For example, when the total blur amount E is calculated by a combination of the B signal and the G signal, (Wr, Wg, Wb) = (0, 0.5, 0.5) may be set.

  The narrow-band light observation mode is the same as the normal light observation mode except that only the G image and the B image are generated from the image data. In this narrow-band light observation mode, the blue narrow-band light Bn contributes to the imaging of the superficial blood vessels of the mucous membrane. Therefore, in order to photograph the superficial blood vessels with high image quality, (Wg, Wb) = (0, 1 ).

  The weighting coefficients Wr, Wg, and Wb may be directly input from the input device 15 or are tabulated in association with the observation mode and image type selected by the input device 15, and this table is a blur amount calculation unit 37. It may be stored in advance.

  The calculation of the image blur amount is not limited to the RGB signal, and the image blur amount may be calculated based on the luminance signal Y. In this case, since it is necessary to generate the luminance signal Y by Y / C converting the RGB signal of the image data, the time until the image blur amount is calculated is longer than that in the above case, but the color dependence is increased. Therefore, the calculation accuracy of the image blur amount is improved.

  In the above embodiment, the amount of image blur is calculated by integrating the high frequency components of the spatial frequency. Instead, as described in Japanese Patent No. 3497231, they are temporally adjacent. The image blur amount may be calculated by detecting a difference (motion component) between the two image data.

  Also, when detecting the motion component of the image data in this way, the synchronization processing unit 54 aligns and synthesizes (synchronizes) the image data of each color component based on the motion component of each image data. With this configuration, it is possible to reduce color misregistration caused by image blur in the case of the frame sequential method. This improves the color reproducibility, which is a drawback for the frame sequential method.

  Specifically, as shown in FIG. 15, when the illumination method is switched from the simultaneous method to the frame sequential method in the normal light observation mode, an image for one frame obtained by the simultaneous method with no color misregistration problem. The R, G, and B image misregistration amounts Dr, Dg, and Db that are used for synchronization are calculated by the frame sequential method, using the R image, G image, and B image obtained by decomposing data as references. This positional deviation amount is, for example, a value obtained by averaging motion components (motion vectors) for each pixel of each image. Then, the image data with reduced color misregistration is generated by aligning and correcting the images so as to cancel out the misregistration amounts Dr, Dg, and Db and then synthesizing (simulating) the images.

  In the case of the narrow-band light observation mode, it is possible to reduce color misregistration by performing the same processing. Further, in the case of the narrow band light observation mode, as described above, the blue narrow band light Bn contributes to the imaging of the surface blood vessels of the mucous membrane. When the B image and the G image whose position deviation is corrected in the frame sequential method are combined, the B image of the simultaneous method may be added.

  In the above embodiment, the blur amount calculation unit 37 calculates the image blur amount, and the still image selection unit 38 selects image data to be a still image based on the image blur amount. In the observation mode, it is also preferable to select image data to be a still image in consideration of the color separation amount in addition to the image blur amount. This color separation amount C is expressed by equation (1).

  Here, Sb is the standard deviation of the pixel value of the B image, and Sg is the standard deviation of the pixel value of the G pixel. Therefore, the color separation amount C is expressed as the root mean square of the standard deviations Sb and Sg, and is a value related to the contrast width of the composite image of the B image and the G image.

  In the narrow-band light observation mode, the depth at which the blue narrow-band light Bn and the green narrow-band light Gn enter the specimen is different, the B image includes a lot of surface layer structures, and the G image has a medium-deep layer structure. Many are included. For this reason, in particular, in the simultaneous method, each pixel detects both the blue narrow-band light Bn and the green narrow-band light Gn that are simultaneously irradiated (that is, color mixing), and the two included in the B image and the G image The images of different structures are mixed. For example, the surface layer has a fine structure such as a capillary, and the middle deep layer has a larger structure. Therefore, when both are mixed, the high-frequency component contained in the fine structure is reduced. Thus, when the color separation is poor, the contrast width is narrowed and the color separation amount C is reduced.

Therefore, the still image selection unit 38, for example, “H = W ₁ × E−W ₂ × C” based on the total blur amount E and the color separation amount C in the case of the simultaneous mode in the narrow-band light observation mode. It is preferable to select image data to be a still image using the comprehensive evaluation value H expressed by Here, W ₁ and W ₂ are weighting coefficients. In addition, when switching the illumination method in the narrow-band light observation mode, when selecting one image data from a plurality of image data obtained by the simultaneous method (specify the minimum value of the blur amount in the simultaneous method) Only when the color separation amount C is considered.

  In addition, as shown in FIG. 17, the endoscope 13 uses a substantially circular effective area 61 corresponding to the observation window among the rectangular area 60 captured by the complementary color imaging element 28 for imaging the specimen image. Since the outside of the effective area 61 in the rectangular area 60 is dark and the pixel value takes a substantially constant value (spatial frequency is low), even if the image blur amount is calculated for the entire rectangular area 60, the pixels outside the effective area 61 The value hardly affects the calculation result of the image blur amount. However, color deviation may exist in the pixel values outside the effective area 61, which may affect the calculation result of the color separation amount C.

  Therefore, it is preferable that the color separation amount C is targeted only within the effective region 61 in order to increase detection accuracy. Although the color separation amount C may be calculated for the entire effective area 61, the color separation amount C may be calculated for the maximum rectangle in the effective area 61 for simplification of processing. In addition, since the examination region of the specimen is often arranged near the center in the effective region 61, the color separation amount C may be calculated for the vicinity of the center in the effective region 61. Further, since the specimen image exists in a certain luminance range (an area that is neither too dark nor too bright) in the effective area 61, an area of this luminance range is extracted from the effective area 61, and only this area is targeted. The color separation amount C may be calculated.

  As shown in FIG. 18, a special image processing unit 70 for generating a special image such as FICE (Flexible Spectral Imaging Color Enhancement) may be provided in the processor device. In this case, the first still image 71 stored in the still image memory 39 and the second still image 72 obtained by performing image processing on the first still image 71 by the special image processing unit 70 are displayed on the image display device 14. . This is called a dual mode function.

  Further, the second still image 72 is not limited to the image obtained by subjecting the first still image 71 to the image processing, and the image processing is performed on the image obtained by the illumination method separately designated for the special image by the input device 15 or the like. It may be given. For example, since FICE is a process for emphasizing color, priority is given to the frame rate over color reproducibility, and simultaneous image data is designated for special images.

  In the above embodiment, when the illumination method is switched in response to the reception of the freeze instruction signal, the blur amount calculation unit 37 determines the second blur amount of the image data before switching after the freeze instruction signal is received. Although the calculation is performed, the second shake amount is sequentially calculated every time image data is generated during moving image shooting before the freeze instruction signal is received, and the image after switching is received after the freeze instruction signal is received. Only the calculation of the first blur amount of data may be performed. By doing so, the time required from the freeze instruction to the still image display is shortened.

  In the above embodiment, when the illumination method is switched in response to the reception of the freeze instruction signal, the still image selection unit 38 is either the simultaneous method or the frame sequential method based on the first and second blur amounts. Image data is selected as a still image, but if the minimum values of the first and second blur amounts are both smaller than the reference value, the image data having the minimum value from the simultaneous method and the frame sequential method are still The selected still images may be displayed on the image display device 14 selectively or simultaneously.

  In the above embodiment, the blue light BL emitted from the B-LED 20a is used as it is as the blue narrowband light Bn in the narrowband light observation mode. However, when the wavelength range of the blue light BL is relatively wide, As shown in FIG. 19, a blue narrow band filter 80 and a filter insertion / removal unit 81 may be provided in the light source device 11.

  The filter insertion / removal unit 81 inserts / removes the blue narrow band filter 80 on the optical path of the blue light BL emitted from the B-LED 20a. The blue narrow band filter 80 is inserted into the optical path of the blue light BL in the narrow band light observation mode, and is removed from the optical path of the blue light BL in the normal light observation mode. In the narrowband light observation mode, the blue narrowband filter 80 generates blue narrowband light Bn by limiting the wavelength of the blue light BL emitted from the B-LED 20a. Note that the filter insertion / removal unit 81 may be omitted, and the blue narrowband filter 80 may be inserted / removed in conjunction with the green narrowband filter 22 by the filter insertion / removal unit 23 for the green narrowband filter 22 described above.

  In the above-described embodiment, the blue narrowband light Bn and the green narrowband light Gn are used as the narrowband light in the narrowband light observation mode, but instead of the blue narrowband light Bn, purple narrowband light ( A central wavelength of around 405 nm may be used.

  In the above embodiment, the imaging control unit 31, the CDS circuit 32, the A / D conversion circuit 33, and the like are provided in the processor device 12, but these may be provided in the endoscope 13.

  In the above-described embodiment, the complementary color image sensor 28 is used, but a primary color image sensor may be used instead. Further, the image sensor may be a CMOS image sensor. In the case of a CMOS image sensor, the imaging control unit 31, the CDS circuit 32, the A / D conversion circuit 33, and the like can be formed in a CMOS semiconductor substrate on which the image sensor is formed.

  Moreover, in the said embodiment, although the LED light source 20 is used for the light source device 11, other semiconductor light sources, such as LD (Laser Diode), may be used instead of LED.

  Moreover, in the said embodiment, although the light source device 11 and the processor apparatus 12 are comprised as a separate apparatus, they are good also as a single apparatus. Further, the light source device 11 may be incorporated in the endoscope 13.

DESCRIPTION OF SYMBOLS 10 Endoscope system 11 Light source apparatus 12 Processor apparatus 13 Endoscope 14 Image display apparatus 17a Mode changeover switch 17b Freeze switch 20 LED light source 21 Light source control part 22 Green narrow-band filter 27 Light guide 28 Complementary color system image sensor 28a Complementary color system color Separation filter 36 Image signal processing unit 37 Blur amount calculation unit 38 Still image selection unit 39 Still image memory 54 Synchronization processing unit

Claims (13)

A light source device that generates illumination light of multiple colors;
A simultaneous imaging device having a color separation filter;
An image signal processing unit that generates image data based on a pixel signal read from the simultaneous image sensor;
In response to the input of the freeze instruction signal for a still image by freeze the video, and concurrency for simultaneous irradiation of the plurality of colors illumination light, the field sequential method is time division irradiating said plural colors illumination light A controller for switching the illumination method of the light source device between,
A first blur amount is calculated from each image data of the first image data group generated after switching the illumination method, and a second blur is calculated from each image data of the second image data group generated before switching the illumination method. A blur amount calculation unit for calculating the amount;
A still image selection unit that selects image data to be a still image from the first and second image data groups based on the first and second blur amounts;
With
When the minimum value of the first blur amount is smaller than a reference value, the still image selection unit selects image data having the minimum value from the first image data group as a still image, and the first blur amount If the minimum value of the second blur amount is less than the reference value, it is determined whether or not the minimum value of the second blur amount is smaller than the reference value. When image data having the minimum value is selected as a still image from the data group, and the minimum value of the second blur amount is equal to or greater than a reference value, the smaller one of the minimum values of the first and second blur amounts The image system is selected from the first and second image data groups . In the case of the frame sequential method, comprising a synchronization processing unit that generates image data that is synchronized by synthesizing image data obtained at the time of irradiation of the illumination lights of the plurality of colors,
The synchronization processing unit switches a plurality of image data to be combined when the illumination method is switched from the simultaneous method to the frame sequential method and the frame sequential method image data is selected by the still image selection unit. Each of the image data obtained by the simultaneous method is compared with a corresponding color separation image to calculate a positional deviation amount, and after being aligned based on the calculated positional deviation amount, they are combined. The endoscope system according to claim 1 .   The light source device has a normal light observation mode for generating red light, green light, and blue light, and a narrowband light observation mode for generating first and second narrowband light in the light source device. The endoscope system according to claim 2. In the case of the narrowband light observation mode, the synchronization processing unit outputs one color separation image corresponding to the first narrowband light or the second narrowband light of the image data generated by the simultaneous method, The endoscope system according to claim 3 , wherein the endoscope system is added to one of a plurality of image data to be synthesized after the alignment. In the case of the narrowband light observation mode and the simultaneous method, the still image selection unit performs a still image based on a color separation amount related to a contrast width in addition to the first blur amount or the second blur amount. The endoscope system according to claim 3 , wherein image data to be a picture is selected. The blur amount calculating unit includes one color signal included in the image data generated by the simultaneous method, and the color signal of the color included in the image data when the illumination light of the color is irradiated by the frame sequential method. The endoscope system according to any one of claims 1 to 5 , wherein the first and second blur amounts are calculated on the basis of the first and second blur amounts. The said blur amount calculation part calculates the said 1st and 2nd blur amount based on the integrated value which integrated the high frequency component of the spatial frequency in image data, The any one of Claim 1 to 6 characterized by the above-mentioned. The endoscope system described. The shake amount calculation unit, by detecting the difference between the two image data temporally adjacent, 6 any one of claims 1, characterized by calculating said first and second shake amount The endoscope system described in 1. The blur amount calculation unit calculates an image blur amount based on each color signal included in the image data, and multiplies each image blur amount by a weighting coefficient to add the first and second blur amounts. the endoscope system according to any of the preceding claims 1, wherein the calculating. The color separation filter, an endoscope system according to any one of claims 1 9, characterized in that the complementary colors. 2. A dual mode in which image data selected as a still image by the still image selection unit and special image processing on the image data are simultaneously displayed on an image display device. The endoscope system according to any one of 10 . The shake amount calculation unit before the freeze instruction signal is input from claim 1, characterized in that to calculate the second shake amount each time the image data is generated by the image signal processing unit 11 The endoscope system according to any one of the above. A first step in which the light source device generates illumination light of a plurality of colors;
A second step in which the image signal processing unit generates image data based on the pixel signal read from the simultaneous imaging device having a color separation filter;
The control unit is configured to simultaneously irradiate a plurality of colors of illumination light in response to an input of a freeze instruction signal for freezing a moving image to be a still image, and a surface sequential to irradiate the plurality of colors of illumination light in a time-sharing manner. A third step of switching the illumination method of the light source device between the methods,
A blur amount calculation unit calculates a first blur amount from each image data of the first image data group generated after switching the illumination method, and each of the second image data groups generated before switching the illumination method. A fourth step of calculating a second blur amount from the image data;
Still image selecting unit, based on the first and second shake amount, from the first and second image data group, and a fifth step of selecting the image data to be a still image,
With
In the fifth step, when the minimum value of the first blur amount is smaller than a reference value, the still image selection unit selects image data having the minimum value from the first image data group as a still image. If the minimum value of the first blur amount is greater than or equal to a reference value, it is determined whether or not the minimum value of the second blur amount is smaller than the reference value. If the minimum value is smaller than the reference value, Selects the image data having the minimum value from the second image data group as a still image, and if the minimum value of the second blur amount is equal to or greater than a reference value, the first and second blur amounts are selected. A method of operating an endoscope system , wherein the smaller image data among the minimum values is selected from the first and second image data groups .

Images