JP6099445B2 - Imaging system - Google Patents

Description

  The present invention relates to an imaging system capable of outputting an electrical signal after photoelectric conversion as image information from a pixel arbitrarily designated as a readout target among a plurality of pixels for photographing.

  Conventionally, in the medical field, an endoscope system is used when observing an organ of a subject such as a patient. An endoscope system has, for example, a flexible elongated shape, an imaging device that is inserted into a body cavity of a subject, an imaging element that captures an in-vivo image provided at the tip of the imaging device, and a subject. A light source device that emits illumination light to illuminate, a processing device that performs predetermined image processing on an in-vivo image captured by an imaging device, and a display device that can display an in-vivo image that has been subjected to image processing by the processing device. When acquiring an in-vivo image using an endoscope system, after inserting an insertion portion into a body cavity of a subject, illumination light is irradiated from the distal end of the insertion portion to a living tissue in the body cavity, and the imaging device In-vivo images are taken. A user such as a doctor observes the organ of the subject based on the in-vivo image displayed by the display device.

  In some cases, a CMOS (Complementary Metal Oxide Semiconductor) sensor is applied as an imaging element included in such an endoscope system. The CMOS sensor generates image data by a rolling shutter system in which exposure or reading is performed at different timings for each line.

  In the above-described endoscope system, a technique for performing illumination processing and readout processing for each frame in order to prevent color mixing between adjacent frames is disclosed (for example, see Patent Document 1). Moreover, in order to prevent the light from the imaging target from being buried by the background light, a technique is disclosed in which image signals of adjacent frames are added to obtain one image (see, for example, Patent Document 2).

JP 2010-068992 A JP 2002-281387 A

  FIG. 15 is a diagram schematically illustrating an emission timing of illumination light emitted from a light source device at the time of photographing with a conventional endoscope system, exposure and readout timing of an image sensor, and processing by an image processing unit. When the image data D101 to D109 are generated by the rolling shutter method, a series of processes from exposure to reading of charges with a CMOS type image sensor to reading of charges are performed at different timings for each line (FIG. 15). In order to obtain an image signal such as F1), a period longer than the readout period for one frame is substantially required. As a result, when the image sensor or the subject moves, the image is noticeable. In particular, when the movement of the image sensor or the subject is fast, there is a problem that image distortion increases.

  In addition, as shown in FIG. 15, in the case where illumination is continuously performed in a period Te100 corresponding to one frame period, a plurality of types of illumination light having mutually different light wavelengths and intensities are periodically emitted. In this case, during the readout period of a certain frame, the illumination process shifts to the next illumination period, which causes a problem that the illumination light is mixed in one frame image.

  In order to prevent distortion and color mixing of these images, there is a method of performing illumination processing in a simultaneous exposure period (period Tm100 in FIG. 15) in which the exposure times are the same in all lines from the read start line to the end line. In some cases, the illumination period is short and a clear image cannot be obtained.

  The present invention has been made in view of the above, and even if a series of processing from exposure by an image sensor to reading out charges is performed at different timings for each line, image blurring, distortion, and color mixing are performed. An object is to provide an imaging system that can be suppressed.

  In order to solve the above-mentioned problems and achieve the object, an imaging system according to the present invention generates an imaging signal by receiving an illumination unit that emits illumination light that illuminates a subject, and performing photoelectric conversion by receiving the light. A light receiving unit in which pixels to be arranged in a two-dimensional manner, a reading unit that sequentially reads out the imaging signal for each horizontal line from the light receiving unit, and the reading unit starts from the start of reading the first horizontal line of the light receiving unit. A first period that is one frame period corresponding to a period until the start of reading of the first horizontal line, a second period that is equal to or longer than the one frame period after causing the illumination unit to turn off the illumination light, An illumination control unit that causes the illumination unit to execute a series of processes for causing the illumination unit to emit the illumination light; and a first imaging signal in the first period read from the light receiving unit by the reading unit; An image processing unit that generates an image signal corresponding to the illumination light emitted in the second period by adding the second imaging signal in the second period. To do.

  In the imaging system according to the present invention as set forth in the invention described above, the illumination control unit causes the illumination unit to execute the series of processes in synchronization with the readout timing of the first and second periods by the readout unit. It is characterized by that.

  The imaging system according to the present invention further includes an input unit that receives an input of an instruction signal for performing at least still image shooting in the above invention, and the illumination control unit receives the input of the instruction signal by the input unit. If not, the illumination unit emits illumination light continuously, and if the input unit receives the input of the instruction signal, the illumination unit executes the series of processes, and the image processing unit When the instruction signal is not input to the input unit, a display image signal is generated based on the imaging signal according to the illumination light continuously emitted, and the instruction signal is input to the input unit. In this case, the first image pickup signal and the second image pickup signal are added to sequentially generate display image signals.

  According to the present invention, imaging signals corresponding to a plurality of frames including the same illumination period are added to generate one image signal. Even if this processing is performed at different timing for each line, there is an effect that blurring, distortion, and color mixing of the image can be suppressed.

FIG. 1 is a diagram illustrating a schematic configuration of an endoscope system according to the first embodiment of the present invention. FIG. 2 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the first embodiment of the present invention. FIG. 3 schematically shows the emission timing of the illumination light emitted from the light source device, the exposure and readout timing of the image sensor, and the processing by the image processing unit at the time of photographing by the endoscope system according to the first embodiment of the present invention. FIG. FIG. 4 is a diagram for explaining the emission timing of the illumination light emitted from the light source device and the exposure and readout timing of the image sensor at the time of photographing by the endoscope system according to the first embodiment of the present invention. FIG. 5 illustrates the illumination light emission timing emitted by the light source device during imaging of the endoscope system according to the modified example 1-1 of the first embodiment of the present invention, the exposure and readout timing of the image sensor, and the image processing unit. It is a figure which shows a process typically. FIG. 6 illustrates the illumination light emission timing emitted from the light source device at the time of photographing by the endoscope system according to Modification 1-2 of the first embodiment of the present invention, the exposure and readout timing of the image sensor, and the image processing unit. It is a figure which shows a process typically. FIG. 7 illustrates the illumination light emission timing emitted from the light source device during imaging of the endoscope system according to Modification 1-3 of Embodiment 1 of the present invention, the exposure and readout timings of the image sensor, and the image processing unit. It is a figure which shows a process typically. FIG. 8 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the second embodiment of the present invention. FIG. 9 schematically shows the emission timing of the illumination light emitted from the light source device, the exposure and readout timing of the image sensor, and the processing by the image processing unit at the time of photographing by the endoscope system according to the second embodiment of the present invention. FIG. FIG. 10 shows the illumination light emission timing emitted by the light source device during imaging of the endoscope system according to the modified example 2-1 of the second embodiment of the present invention, the exposure and readout timing of the image sensor, and the image processing unit. It is a figure which shows a process typically. FIG. 11 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the third embodiment of the present invention. FIG. 12 schematically shows the emission timing of illumination light emitted by the light source device, the exposure and readout timing of the image sensor, and the processing by the image processing unit at the time of photographing by the endoscope system according to the third embodiment of the present invention. FIG. FIG. 13 schematically shows the emission timing of the illumination light emitted from the light source device, the exposure and readout timing of the image sensor, and the processing by the image processing unit at the time of photographing by the endoscope system according to the fourth embodiment of the present invention. FIG. FIG. 14 shows the emission timing of the illumination light emitted from the light source device at the time of photographing by the endoscope system according to Modification 4-1 of the fourth embodiment of the present invention, the exposure and readout timing of the image sensor, and the image processing unit. It is a figure which shows a process typically. FIG. 15 is a diagram schematically illustrating an emission timing of illumination light emitted from a light source device at the time of photographing with a conventional endoscope system, exposure and readout timing of an image sensor, and processing by an image processing unit.

  Hereinafter, modes for carrying out the present invention (hereinafter referred to as “embodiments”) will be described. In the embodiment, a medical endoscope system that captures and displays an image in a body cavity of a subject such as a patient will be described as an example of an imaging system. Moreover, this invention is not limited by this embodiment. Furthermore, in the description of the drawings, the same portions will be described with the same reference numerals.

(Embodiment 1)
FIG. 1 is a diagram illustrating a schematic configuration of an endoscope system according to the first embodiment of the present invention. FIG. 2 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the first embodiment of the present invention.

  An endoscope system 1 shown in FIGS. 1 and 2 inserts a distal end portion into a body cavity of a subject to capture an in-vivo image of the subject, and emits the light from the distal end of the endoscope 2. A light source device 3 that generates illumination light, a processing device 4 that performs predetermined image processing on an in-vivo image captured by the endoscope 2, and controls the overall operation of the endoscope system 1, and a processing device 4 Includes a display device 5 that displays an in-vivo image subjected to image processing.

  The endoscope 2 includes an insertion portion 21 having an elongated shape having flexibility, an operation portion 22 that is connected to a proximal end side of the insertion portion 21 and receives input of various operation signals, and an insertion portion from the operation portion 22. And a universal cord 23 that includes various cables that extend in a direction different from the direction in which 21 extends and connect to the light source device 3 and the processing device 4.

  The insertion unit 21 receives a light and performs photoelectric conversion to generate a signal to generate a signal. The insertion unit 21 includes an image pickup element 244 in which pixels are arranged in a two-dimensional shape, and a bendable portion formed by a plurality of bending pieces. And a long flexible tube portion 26 connected to the proximal end side of the bending portion 25 and having flexibility.

  The distal end portion 24 is configured using a glass fiber or the like, and forms a light guide path for light emitted from the light source device 3. An illumination lens 242 provided at the distal end of the light guide 241. And an image sensor 244 that is provided at an image forming position of the optical system 243, receives light collected by the optical system 243, photoelectrically converts the light into an electrical signal, and performs predetermined signal processing.

  The optical system 243 is configured using one or a plurality of lenses, and has an optical zoom function for changing the angle of view and a focus function for changing the focus.

  The image sensor 244 photoelectrically converts light from the optical system 243 to generate an electrical signal (imaging signal), and performs noise removal and A / D conversion on the electrical signal output from the sensor unit 244a. An analog front end unit 244b (hereinafter referred to as “AFE unit 244b”), a P / S conversion unit 244c that performs parallel / serial conversion on a digital signal (image signal) output from the AFE unit 244b and transmits the converted signal to the outside, and a sensor unit A timing generator 244d that generates pulses of various signal processing in the AFE unit 244b and the P / S conversion unit 244c, and an imaging control unit 244e that controls the operation of the imaging device 244; The image sensor 244 is a CMOS sensor.

  In the sensor unit 244a, a plurality of pixels each having a photodiode that accumulates electric charge according to the amount of light and an amplifier that amplifies the electric charge accumulated by the photodiode are two-dimensionally arranged, and photoelectrically converts light from the optical system 243. A light receiving unit 244f that generates an electrical signal (imaging signal), and a reading unit 244g that sequentially reads out, as image information, an electrical signal generated by a pixel arbitrarily set as a reading target among a plurality of pixels of the light receiving unit 244f. Have. Further, the reading unit 244g sequentially reads out electrical signals (imaging signals) for each horizontal line from the light receiving unit 244f and outputs them to the AFE unit 244b.

  The AFE unit 244b includes a noise reduction unit 244h that reduces a noise component included in the electrical signal (analog), and an AGC (Automatic Gain Control) unit that adjusts the amplification factor (gain) of the electrical signal and maintains a constant output level. 244i and an A / D conversion unit 244j that performs A / D conversion on an electrical signal as image information (image signal) output via the AGC unit 244i. The noise reduction unit 244h performs noise reduction using, for example, a correlated double sampling method.

  The imaging control unit 244e controls various operations of the distal end portion 24 according to the setting data received from the processing device 4. The imaging control unit 244e is configured using a CPU (Central Processing Unit), a register that records various programs, and the like.

  The operation section 22 includes a bending knob 221 that bends the bending section 25 in the vertical direction and the left-right direction, a treatment instrument insertion section 222 that inserts a treatment instrument such as a biological forceps, an electric knife, and a test probe into the body cavity of the subject. In addition to the processing device 4 and the light source device 3, it has a plurality of switches 223 which are operation input units for inputting operation instruction signals of peripheral devices such as air supply means, water supply means, and screen display control. The treatment tool inserted from the treatment tool insertion portion 222 is exposed from the opening (not shown) via the treatment tool channel (not shown) of the distal end portion 24.

  The universal cord 23 includes at least a light guide 241 and a collective cable 245 in which one or a plurality of signal lines are collected.

  Next, the configuration of the light source device 3 will be described. The light source device 3 includes an illumination unit 31 and an illumination control unit 32.

  The illumination unit 31 emits illumination light that illuminates the subject. The illumination unit 31 includes a light source 33 and a light source driver 34.

  The light source 33 is configured using a white LED, and generates white light under the control of the illumination control unit 32. Light generated by the light source 33 is emitted from the distal end portion 24 toward the subject via a condenser lens (not shown) and a light guide 241.

  The light source driver 34 causes the light source 33 to generate white light by supplying current to the light source 33 under the control of the illumination control unit 32.

  The illumination control unit 32 controls the emission and extinguishing of illumination light by the illumination unit 31. Further, the illumination control unit 32 variably controls the intensity of the illumination light emitted from the illumination unit 31, such as keeping the intensity of the illumination light emitted from the illumination unit 31 constant. In this specification, one frame period refers to a period from the start of reading the first line to the start of reading the next first line.

  Next, the configuration of the processing device 4 will be described. The processing device 4 includes an S / P conversion unit 401, an image processing unit 402, a brightness detection unit 403, a dimming unit 404, a read address setting unit 405, a drive signal generation unit 406, and an input unit 407. A recording unit 408, a control unit 409, and a reference clock generation unit 410.

  The S / P conversion unit 401 performs serial / parallel conversion on the image information (imaging signal) input from the image sensor 244 and outputs the image information to the image processing unit 402. Note that the image information includes an imaging signal, a correction parameter for correcting the imaging element 244, and the like.

  The image processing unit 402 generates in-vivo image information (image signal) displayed by the display device 5 based on the image information input from the S / P conversion unit 401. The image processing unit 402 performs predetermined image processing on the image information to generate in-vivo image information. Here, as image processing, synchronization processing, optical black reduction processing, white balance adjustment processing, color matrix calculation processing, gamma correction processing, color reproduction processing, edge enhancement processing, composition processing and format for combining a plurality of image data Conversion processing and the like. Further, the image processing unit 402 generates one in-vivo image by adding image information corresponding to a plurality of frames read by the reading unit 244g from the light receiving unit 244f. Further, the image processing unit 402 outputs the image information input from the S / P conversion unit 401 to the control unit 409 or the brightness detection unit 403.

  The brightness detection unit 403 detects the brightness level corresponding to each pixel from the image information input from the image processing unit 402, records the detected brightness level in a memory provided therein, and the control unit 409. Output to. The brightness detection unit 403 calculates a gain adjustment value based on the detected brightness level, and outputs the gain adjustment value to the image processing unit 402.

  Under the control of the control unit 409, the light control unit 404 sets the amount of light generated by the light source device 3, the light emission timing, and the like based on the light irradiation amount calculated by the brightness detection unit 403. The dimming signal including it is output to the light source device 3.

  The read address setting unit 405 has a function of setting a pixel to be read and a reading order on the light receiving surface of the sensor unit 244a. That is, the read address setting unit 405 has a function of setting an address of a pixel that the AFE unit 244b reads from the sensor unit 244a. Further, the read address setting unit 405 outputs the set address information of the pixel to be read to the image processing unit 402.

  The drive signal generation unit 406 generates a driving timing signal for driving the image sensor 244 and transmits the timing signal to the timing generator 244d via a predetermined signal line included in the collective cable 245. This timing signal includes address information of a pixel to be read.

  The input unit 407 receives input of various signals such as an operation instruction signal that instructs the operation of the endoscope system 1.

  The recording unit 408 is realized using a semiconductor memory such as a flash memory or a DRAM (Dynamic Random Access Memory). The recording unit 408 records data including various programs for operating the endoscope system 1 and various parameters necessary for the operation of the endoscope system 1. The recording unit 408 records the identification information of the processing device 4. Here, the identification information includes unique information (ID) of the processing device 4, year type, specification information, transmission method, transmission rate, and the like.

  The control unit 409 is configured using a CPU or the like, and performs drive control of each component including the image sensor 244 and the light source device 3, and input / output control of information with respect to each component. The control unit 409 transmits setting data for imaging control to the imaging control unit 244e via a predetermined signal line included in the collective cable 245. The control unit 409 outputs a synchronization signal including the exposure timing and readout timing of each line of the image sensor 244 to the light source device 3.

  The reference clock generation unit 410 generates a reference clock signal that serves as a reference for the operation of each component of the endoscope system 1 and supplies the generated reference clock signal to each component of the endoscope system 1.

  Next, the display device 5 will be described. The display device 5 receives and displays the in-vivo image corresponding to the in-vivo image information generated by the processing device 4 via the video cable. The display device 5 is configured using a liquid crystal display or an organic EL (Electro Luminescence) display.

  The emission timing of the illumination light emitted from the light source device 3 at the time of photographing with the endoscope system 1 having the above configuration, the exposure and readout timing of the image sensor 244, and the processing by the image processing unit 402 will be described.

  FIG. 3 is a diagram schematically showing the emission timing of illumination light emitted from the light source device 3 at the time of photographing by the endoscope system 1, the exposure and readout timing of the image sensor 244, and the processing by the image processing unit 402. FIG. 3A shows time. FIG. 3B shows the emission timing of the illumination light emitted from the light source device 3. FIG. 3C shows the exposure and readout timing of the image sensor 244. FIG. 3D shows the contents of image generation processing by the image processing unit 402. In the first embodiment, it is assumed that in the horizontal line of one frame, the simultaneous exposure period Tm1 is short or almost zero between the read start line and the read end line.

  As shown in FIG. 3, the illumination control unit 32 controls the illumination unit 31 to emit or illuminate illumination light to the illumination unit 31 in synchronization with the frame rate of the image sensor 244.

  Specifically, the illumination control unit 32 has a period (one frame period) until the reading unit 244g reads the first horizontal line of the next frame F1-2 from the first horizontal line of the frame F1-1 of the light receiving unit 244f. The illumination light is turned off by the illumination unit 31 (turn-off period Ts1, first period).

  Subsequently, the illumination controller 32 illuminates a period (one frame period) from when the reading unit 244g reads the first horizontal line of the next frame F2-1 from the first horizontal line of the frame F1-2 of the light receiving unit 244f. The illumination light is emitted from the unit 31 (illumination period Te1, second period). At this time, the reading unit 244g captures the imaging data D1-1 and D1-2 (first and second imaging) of the frames F1-1 and F1-2 including the pixels exposed by the illumination light emitted in the illumination period Te1. Signal) from the light receiving unit 244f and output to the image processing unit 402.

  Further, the image processing unit 402 generates image data Dd1 (image signal) obtained by adding the imaging data D1-1 and the imaging data D1-2. Specifically, the image processing unit 402 generates the image data Dd1 by adding the luminance values of the imaging data D1-1 and the imaging data D1-2 for each pixel. Thereby, an image signal corresponding to an image exposed with the illumination light in the illumination period Te1 is generated.

  Thereafter, the illumination control unit 32 performs illumination by the illumination unit 31 for one frame period until the reading unit 244g reads the first horizontal line of the next frame F2-2 from the first horizontal line of the frame F2-1 of the light receiving unit 244f. The light is turned off (turn-off period Ts2). Subsequently, the illumination control unit 32 sets the illumination unit 31 for one frame period until the reading unit 244g reads the first horizontal line of the next frame F3-1 from the first horizontal line of the frame F2-2 of the light receiving unit 244f. Illumination light is emitted (illumination period Te2). At this time, the reading unit 244g reads the imaging data D2-1 and D2-2 of the frames F2-1 and F2-2 including the pixels exposed by the illumination light emitted in the illumination period Te2 from the light receiving unit 244f, respectively. The image is output to the image processing unit 402. Further, as described above, the image processing unit 402 generates image data Dd2 obtained by adding the imaging data D2-1 and the imaging data D2-2.

  In this way, the illumination control unit 32 also illuminates every frame period in the frames F3-1, F3-2, F4-1, F4-2, the frame F5-1, etc. following the frame F2-2. Repeat turning off and emitting light. The reading unit 244g reads the imaging data D3-1, D3-2, imaging data D4-1, D4-2, imaging data D5-1,... Corresponding to each frame from the light receiving unit 244f, respectively, and the image processing unit 402. Output to. Further, as described above, the image processing unit 402 generates image data Dd3 and Dd4 obtained by adding the imaging data D3-1 and imaging data D3-2, the imaging data D4-1, and the imaging data D4-2, respectively.

FIG. 4 is a diagram illustrating the emission timing of the illumination light emitted from the light source device 3 and the exposure and readout timing of the image sensor at the time of photographing with the endoscope system 1. When the reading unit 244g reads the n-th line, the reading period at the time of illumination light irradiation in the frame F1-1 is T1 _n , the reading period at the time of illumination light irradiation in the frame F1-2 is T2 _n , and according to one frame period. Assuming that the readout period is Tf and the reset period between the frames F1-1 and F1-2 is Tr, the illumination period Te1 for emitting illumination light satisfies the relationship of the following expression (1).
_{T1 n + T2 n = Tf-} Tr ··· (1)

Further, the following equation (2) is satisfied in the (n + 1) th line as the next line.
T1 _{n + 1} + T2 _{n + 1} = Tf−Tr (2)

Here, since the readout period Tf and the reset period Tr are constants, the relationship of the following expression (3) is established from the expressions (1) and (2).
T1 _n + T2 _n = T1 _{n + 1} + T2 _{n + 1} (3)
That is, the exposure period in the nth line is equal to the exposure period in the (n + 1) th line. When the illumination period Te1 is equal to the readout period Tf, the relationship of the following equation (1) ′ is established from the equation (1).
T1 _n + T2 _n = Te1-Tr (1) ′

At this time, if Tf (Te1) >> Tr, for example, T1 _n + T2 _n in equation (1) becomes T1 _n + T2 _n ≈Tf, and the imaging data D1-1 and D1 of the frames F1-1 and F1-2 -2 (image data Dd1) can be said to be image data that is read and generated continuously in the illumination period Te1. As a result, the image data Dd1 obtained by adding the imaging data D1-1 and D1-2 is kept simultaneity and becomes an image without distortion for each line.

  According to the first embodiment of the present invention described above, since image signals corresponding to a plurality of frames including the same illumination period are added to generate one image data (image signal), imaging is performed. Even if a series of processing from exposure by the element to reading of charges is performed at different timings for each line, image blurring and distortion can be suppressed.

  Further, according to Embodiment 1 of the present invention, the illumination control unit 32 controls the illumination unit 31 with the illumination period before and after the illumination period, so that image signals exposed in different illumination periods are generated as images. The color mixture in the image can be prevented.

(Modification 1-1)
FIG. 5 shows the emission timing of the illumination light emitted from the light source device, the exposure and readout timing of the image sensor, and the processing by the image processing unit when the endoscope system according to Modification 1-1 of the first embodiment performs imaging. It is a figure shown typically. (A)-(d) in FIG. 5 is the same as that of (a)-(d) of FIG. In the first embodiment described above, it has been described that the simultaneous exposure period Tm1 is short between the readout start line and the readout end line in the horizontal line of one frame. However, as in Modification 1-1, 1 is used. The present invention is also applicable to a simultaneous exposure period Tm2 that is ½ or more of the line readout period.

(Modification 1-2)
FIG. 6 illustrates the emission timing of the illumination light emitted from the light source device, the exposure and readout timing of the image sensor, and the processing by the image processing unit when the endoscope system according to Modification 1-2 of the first embodiment performs imaging. It is a figure shown typically. (A) to (d) in FIG. 6 are the same as (a) to (d) in FIG. In Embodiment 1 described above, the illumination light emission timing of the illumination unit 31 by the illumination control unit 32 has been described as being performed in synchronization with the readout start timing of the readout unit 244g. However, in Modification 1-2, readout is performed. Illumination light is emitted at a timing different from the readout start timing of the unit 244g.

  As illustrated in FIG. 6, when the illumination control unit 32 emits illumination light at a timing different from the readout start timing of the readout unit 244g, the illumination period Te11 of illumination light corresponding to one frame period includes three frames (frames). F1-1, F1-2, F1-3). At this time, the reading unit 244g captures the imaging data D1-1, D1-2, and D1-3 of the frames F1-1, F1-2, and F1-3 including the pixels exposed by the illumination light emitted in the illumination period Te11. Are respectively read from the light receiving unit 244f and output to the image processing unit 402. Further, as described above, the image processing unit 402 generates image data Dd11 obtained by adding the imaging data D1-1, the imaging data D1-2, and the imaging data D1-3.

  At this time, the illumination control unit 32 provides extinguishing periods Ts11a and Ts11b corresponding to one frame period before and after the illumination period Te11. That is, the illumination control unit 32 controls the illumination unit 31 in the order of extinction, illumination, and extinction when generating one added image data Dd11.

  As described above, the illumination control unit 32 also emits illumination light from the illumination unit 31 in the frames F2-1, F2-2, and F2-3 following the frame F1-3 for a period of one frame period or more (illumination period Te12). Control is performed to turn off the illumination unit 31 during the period before and after the illumination period Te12 and corresponding to one frame period (the extinction periods Ts12a and Ts12b). Further, the illumination control unit 32 also emits illumination light from the illumination unit 31 in the frames F3-1, F3-2, and F3-3 following the frame F2-3 for a period of one frame period or more (illumination period Te13). The illumination unit 31 is controlled to be extinguished during the period before and after the illumination period Te13 and corresponding to one frame period (extinguishing periods Ts13a and Ts13b). The illumination control unit 32 repeats this illumination process.

  At this time, the reading unit 244g receives the imaging data D2-1, D2-2, D2-3, the imaging data D3-1, D3-2, D3-3,... According to each frame from the light receiving unit 244f. Read out and output to the image processing unit 402. Further, as described above, the image processing unit 402 receives the imaging data D2-1, imaging data D2-2, imaging data D2-3, imaging data D3-1, imaging data D3-2, and imaging data D3-3, respectively. The added image data Dd12 and Dd13 are generated.

(Modification 1-3)
FIG. 7 shows the emission timing of the illumination light emitted from the light source device, the exposure and readout timing of the image sensor, and the processing by the image processing unit at the time of photographing by the endoscope system according to the modification 1-3 of the first embodiment. It is a figure shown typically. (A)-(d) in FIG. 7 is the same as that of (a)-(d) of FIG. In Embodiment 1 described above, the illumination period Te1 of the illumination light of the illumination unit 31 by the illumination control unit 32 has been described as a period corresponding to one frame period. However, in Modification 1-3, one frame period The illumination light is emitted as described above.

  As shown in FIG. 7, when the illumination control unit 32 emits illumination light from the illumination unit 31 for a period of one frame period or more (illumination period Te21) in synchronization with the readout start timing of the readout unit 244g, this illumination period Te21 includes three frames (frames F1-1, F1-2, and F1-3). At this time, the reading unit 244g captures the imaging data D1-1, D1-2, and D1-3 of the frames F1-1, F1-2, and F1-3 including the pixels exposed by the illumination light emitted in the illumination period Te21. Are respectively read from the light receiving unit 244f and output to the image processing unit 402. Furthermore, as described above, the image processing unit 402 generates image data Dd21 obtained by adding the imaging data D1-1, the imaging data D1-2, and the imaging data D1-3.

  At this time, the illumination control unit 32 provides a turn-off period Ts21 corresponding to one frame period before the illumination period Te21. That is, the illumination control unit 32 controls the illumination unit 31 in the order of turning off and lighting when generating one image data Dd21.

  In this way, the illumination control unit 32 also performs the one frame period in the frames F2-1, F2-2, F2-3, the frames F3-1, F3-2, F3-3,. The illumination unit 31 emits illumination light for a period of one frame period or more (illumination periods Te22, Te23...) Repeat the control to be performed. The readout unit 244g reads out the imaging data D2-1, D2-2, D2-3, the imaging data D3-1, D3-2, D3-3,... Corresponding to each frame from the light receiving unit 244f, respectively, and performs image processing. Output to the unit 402. Further, as described above, the image processing unit 402 adds the imaging data D2-1, the imaging data D2-2, the imaging data D2-3, the imaging data D3-1, the imaging data D3-2, and the imaging data D3-3. The generated image data Dd22 and Dd23 are generated.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. FIG. 8 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the second embodiment. In addition, the same code | symbol is attached | subjected and demonstrated to the structure same as Embodiment 1 mentioned above. In the first embodiment described above, the single illumination light is irradiated by the single light source 33. However, in the second embodiment, illumination is performed by a plurality of (two in the second embodiment) light sources 33a and 33b. Light is emitted.

  First, the configuration of the light source device 3a of the endoscope system 1a according to the second embodiment will be described. The light source device 3a includes an illumination unit 31a and an illumination control unit 32a.

  The illumination unit 31a emits illumination light that illuminates the subject. The illumination unit 31a includes light sources 33a and 33b, and a first light source driver 34a and a second light source driver 34b provided according to the light sources 33a and 33b.

  The light source 33a is configured using a white LED, and generates white light under the control of the illumination control unit 32a. The light generated by the light source 33a is emitted from the distal end portion 24 toward the subject via a condenser lens (not shown) and a light guide 241.

  The first light source driver 34a supplies white light to the light source 33a by supplying current to the light source 33a under the control of the illumination control unit 32a.

  The light source 33b is configured by using a white LED with a filter attached to the exit surface side, and generates blue light extracted by the filter under the control of the illumination control unit 32a. The light generated by the light source 33b is emitted toward the subject from the distal end portion 24 via the condenser lens and the light guide 241. The blue light from the light source 33b is used as illumination light for NBI (Narrow Band Imaging). The light source 33b may be configured by a blue LED without providing a filter, for example.

  The second light source driver 34b supplies blue light to the light source 33b by supplying current to the light source 33b under the control of the illumination control unit 32a.

  The illumination control unit 32a controls the emission and extinction of illumination light by the illumination unit 31a. Further, the illumination control unit 32a variably controls the intensity of the illumination light emitted from the illumination unit 31a, such as selecting the light sources 33a and 33b of the illumination unit 31a, and keeping the intensity of the illumination light emitted from the illumination unit 31a constant. To do.

  FIG. 9 is a diagram schematically illustrating the emission timing of illumination light emitted from the light source device at the time of photographing by the endoscope system according to the second embodiment, the exposure and readout timing of the image sensor, and the processing by the image processing unit. is there. (A) to (d) in FIG. 9 are the same as (a) to (d) in FIG.

  As shown in FIG. 9, the illumination control unit 32a controls the illumination unit 31a to emit or extinguish illumination light to the illumination unit 31a in synchronization with the frame rate of the image sensor 244.

  Specifically, the illumination control unit 32a first turns off the illumination light by the illumination unit 31a for a period corresponding to one frame period (extinction period Ts31).

  Subsequently, the illumination control unit 32a causes the light source 33a of the illumination unit 31a to emit illumination light for a period corresponding to one frame period (first illumination, illumination period Te31). At this time, the readout unit 244g reads out the imaging data D1-1 and D1-2 of the frames F1-1 and F1-2 including the pixels exposed by the illumination light emitted in the illumination period Te31 from the light receiving unit 244f, respectively. The image is output to the image processing unit 402. Further, as described above, the image processing unit 402 generates image data Dd31 obtained by adding the imaging data D1-1 and the imaging data D1-2.

  Thereafter, the illumination control unit 32a turns off the illumination unit 31a for a period corresponding to one frame period (extinguishing period Ts32). Subsequently, in the illumination control unit 32a, the reading unit 244g causes the light source 33b of the illumination unit 31a to emit illumination light for one frame period of the light receiving unit 244f (second illumination, illumination period Te32). At this time, the reading unit 244g reads the imaging data D2-1 and D2-2 of the frames F2-1 and F2-2 including the pixels exposed by the illumination light emitted in the period Te2 from the light receiving unit 244f, respectively. The data is output to the processing unit 402. Further, as described above, the image processing unit 402 generates image data Dd41 obtained by adding the imaging data D2-1 and the imaging data D2-2.

  In this way, the illumination control unit 32a also responds to one frame period in frames F3-1, F3-2, frames F4-1, F4-2, frames F5-1,. , The illumination unit 31a is turned off, the period corresponding to one frame period (illumination periods Te33, Te34,...), And the light sources 33a and 33b. The control for illuminating the illumination light is repeated. At this time, the reading unit 244g reads the imaging data D3-1, D3-2, imaging data D4-1, D4-2, imaging data D5-1,... Corresponding to each frame from the light receiving unit 244f, respectively. The data is output to the processing unit 402. Further, as described above, the image processing unit 402 generates image data Dd32 and Dd42 obtained by adding the imaging data D3-1 and imaging data D3-2, the imaging data D4-1, and the imaging data D4-2.

  In the second embodiment, the illumination light is alternately illuminated by the light sources 33a and 33b in FIG. 9, but the illumination light is not limited to the alternate illumination process and may be switched every predetermined number of times.

  According to the second embodiment of the present invention described above, even when different light sources are used, image signals corresponding to a plurality of frames including the same illumination period are added to obtain one image data (image signal). ), The image blur and distortion caused by each illumination light can be suppressed even when a series of processing from exposure to image pickup to readout of charge is performed at different timings for each line. it can.

  In addition, according to the second embodiment of the present invention, the illumination control unit 32a controls the illumination unit 31a with the illumination period before and after the illumination period as the extinguishing period. The color mixture in the image can be prevented.

  In the second embodiment of the present invention, the illumination light for NBI is described as being used. However, the illumination light for AFI (Auto Fluorescence Imaging) and the illumination light for IRI (Infrared Imaging) are used. Even when illumination light or the like is used, the present invention can be applied.

(Modification 2-1)
FIG. 10 shows the emission timing of the illumination light emitted from the light source device at the time of photographing by the endoscope system according to the modified example 2-1 of the second embodiment, the exposure and readout timing of the image sensor, and the processing by the image processing unit. It is a figure shown typically. In the second embodiment described above, the illumination light irradiation periods Te32 and Te34 of the light source 33b of the illumination unit 31a by the illumination control unit 32a have been described as periods corresponding to one frame period, but Modification 2-1 Then, the illumination light from the light source 33b is irradiated for a period of one frame period or more.

  As shown in FIG. 10, after the illumination control unit 32a turns off the illumination unit 31a for a period corresponding to one frame period (light-out period Ts41) in synchronization with the readout start timing of the readout unit 244g, The illumination light emitted from the light source 33b is emitted (second illumination) during the period (illumination period Te41) as described above (two frame periods in the present modification 2-1). In this case, the illumination period Te41 includes three frames (frames F1-1, F1-2, and F1-3). At this time, the reading unit 244g receives the imaging data D1-1, D1-2, and D1-3 of the frames F1-1, F1-2, and F1-3 including the pixels exposed by the illumination light emitted in the period Te41. Each is read from the light receiving unit 244 f and output to the image processing unit 402. Furthermore, as described above, the image processing unit 402 generates the image data Dd51 obtained by adding the imaging data D1-1, the imaging data D1-2, and the image data D1-3.

  Subsequently, the illumination control unit 32a turns off the illumination unit 31a for a period corresponding to one frame period (extinguishing period Ts51), and then the period corresponding to one frame period (illumination period Te51), and the light source of the illumination unit 31a. The illumination light is emitted to 33a (first illumination). At this time, the readout unit 244g reads out the imaging data D2-1 and D2-2 of the frames F2-1 and F2-2 including the pixels exposed by the illumination light emitted in the illumination period Te51 from the light receiving unit 244f, respectively. The image is output to the image processing unit 402. Further, as described above, the image processing unit 402 generates image data Dd61 obtained by adding the imaging data D2-1 and the imaging data D2-2.

  As described above, the illumination control unit 32a also turns off the illumination unit 31a in the frames F3-1, F3-2, and F3-3 following the frame F2-2, in a period corresponding to one frame period (extinction period Ts42). Then, control is performed to emit illumination light from the light source 33b for a period of one frame period or more (illumination period Te42). The reading unit 244g reads the imaging data D3-1, D3-2, and D3-3 corresponding to each frame from the light receiving unit 244f and outputs them to the image processing unit 402. Further, as described above, the image processing unit 402 generates image data Dd52 obtained by adding the imaging data D3-1, the imaging data D3-2, and the imaging data D3-3.

  Thereafter, as described above, the illumination control unit 32a repeats the turn-off, the second illumination, the turn-off, and the first illumination, and generates the added image data obtained by the illumination process by the light sources 33a and 33b. Thereby, even when the illumination light itself is dark or when imaging a far point where the illumination light is difficult to reach, it is possible to secure the exposure time and to suppress image blurring, distortion, and color mixing.

  In the above-described modification 2-1, the illumination period for the second illumination has been described as corresponding to a two-frame period. However, an illumination period corresponding to a period of three or more frame periods is set according to the intensity of illumination light or the like. It is also possible to do.

(Embodiment 3)
Next, a third embodiment of the present invention will be described. FIG. 11 is a block diagram illustrating a functional configuration of a main part of the endoscope system according to the third embodiment. FIG. 12 is a diagram schematically illustrating the emission timing of illumination light emitted from the light source device at the time of photographing with the endoscope system according to the third embodiment, the exposure and readout timing of the image sensor, and the processing by the image processing unit. is there. In addition, the same code | symbol is attached | subjected and demonstrated to the structure same as Embodiment 1 mentioned above. In the first embodiment described above, the single illumination light is irradiated by the single light source 33. However, in the third embodiment, illumination light having different wavelengths is selectively emitted to perform illumination.

  First, the configuration of the light source device 3b of the endoscope system 1b according to the third embodiment will be described. The light source device 3b includes an illumination unit 31b and an illumination control unit 32b.

  The illumination unit 31b sequentially switches and emits a plurality of illumination lights having different wavelength bands to the subject under the control of the illumination control unit 32b. The illumination unit 31 b includes a light source 33, a light source driver 34, a rotary filter 35, a drive unit 36, and a drive driver 37.

  The light source unit 33 is configured using a white LED and one or a plurality of lenses, and emits white light to the rotary filter 35 under the control of the light source driver 34. The white light generated by the light source unit 33 is emitted from the tip of the tip part 24 toward the subject via the rotary filter 35 and the light guide 241.

  The light source driver 34 causes the light source unit 33 to emit white light by supplying a current to the light source unit 33 under the control of the illumination control unit 32b.

  The rotary filter 35 is disposed on the optical path of the white light emitted from the light source unit 33 and rotates to transmit only light in a predetermined wavelength band among the white light emitted from the light source unit 33. Specifically, the rotary filter 35 includes a red filter 311, a green filter 312, and a blue filter 313 that transmit light having respective wavelength bands of red light (R), green light (G), and blue light (B). . The rotary filter 35 sequentially transmits light in red, green, and blue wavelength bands (for example, red: 600 nm to 700 nm, green: 500 nm to 600 nm, blue: 400 nm to 500 nm) by rotating. As a result, the white light (W illumination) emitted from the light source unit 31a is converted into any one of red light (R illumination), green light (G illumination), and blue light (B illumination) with a narrow band. Can be sequentially emitted (surface sequential method).

  The drive unit 36 is configured using a stepping motor, a DC motor, or the like, and rotates the rotary filter 35.

  The drive driver 37 supplies a predetermined current to the drive unit 36 under the control of the illumination control unit 32b.

  The illumination control unit 32b is synchronized with the readout start timing of the readout unit 244g and is extinguished during the extinguishing period Ts61 corresponding to one frame period, and then in a period (one frame period) until the start of readout of the first line of the light receiving unit 244f. During the corresponding period (first illumination period Te61), illumination light (for example, light transmitted through the red filter 311) is emitted to the illumination unit 31b. After the end of the first illumination period Te61, a period corresponding to one frame period (light-out period Ts62), a period from the time when the illumination unit 33b is turned off to the start of reading the first line of the next light receiving unit 244f (one frame) Period (second illumination period Te62), illumination light having a wavelength band different from that of the first period (for example, light transmitted through the green filter 312) is emitted to the illumination unit 31b. After the end of the second illumination period Te62, a period corresponding to one frame period (light extinction period Ts63), a period from when the illumination unit 33b is extinguished to the start of reading the first line of the next light receiving unit 244f (one frame) Period) (a third illumination period Te63), a series of illumination light having a wavelength band different from the illumination light in the first and second illumination periods (for example, light transmitted through the blue filter 313) is emitted to the illumination unit 31b. The process is repeatedly executed by the illumination unit 31b.

  Specifically, the illumination control unit 32b emits R illumination to the illumination unit 31b in the first illumination period Te61, emits G illumination to the illumination unit 31b in the second illumination period Te62, and B in the third illumination period Te63. Illumination is emitted to the illumination unit 31b. Thereafter, the illumination control unit 32b repeats a series of illumination processes by the illumination periods Te64 and Te65 and the extinguishing periods Ts64 and Ts65 according to the first to third periods described above.

  The readout unit 244g reads out the first R imaging data Dr1-1 of the frame F1-1 including the pixels exposed by the illumination light (R illumination) emitted in the first illumination period Te61 from the light receiving unit 244f, and reads out the image processing unit 402. Output to. The second R imaging data Dr1-2 of the frame F1-2 including the pixels exposed by the illumination light (R illumination) emitted in the first illumination period Te61 is read from the light receiving unit 244f and output to the image processing unit 402. Furthermore, as described above, the image processing unit 402 generates R-added imaging data Dr1 obtained by adding the first R imaging data Dr1-1 and the second R imaging data Dr1-2.

  The readout unit 244g reads out the first G imaging data Dg1-1 of the frame F2-1 including the pixels exposed by the illumination light (G illumination) emitted in the second illumination period Te62 from the light receiving unit 244f, and performs image processing. Output to the unit 402. Subsequently, the readout unit 244g reads out the second G captured image data Dg1-2 of the frame F2-2 including the pixels exposed by the illumination light (G illumination) emitted in the second illumination period Te62 from the light receiving unit 244f. The image is output to the image processing unit 402. Furthermore, as described above, the image processing unit 402 generates G-added imaging data Dg1 obtained by adding the first G imaging data Dg1-1 and the second G imaging data Dg1-2.

  Further, the readout unit 244g reads out the first B imaging data Db1-1 of the frame F3-1 including the pixels exposed by the illumination light (B illumination) emitted in the third illumination period Te63 from the light receiving unit 244f, and performs image processing. Output to the unit 402. Subsequently, the readout unit 244g reads out the second B image data Db1-2 of the frame F3-2 including the pixels exposed by the illumination light (B illumination) emitted in the third illumination period Te63 from the light receiving unit 244f, and outputs an image. The data is output to the processing unit 402. Furthermore, as described above, the image processing unit 402 generates B addition imaging data Db1 obtained by adding the first B imaging data Db1-1 and the second B imaging data Db1-2.

  As described above, the reading unit 244g adds the image data of a plurality of frames including the illumination period of each illumination to the R illumination, the G illumination, and the B illumination that are sequentially illuminated, thereby adding according to each illumination. Image data can be obtained. In addition, the reading unit 244g subsequently performs the first R imaging data Dr2-1 and the first R imaging data Dr2-1 of the frame F4-1 and the frame F4-2 including the pixels exposed by the illumination light (R illumination) emitted in the first illumination period Te64. The 2R imaging data Dr2-2 is read from the light receiving unit 244f and output to the image processing unit 402. The image processing unit 402 adds the first R imaging data Dr2-1 and the second R imaging data Dr2-2, and R addition imaging data Dr2 Is generated. Further, similarly in the second illumination period Te65, the first G imaging data Dg2-1 and the second G imaging data Dg2-2 of the frame F5-1 and the frame F5-2 including the pixels exposed by the illumination light (G illumination). Is read from the light receiving unit 244f and output to the image processing unit 402, and the image processing unit 402 generates G added imaging data Dg2 obtained by adding the first G imaging data Dg2-1 and the second G imaging data Dg2-2.

  Here, the image processing unit 402 adds the R addition imaging data, G addition imaging data, and B addition imaging data obtained by each illumination to generate white image data. Specifically, the image processing unit 402 adds the R addition imaging data Dr1, the G addition imaging data Dg1, and the B addition imaging data Db1 to generate white image data W1. Furthermore, when the R addition imaging data Dr2 is obtained by the next illumination process (R illumination), the image processing unit 402 adds the G addition imaging data Dg1, the B addition imaging data Db1, and the R addition imaging data Dr2 to obtain a white color. Image data W2 is generated. Subsequently, when the G addition image data Dg2 is obtained by the next illumination process (G illumination), the image processing unit 402 adds the B addition imaging data Db1, the R addition imaging data Dr2, and the G addition imaging data Dg2, White image data W3 is generated. As described above, the image processing unit 402 generates white image data by using the R addition imaging data, the G addition imaging data, and the B addition imaging data that are sequentially generated.

  According to Embodiment 3 of the present invention described above, even when illumination light with different wavelengths is irradiated, imaging signals corresponding to a plurality of frames including the same illumination period are added, and one Since image data (image signal) is generated, even if a series of processing from exposure to reading of the image sensor to reading of charges is performed at different timings for each line, image blurring and distortion are suppressed. Can do.

  In addition, according to the third embodiment of the present invention, the illumination control unit 32b controls the illumination unit 31b with the illumination period before and after the illumination period as the extinguishing period, so that image signals exposed in different illumination periods are generated as images. The color mixture in the image can be prevented.

(Embodiment 4)
Next, a fourth embodiment of the present invention will be described. FIG. 13 is a diagram schematically illustrating the emission timing of illumination light emitted from the light source device at the time of photographing with the endoscope system according to the fourth embodiment, the exposure and readout timing of the image sensor, and the processing by the image processing unit. is there. In addition, the same code | symbol is attached | subjected and demonstrated to the same structure as Embodiment 1 mentioned above. Moreover, the endoscope system according to the fourth embodiment has the same configuration as the endoscope system according to the first embodiment described above. For this reason, in the following, the emission timing of illumination light emitted by the light source device at the time of photographing with the endoscope system, the exposure and readout timing of the image sensor, and the processing by the image processing unit will be described. In the fourth embodiment, imaging in the case where a moving image and a still image are selectively displayed will be described.

  When the control unit 409 receives from the input unit 407 a signal indicating that an instruction signal related to moving image shooting has been received (when the input unit 407 does not receive an instruction signal indicating that still image shooting is to be performed) The lighting control unit 32 is instructed to emit the illumination light continuously. The illumination control unit 32 causes the light source 33 to emit illumination light under the control of the control unit 409. At this time, the reading unit 244g reads the imaging data D1-1, D1-2, and D1-3 of the frames F1-1, F1-2, and F1-3 including the pixels exposed by the illumination light from the light receiving unit 244f, respectively. To the image processing unit 402.

  The image processing unit 402 performs synchronization processing, optical black reduction processing, white balance adjustment processing, color matrix calculation processing, gamma correction processing, color reproduction processing, edge processing on the imaging data D1-1, D1-2, and D1-3. Image data Dm1 to Dm3 to be displayed by the display device 5 are generated by performing an emphasis process, a composition process for combining a plurality of image data, a format conversion process, and the like.

  Here, when there is an input indicating that the input of an instruction signal related to still image shooting has been received from the input unit 407, the control unit 409 performs illumination processing for still image shooting on the illumination control unit 32. Instruct. The illumination control unit 32 first turns off the illumination unit 31 for a period corresponding to one frame (extinguishment period Ts71).

  Subsequently, the illumination control unit 32 causes the illumination unit 31 to emit illumination light for a period corresponding to one frame period (illumination period Te71). At this time, the reading unit 244g reads the imaging data D2-1 and D2-2 of the frames F2-1 and F2-2 including the pixels exposed by the illumination light emitted in the illumination period Te71 from the light receiving unit 244f, respectively. The image is output to the image processing unit 402. Further, the image processing unit 402 generates image data Dd71 obtained by adding the imaging data D2-1 and the imaging data D2-2.

  Thereafter, the control unit 409 returns to the control of the illumination processing related to moving image shooting. The control unit 409 instructs the illumination control unit 32 to emit illumination light continuously. Under the control of the control unit 409, the illumination control unit 32 turns off the illumination unit 31 for a period corresponding to one frame (extinguishment period Ts72), and then causes the illumination unit 31 to emit illumination light. At this time, the reading unit 244g reads the imaging data D3-1, D3-2, D3-3, and D3-4 of the frames F3-1, F3-2, F3-3, and F3-4 from the light receiving unit 244f, respectively. The image is output to the image processing unit 402.

  The image processing unit 402 performs synchronization processing, optical black reduction processing, white balance adjustment processing, color matrix calculation processing, gamma correction processing, color processing on the imaging data D3-1, D3-2, D3-3, and D3-4. Image data Dm4 to Dm7 to be displayed by the display device 5 are generated by performing reproduction processing, edge enhancement processing, composition processing for combining a plurality of image data, format conversion processing, and the like.

  By the above-described processing, the display image information obtained during the moving image shooting period includes the frames F1-1, F1-2, F1-3, F3-1, F3-2, F3-3, and F3-4. Corresponding imaging data D1-1 to D1-3, D3-1 to D3-4, and image data Dm1 to Dm7 on which image processing has been performed are used, and a display image obtained during a still image capturing period For the information, image data Dd71 generated based on the imaging data D2-1 and D2-2 of the frames F2-1 and F2-2 including the pixels exposed by the illumination light emitted in the illumination period Te71 is used. It is done.

  As a result, in the case of moving image shooting in which image blurring, distortion, and color mixing are not a concern, an image using continuous frame image data is displayed on the display device 5, and the effects of image blurring, distortion, and color mixing are displayed. In the case of still image shooting that can be felt, an image using the added image data obtained by adding a plurality of frames of image data including the same illumination period is provided on the display device 5 before and after the extinction period. The

  According to Embodiment 4 of the present invention described above, when still image shooting is performed during moving image shooting, an extinguishing period is provided before and after, and image signals corresponding to a plurality of frames including the same illumination period are added. Since one image data (additional image data) is generated for a still image, even if a series of processes from exposure by the image sensor to reading of charges are performed at different timing for each line, the image Shake and distortion can be suppressed.

  Further, according to the fourth embodiment of the present invention, the illumination control unit 32 controls the illumination unit 31 with the illumination periods before and after the illumination period, so that image signals exposed in different illumination periods are generated as images. The color mixture in the image can be prevented.

(Modification 4-1)
FIG. 14 shows the emission timing of illumination light emitted from the light source device, the exposure and readout timing of the image sensor, and the processing by the image processing unit at the time of photographing by the endoscope system according to Modification 4-1 of the fourth embodiment. It is a figure shown typically. In the above-described fourth embodiment, the illumination processing in the configuration of the endoscope system according to the first embodiment has been described. However, the configuration of the endoscope system 1a according to the second embodiment (configuration having a plurality of light sources, FIG. 8).

  When the input unit 407 inputs an instruction for moving image shooting, the control unit 409 instructs the illumination control unit 32a to emit illumination light continuously. Specifically, the illumination control unit 32a emits illumination light from the light source 33a and the light source 33b alternately and continuously under the control of the control unit 409. At this time, the reading unit 244g receives the imaging data D1-1 and D1-2 of the frames F1-1 and F1-2 including the pixels exposed by the illumination light (first illumination) from the light source 33a from the light receiving unit 244f, respectively. Read out and output to the image processing unit 402. Further, the reading unit 244g reads the image data D2-1 of the frame F2-1 including the pixels exposed by the illumination light (second illumination) from the light source 33b from the light receiving unit 244f, and outputs the image data D2-1 to the image processing unit 402. .

  The image processing unit 402 performs synchronization processing, optical black reduction processing, white balance adjustment processing, color matrix calculation processing, gamma correction processing, color reproduction processing, edge processing on the imaging data D1-1, D1-2, and D2-1. Image data Dm11 to Dm13 to be displayed by the display device 5 are generated by performing an emphasis process, a composition process for combining a plurality of image data, a format conversion process, and the like.

  Here, when an instruction related to still image shooting is input from the input unit 407, the control unit 409 instructs the illumination control unit 32a to perform illumination processing for still image shooting. The illumination control unit 32a first turns off the illumination unit 31a for a period corresponding to one frame (extinguishment period Ts101).

  Subsequently, the illumination control unit 32a causes the light source 33a of the illumination unit 31a to emit illumination light (first illumination) for a period corresponding to one frame period (illumination period Te81). At this time, the reading unit 244g reads the imaging data D3-1 and D3-2 of the frames F3-1 and F3-2 including the pixels exposed by the illumination light emitted in the illumination period Te81 from the light receiving unit 244f, respectively. The image is output to the image processing unit 402. Further, the image processing unit 402 generates image data Dd81 obtained by adding the imaging data D3-1 and the imaging data D3-2.

  Subsequently, the illumination control unit 32a turns off the illumination unit 31a for a period corresponding to one frame (extinguishing period Ts102), and then the period corresponding to one frame period (illumination period Te91), and the light source 33b of the illumination unit 31a. To emit illumination light (second illumination). At this time, the reading unit 244g reads the imaging data D4-1 and D4-2 of the frames F4-1 and F4-2 including the pixels exposed by the illumination light emitted in the illumination period Te91 from the light receiving unit 244f, respectively. The image is output to the image processing unit 402. Further, the image processing unit 402 generates image data Dd91 obtained by adding the imaging data D4-1 and the imaging data D4-2.

  Thereafter, the control unit 409 returns to the control of the illumination processing related to moving image shooting. The control unit 409 instructs the illumination control unit 32a to continuously emit illumination light. Specifically, the illumination control unit 32a alternately turns off the illumination light from the light source 33a and the light source 33b after the illumination unit 31a is turned off in the turn-off period Ts103 corresponding to one frame under the control of the control unit 409. It emits continuously. At this time, the reading unit 244g reads the imaging data D0-1 and D2-2 of the frames F0-1 and F2-2 (second illumination) subsequent to the frame F4-2 from the light receiving unit 244f, and sends them to the image processing unit 402. Output.

  The image processing unit 402 performs synchronization processing, optical black reduction processing, white balance adjustment processing, color matrix calculation processing, gamma correction processing, color reproduction processing, edge enhancement processing, and a plurality of processing on the imaging data D0-1 and D2-2. The image data Dm14 and Dm15 to be displayed by the display device 5 are generated by performing a combining process and a format conversion process for combining the image data.

  As a result of the processing described above, the display image information obtained during the moving image shooting period includes the frames F0-1, F1-1, F1-2, F2-1, F2-2, F3-1, F3-2. Imaging data D0-1, D1-1, D1-2, D2-1, and D2-2 corresponding to F4-1 and F4-2, and image data Dm11 to Dm15 subjected to image processing are used. Display image information obtained in the still image shooting period includes frames F3-1, F3-2, F4-1, and F4-2 including pixels exposed by the illumination light emitted in the illumination periods Te81 and Te91. The image data Dd81 and Dd91 generated based on the imaging data D3-1, D3-2, D4-1, and D4-2 are used.

  As a result, even when illumination processing is performed using a plurality of light sources, in the case of moving image shooting in which image blurring, distortion, and color mixing are not a concern, an image using image data of consecutive frames is obtained. In the case of still image shooting, which is displayed on the display device 5 and may be affected by image blurring, distortion, and color mixing, an extinguishing period is provided before and after, and imaging data of a plurality of frames including the same illumination period are added. An image using the obtained image data is displayed on the display device 5.

DESCRIPTION OF SYMBOLS 1, 1a, 1b Endoscope system 2 Endoscope 3, 3a, 3b Light source device 4 Processing apparatus 5 Display apparatus 21 Insertion part 22 Operation part 23 Universal cord 24 Tip part 25 Bending part 26 Flexible pipe part 31, 31a, 31b Illumination unit 32, 32a, 32b Illumination control unit 33, 33a, 33b Light source 34, 34a, 34b Light source driver 35 Rotating filter 36 Drive unit 37 Drive driver 221 Bending knob 222 Treatment instrument insertion unit 223 Switch 241 Light guide 242 Illumination lens 243 Optical system 244 Imaging device 244a Sensor unit 244b Analog front end (AFE) unit 244c P / S conversion unit 244d Timing generator 244e Imaging control unit 244f Light receiving unit 244g Reading unit 244h Noise reduction unit 244i AGC unit 244j A / D Section 245 collective cable 401 S / P conversion unit 402 image processing unit 403 brightness detector 404 dimmer 405 read address setting section 406 driving signal generation unit 407 input unit 408 recording unit 409 the control unit 410 reference clock generation unit

Claims (5)

An illumination unit that emits illumination light to illuminate the subject;
A light receiving unit in which pixels that generate an imaging signal by receiving light and performing photoelectric conversion are arranged two-dimensionally;
A readout unit that sequentially reads out the imaging signals for each horizontal line from the light receiving unit;
The reading unit turns off the illumination light in the illumination unit for a first period which is one frame period corresponding to a period from the start of reading of the first horizontal line of the light receiving unit to the start of reading of the next first horizontal line. Control for causing the illumination unit to execute a series of processes for causing the illumination unit to emit the illumination light within a second period having a length equal to or longer than the one frame period following the first period. And
By adding the previous SL second image pickup signal in which the reading unit has read to have you in the first image pickup signal and the second period in which the reading portion have contact to the first period is read out, the second An image processing unit that generates an image signal corresponding to the illumination light emitted during the period;
An imaging system comprising:   The imaging system according to claim 1, wherein the illumination control unit causes the illumination unit to execute the series of processes in synchronization with the readout timing of the first and second periods by the readout unit. An input unit for receiving at least an instruction signal for taking a still image;
The lighting control section, when the input unit is not accepting input of the instruction signal to emit the illumination light continuously to the illumination unit, when the input unit accepts an input of the instruction signal is the Let the lighting unit execute the series of processes,
The image processing unit generates an image signal for display based on an imaging signal according to the illumination light continuously emitted when the instruction signal is not input to the input unit, and 3. The display image signal is sequentially generated by adding the first imaging signal and the second imaging signal when the instruction signal is input. The imaging system described. The illumination control unit performs a process of causing the illumination unit to emit the illumination light within the second period having a length that is an integer multiple of 1 or more of the one frame period. The imaging system according to any one of? The said illumination control part controls the said illumination part so that the emission of the said illumination light continues from the start to the end of the said 2nd period, The Claim 1 characterized by the above-mentioned. Imaging system.

Images