WO2015114906A1

WO2015114906A1 - Imaging system and imaging device

Info

Publication number: WO2015114906A1
Application number: PCT/JP2014/079977
Authority: WO
Inventors: 紗依里齋藤; 秀範橋本; 田中　靖洋
Original assignee: オリンパス株式会社
Priority date: 2014-01-29
Filing date: 2014-11-12
Publication date: 2015-08-06
Also published as: JPWO2015114906A1

Abstract

This imaging system comprises: an imaging element (244) that has a sensor unit (244a) having a light receiving unit (244f) provided with a plurality of pixels and a reading unit (244g) to read an electric signal generated by the plurality of pixels as image information, an imaging control unit (244e) to generate a read signal on the basis of an input synchronization signal and output the read signal to the reading unit (244g), and a timing information generating unit (2441) to generate timing information pertaining to the timing of illumination by way of an illuminating unit (31) according to read operations by the reading unit; a light source device (3) to emit illuminating light; and a control unit (408) that acquires the timing information generated by the timing information generating unit (2441) and controls the illuminating light emissions by way of an illuminating control unit (32) on the basis of the acquired timing information.

Description

Imaging system and imaging apparatus

The present invention relates to, for example, an imaging device including an imaging device having a plurality of pixels, and an imaging system that acquires an imaging signal captured by the imaging device.

Conventionally, in the medical field, an endoscope system is used to observe an organ of a subject such as a patient. An endoscope system includes, for example, an endoscope having an imaging element provided at the distal end and having an elongated shape having flexibility and being inserted into a body cavity of a subject, and a proximal end side of the insertion portion. And a processing device that is connected via a cable and performs image processing of an in-vivo image in accordance with an imaging signal imaged by an imaging element, and displays the in-vivo image on a display unit or the like.

When acquiring an in-vivo image using the endoscope system, after inserting the endoscope into the body cavity of the subject, irradiate the living tissue in the body cavity with illumination light such as white light from the distal end of the endoscope Then, the imaging element captures an in-vivo image. The insertion unit performs signal processing such as A / D conversion on the video signal picked up by the image sensor, and outputs the signal-processed video signal to the processing device. A user such as a doctor observes the organ of the subject based on the in-vivo image displayed by the processing device.

A CMOS (Complementary Metal Oxide Semiconductor) sensor is used for the image sensor. The CMOS sensor generates image data by, for example, a rolling shutter method in which reading is performed while shifting the timing for each line.

As an endoscope system using such a CMOS sensor, a technique for obtaining illumination light using a semiconductor light source such as an LED (Light Emitting Diode) is disclosed (for example, see Patent Document 1).

International Publication No. 2012/033200

However, in Patent Document 1 described above, since the control device controls the readout timing of the image sensor and the light emission timing of the semiconductor light source, if the parameters for each timing do not match, the readout timing of the image sensor and the semiconductor There may be a deviation in the light emission timing of the light source. In particular, since the parameters related to the readout timing of the image sensor differ depending on the specifications of the image sensor, when trying to control with the parameters related to the uniform timing, the specifications of the image sensor are limited to those that can operate with the parameters related to the uniform timing Will be. In addition, the image sensor has variations in characteristics for each individual, and it is necessary to individually set parameters related to the readout timing of the image sensor.

The present invention has been made in view of the above, and an object of the present invention is to provide an imaging system and an imaging apparatus that can eliminate a deviation between the readout timing of the imaging element and the light emission timing of the light source.

In order to solve the above-described problems and achieve the object, an imaging system according to the present invention includes an illumination unit that emits illumination light, and an illumination control unit that controls emission of the illumination light by the illumination unit, Based on a light receiving unit provided with a plurality of pixels that photoelectrically convert received light to generate an electric signal, a reading unit that reads out the electric signals generated by the plurality of pixels as image information, and an input synchronization signal An imaging control unit that generates a readout signal and outputs the readout signal to the readout unit; and a timing information generation unit that generates timing information related to illumination timing by the illumination unit according to a readout operation by the readout unit. An image sensor and the timing information generated by the timing information generation unit are acquired, and the illumination control unit is based on the acquired timing information A timing controller for controlling the emission of the illumination light by, characterized by comprising a.

In the imaging system according to the present invention, in the above invention, the timing information includes a delay time that is a processing time from when the imaging element receives the synchronization signal until the reading unit starts reading. It is characterized by.

In the imaging system according to the present invention, the imaging device includes a superimposing unit that superimposes a timing signal including the timing information on an image signal including the image information output from the reading unit. Features.

In the imaging system according to the present invention as set forth in the invention described above, the superimposing unit superimposes the timing information on an information area other than information on an effective pixel area among information included in the image signal. .

In the imaging system according to the present invention, in the above invention, the delay time is a time from a rising or falling edge of a vertical synchronizing signal of the inputted synchronizing signals to a reading start by the reading unit. Features.

In the imaging system according to the present invention as set forth in the invention described above, the timing information includes a time from a read start timing to a read end timing in one frame.

In the imaging system according to the present invention as set forth in the invention described above, the timing information includes information indicating a timing at which exposure to the pixel is started by electronic shutter control.

Moreover, in the imaging system according to the present invention, in the above invention, the illumination control unit sequentially emits each illumination light of red light (R), green light (G), and blue light (B) at a predetermined timing. The illumination unit is controlled.

In the imaging system according to the present invention as set forth in the invention described above, the timing information generation unit generates the timing information during white balance adjustment.

In the image pickup system according to the present invention, in the above invention, the image pickup device operates a signal processing unit that performs signal processing on the electrical signal output from the readout unit, and the image pickup device and the illumination unit. A first clock generation unit that generates a clock signal serving as a reference for the operation timing of the image processing unit, and a predetermined image process performed on the image information output from the image sensor, and the image processing unit And a second clock generation unit that generates a clock signal that serves as a reference for operation timing for the operation.

An imaging apparatus according to the present invention is an imaging apparatus that captures an in-vivo image of a subject illuminated by illumination light emitted from a light source device, and generates an electrical signal by photoelectrically converting the light received by each A sensor unit having a light receiving unit provided with a plurality of pixels, a readout unit that reads out electrical signals generated by the plurality of pixels as image information, and generates a readout signal based on an input synchronization signal An imaging control unit that outputs the readout signal to the readout unit; and a timing information generation unit that generates timing information related to illumination timing by the light source device according to a readout operation by the readout unit. .

According to the present invention, it is possible to eliminate the difference between the readout timing of the image sensor and the light emission timing of the light source.

FIG. 1 is a diagram illustrating a schematic configuration of an endoscope system according to the first embodiment of the present invention. FIG. 2 is a block diagram illustrating a functional configuration of the endoscope system according to the first embodiment of the present invention. FIG. 3 is a diagram for explaining the exposure and readout timing of the image sensor at the time of photographing by the endoscope system according to the first embodiment of the present invention. FIG. 4 is a diagram for explaining the exposure and readout timing of the image sensor at the time of photographing by the endoscope system according to the first embodiment of the present invention. FIG. 5 is a diagram for explaining exposure and readout timing of the image sensor at the time of photographing by the endoscope system according to the modification of the first embodiment of the present invention. FIG. 6 is a block diagram illustrating a schematic configuration of the endoscope system according to the second embodiment of the present invention. FIG. 7A is a diagram for explaining an example of a data structure of an image signal output by the endoscope according to the second embodiment of the present invention. FIG. 7B is a diagram for explaining an example of a data structure of an image signal output by the endoscope according to the second embodiment of the present invention. FIG. 8 is a diagram for explaining exposure and readout timing of the image sensor at the time of photographing by the endoscope system according to the second modification of the second embodiment of the present invention. FIG. 9 is a diagram for explaining the exposure and readout timing of the image sensor at the time of photographing with the endoscope system according to the third modification of the second embodiment of the present invention. FIG. 10 is a diagram for explaining the exposure and readout timing of the image sensor at the time of photographing with the endoscope system according to the fourth modification of the second embodiment of the present invention. FIG. 11 is a diagram for explaining exposure and readout timing of the image sensor at the time of photographing by the endoscope system according to the sixth modification of the second embodiment of the present invention. FIG. 12 is a block diagram illustrating a schematic configuration of the endoscope system according to the third embodiment of the present invention.

Hereinafter, modes for carrying out the present invention (hereinafter referred to as “embodiments”) will be described. In the embodiment, a medical endoscope system that captures and displays an image of a body cavity of a subject such as a patient as an example of an imaging system will be described. Moreover, this invention is not limited by this embodiment. Furthermore, in the description of the drawings, the same portions will be described with the same reference numerals.

(Embodiment 1)
FIG. 1 is a diagram illustrating a schematic configuration of an endoscope system according to the first embodiment of the present invention. FIG. 2 is a block diagram illustrating a functional configuration of the endoscope system according to the first embodiment of the present invention.

An endoscope system 1 shown in FIGS. 1 and 2 inserts a distal end portion into a body cavity of a subject to capture an in-vivo image of the subject, and emits the light from the distal end of the endoscope 2. A light source device 3 that generates illumination light, a processing device 4 (control device) that performs predetermined image processing on an in-vivo image captured by the endoscope 2 and comprehensively controls the operation of the entire endoscope system 1. The display device 5 displays the in-vivo image on which the processing device 4 has performed image processing.

The endoscope 2 includes an insertion portion 21 having an elongated shape having flexibility, an operation portion 22 that is connected to a proximal end side of the insertion portion 21 and receives input of various operation signals, and an insertion portion from the operation portion 22. And a universal cord 23 that includes various cables that extend in a direction different from the direction in which 21 extends and connect to the light source device 3 and the processing device 4.

The insertion unit 21 includes a distal end portion 24 including an imaging element 244 (imaging device) in which pixels that generate signals by receiving light and performing photoelectric conversion are arranged in a two-dimensional manner, and a plurality of bending pieces. The bendable bending portion 25 is connected to the proximal end side of the bending portion 25 and has a long flexible tube portion 26 having flexibility.

The distal end portion 24 is configured using a glass fiber or the like, and forms a light guide path for light emitted from the light source device 3. An illumination lens 242 provided at the distal end of the light guide 241. And an image sensor 244 that is provided at an image forming position of the optical system 243, receives light collected by the optical system 243, photoelectrically converts the light into an electrical signal, and performs predetermined signal processing.

The optical system 243 is configured by using one or a plurality of lenses, and has an optical zoom function for changing the angle of view and a focus function for changing the focus.

The image sensor 244 photoelectrically converts light from the optical system 243 to generate an electrical signal (imaging signal), and performs noise removal and A / D conversion on the electrical signal output from the sensor unit 244a. An analog front end unit 244b (hereinafter referred to as “AFE unit 244b”), a P / S conversion unit 244c that performs parallel / serial conversion on a digital signal (image signal) output from the AFE unit 244b and transmits the converted signal to the outside, and a sensor unit A timing generator 244d that generates pulses of various signal processing in the AFE unit 244b and the P / S conversion unit 244c, and an imaging control unit 244e that controls the operation of the imaging device 244; The image sensor 244 is a CMOS (Complementary Metal Oxide Semiconductor) sensor.

In the sensor unit 244a, a plurality of pixels each having a photodiode that accumulates electric charge according to the amount of light and an amplifier that amplifies the electric charge accumulated by the photodiode are two-dimensionally arranged, and photoelectrically converts light from the optical system 243. A light receiving unit 244f that generates an electrical signal (imaging signal), and a reading unit 244g that sequentially reads out, as image information, an electrical signal generated by a pixel arbitrarily set as a reading target among a plurality of pixels of the light receiving unit 244f. Have. Further, the reading unit 244g sequentially reads out electrical signals (imaging signals) for each horizontal line from the light receiving unit 244f and outputs them to the AFE unit 244b.

The image sensor 244 (CMOS sensor) according to the first embodiment generates an electrical signal by a rolling shutter system in which exposure or reading is performed at different timings for each horizontal line. Further, the image sensor 244 outputs image information for each line (line data unit described later) to the processing device 4.

The AFE unit 244b reduces the noise component included in the electrical signal and adjusts the amplification factor of the electrical signal to maintain a constant output level, and outputs it via a CDS (Correlated Double Sampling) unit 244h and the CDS unit 244h. An A / D converter 244i that performs A / D conversion on the electrical signal that has been converted, and a correction unit 244j that corrects the electrical signal digitally converted by the A / D converter 244i. The CDS unit 244h performs noise reduction using, for example, a correlated double sampling method. The correction unit 244j performs image correction and color correction (tone correction (γ correction) of the RGB video signal) for pixel defects.

The P / S conversion unit 244c performs parallel / serial conversion on the digital signal (image signal) output from the AFE unit 244b, transmits the digital signal to the outside, and outputs the electric signal output from the AEF unit 244b before the parallel / serial conversion. The signal may include processing such as N-bit / M-bit encoding (N <M, hereinafter, the bit is expressed as “b”) and synchronization signal superposition. For example, the P / S conversion unit 244c performs 8b / 10b encoding processing based on the stored conversion table to convert the 8b electrical signal into the 10b electrical signal.

The imaging control unit 244e controls various operations of the distal end portion 24 according to the setting data received from the processing device 4. For example, the imaging control unit 244e outputs a readout signal to the readout unit 244g, and controls an output mode of an electrical signal output from each pixel in units of pixels. In addition, the imaging control unit 244e includes a timing information generation unit 2441 that generates timing information related to the timing at which the reading unit 244g starts reading. The imaging control unit 244e outputs the timing information generated by the timing information generation unit 2441 to the processing device 4. The timing information is information related to the illumination timing by the illumination unit 31 according to the readout operation by the readout unit 244g. The imaging control unit 244e is configured using a CPU (Central Processing Unit), a register that records various programs, and the like.

The operation section 22 includes a bending knob 221 that bends the bending section 25 in the vertical direction and the left-right direction, a treatment instrument insertion section 222 that inserts a treatment instrument such as a biological forceps, an electric knife, and a test probe into the body cavity of the subject. In addition to the processing device 4 and the light source device 3, it has a plurality of switches 223 which are operation input units for inputting operation instruction signals of peripheral devices such as air supply means, water supply means, and screen display control. The treatment tool inserted from the treatment tool insertion portion 222 is exposed from the opening (not shown) via the treatment tool channel (not shown) of the distal end portion 24.

The universal cord 23 includes at least a light guide 241 and a collective cable 245 in which one or a plurality of signal lines are collected. The collective cable 245 includes a signal line for transmitting / receiving setting data, a signal line for transmitting / receiving an image signal, a signal line for transmitting / receiving a timing signal for driving for driving the image sensor 244, and a reading unit 244g. Includes a signal line for transmitting timing information related to the timing of starting reading.

Next, the configuration of the light source device 3 will be described. The light source device 3 includes an illumination unit 31 and an illumination control unit 32.

The illumination unit 31 sequentially switches and emits a plurality of illumination lights having different wavelength bands to the subject (subject) under the control of the illumination control unit 32. The illumination unit 31 includes a light source unit 31a, a light source driver 31b, a rotary filter 31c, a drive unit 31d, and a drive driver 31e.

The light source unit 31a is configured using a white LED and one or a plurality of lenses, and emits white light to the rotary filter 31c under the control of the light source driver 31b. The white light generated by the light source unit 31a is emitted from the tip of the tip part 24 toward the subject via the rotary filter 31c and the light guide 241.

The light source driver 31b supplies white light to the light source unit 31a by supplying a current to the light source unit 31a under the control of the illumination control unit 32.

The rotary filter 31c is disposed on the optical path of white light emitted from the light source unit 31a and rotates to transmit only light in a predetermined wavelength band among the white light emitted from the light source unit 31a. Specifically, the rotary filter 31c includes a red filter 311, a green filter 312 and a blue filter 313 that transmit light having respective wavelength bands of red light (R), green light (G), and blue light (B). . The rotary filter 31c sequentially transmits light in the red, green, and blue wavelength bands (for example, red: 600 nm to 700 nm, green: 500 nm to 600 nm, blue: 400 nm to 500 nm) by rotating. As a result, the white light (W illumination) emitted from the light source unit 31a is converted into any one of red light (R illumination), green light (G illumination), and blue light (B illumination) with a narrow band. Can be sequentially emitted (surface sequential method).

The drive unit 31d is configured using a stepping motor, a DC motor, or the like, and rotates the rotary filter 31c.

The drive driver 31e supplies a predetermined current to the drive unit 31d under the control of the illumination control unit 32.

For example, the illumination control unit 32 supplies first illumination light (for example, R illumination) to the illumination unit 31 during a first period (for example, Texposure in FIG. 3) in which all lines to be read in a certain frame are blanking periods. After the first period, the second illumination light having a wavelength band different from that of the first illumination light (for example, G illumination) is supplied to the illumination unit 31 during the second period which is the blanking period of the next frame. The illumination unit 31 performs a series of processes for emitting the third illumination light (for example, B illumination) to the illumination unit 31 during the third period, which is the blanking period of the next frame, after the second period ends. Let it run repeatedly.

The light source unit 31a may be composed of a red LED, a green LED, and a blue LED, and the light source driver 31b may sequentially emit red light, green light, or blue light by supplying current to each LED. Good. Alternatively, the white LED, the red LED, the green LED, and the blue LED may emit light simultaneously, or the subject may be irradiated with white light using a discharge lamp such as a xenon lamp to acquire an image. .

Next, the configuration of the processing device 4 will be described. The processing device 4 includes an S / P conversion unit 401, an image processing unit 402, a brightness detection unit 403, a light control unit 404, a synchronization signal generation unit 405, an input unit 406, a recording unit 407, and a control. Unit 408 and a reference clock generation unit 409.

The S / P conversion unit 401 performs serial / parallel conversion on the image information (electric signal) input from the image sensor 244 and outputs the converted image information to the image processing unit 402. Note that the image information includes an imaging signal, a correction parameter for correcting the imaging element 244, and the like.

The image processing unit 402 generates an in-vivo image displayed by the display device 5 based on the image information input from the S / P conversion unit 401. The image processing unit 402 performs predetermined image processing on the image information to generate an in-vivo image. Here, as image processing, synchronization processing, optical black subtraction processing, white balance adjustment processing, color matrix calculation processing, gamma correction processing, color reproduction processing, edge enhancement processing, composition processing and format for combining a plurality of image data Conversion processing and the like. Further, the image processing unit 402 outputs the image information input from the S / P conversion unit 401 to the control unit 408 or the brightness detection unit 403.

The brightness detection unit 403 detects the brightness level corresponding to each pixel from the RGB image information input from the image processing unit 402, records the detected brightness level in a memory provided therein, and controls the control unit. Output to 408. The brightness detection unit 403 calculates a gain adjustment value based on the detected brightness level, and outputs the gain adjustment value to the image processing unit 402.

Under the control of the control unit 408, the light control unit 404 sets the amount of light generated by the light source device 3, the light emission timing, and the like based on the light irradiation amount calculated by the brightness detection unit 403. The dimming signal including it is output to the light source device 3.

The synchronization signal generation unit 405 generates a synchronization signal including at least a vertical synchronization signal, transmits the synchronization signal to a timing generator 244d via a predetermined signal line included in the aggregate cable 245, and also transmits to the image processing unit 402 inside the processing device 4. Send.

The input unit 406 receives input of various signals such as an operation instruction signal that instructs the operation of the endoscope system 1.

The recording unit 407 is realized by using a semiconductor memory such as a flash memory or a DRAM (Dynamic Random Access Memory). The recording unit 407 records various programs for operating the endoscope system 1 and data including various parameters necessary for the operation of the endoscope system 1. The recording unit 407 records the identification information of the processing device 4. Here, the identification information includes unique information (ID) of the processing device 4, year type, specification information, transmission method, transmission rate, and the like.

The control unit 408 is configured using a CPU or the like, and performs drive control of each component including the imaging device 244 and the light source device 3, input / output control of information with respect to each component, and the like. The control unit 408 transmits setting data (for example, pixels to be read) recorded in the recording unit 407 for imaging control to the imaging control unit 244e via a predetermined signal line included in the collective cable 245. To do. The control unit 408 functions as a timing control unit that generates a drive signal for driving the light source based on timing information including the exposure timing and readout timing of each line of the image sensor 244 and outputs the drive signal to the light source device 3.

The reference clock generation unit 409 generates a clock signal that is a reference for the operation of each component of the endoscope system 1 and supplies the generated clock signal to each component of the endoscope system 1.

Next, the display device 5 will be described. The display device 5 receives and displays the in-vivo image corresponding to the in-vivo image information generated by the processing device 4 via the video cable. The display device 5 is configured using liquid crystal or organic EL (Electro Luminescence).

Subsequently, the exposure and readout timing of the image sensor at the time of photographing by the endoscope system 1 will be described with reference to the drawings. 3 and 4 are diagrams for explaining the exposure and readout timing of the image sensor when photographing with the endoscope system 1. In the image sensor 244 according to the first embodiment, the readout unit 244g shifts the timing for each horizontal line and reads out the electrical signals of the first to nth lines (one frame), and the subject by illumination light. The in-vivo image of the subject is acquired by alternately repeating the blanking period for performing the illumination.

When the first to nth lines (one frame) are read by the reading unit 244g, the read start time of the first line is T read, the read end time of the nth line is T read-end, and one line is read. In the blanking period, the illumination light is irradiated on all lines from the 1st line to the nth line during the period of T read-length (line), the reading period of 1 frame is T read-length (Frame) Let T exposure be a period. The read start time T read for the first line corresponds to a delay period (time, number of clocks, or number of lines and number of clocks) from when the vertical synchronization signal rises until the reading unit 244g starts reading. Note that the read start time T read for the first line may be a delay period from when the vertical synchronization signal falls until the read unit 244g starts reading.

As shown in FIG. 4, when the vertical synchronizing signal rises (the pulse goes high), the reading unit 244g sequentially reads out an electrical signal (imaging signal) for each horizontal line from the light receiving unit 244f and outputs it to the AFE unit 244b. . In other words, when the vertical synchronization signal rises, after a predetermined delay time elapses, the readout start time Tread switches from the blanking period to the readout period. At this time, the delay time from the rise of the vertical synchronization signal to the start of reading varies depending on the endoscope 2 to be used and the drive mode. Specifically, the delay time varies depending on the individual difference (processing ability) of the image sensor 244 employed by the endoscope 2. Even if the same image sensor 244 is used, the delay time calculation method varies depending on the drive mode, and therefore the delay time itself also varies depending on the drive mode.

In the first embodiment, the start timing (start time T read) at which the reading unit 244g starts reading is output as timing information to the processing device 4 via the signal line. Thereby, even when the reading start timing by the reading unit 244g of the image pickup device 244 is different, the read start timing according to the image pickup device or the drive mode can be acquired.

The control unit 408 controls the illumination control unit 32 based on the acquired timing information, so that the parameters relating to the timings of imaging and illumination are different regardless of the difference in readout start timing depending on the specifications of the imaging element and the drive mode. Even in this case, it is possible to irradiate the illumination light during the period T exposure during which the illumination light is irradiated on all the lines from the first line to the n-th line. As a result, it is possible to acquire an image in which illumination light is uniformly irradiated on each line.

According to the first embodiment described above, the image sensor 244 outputs the start timing (start time T read) at which the reading unit 244g starts reading to the processing device 4 via the predetermined signal line as timing information. Since the control unit 408 controls the illumination control unit 32 based on the acquired timing information, the readout timing of the image sensor is different even when the readout start timing by the readout unit 244g of the image sensor 244 is different. And the light emission timing of the light source can be eliminated. Thereby, an in-vivo image with uniform brightness and an in-vivo image without color mixture can be obtained regardless of the specifications of the image sensor and the drive mode.

In addition, according to the first embodiment described above, the light emission timing of the light source can be controlled in accordance with the readout timing of the image sensor regardless of the specifications of the image sensor and the drive mode, so only reading out all pixels. Even if a CMOS sensor that can read out pixels in a specific range or read out the order of the physical arrangement of pixels is used, there is a difference between the readout timing of the image sensor and the light emission timing of the light source. The imaging process can be performed without the occurrence of.

In the first embodiment described above, it has been described that the start timing (start time T read) at which the reading unit 244g starts reading is output as timing information to the processing device 4 via the signal line. Alternatively, a high level signal may be output, and a low level signal may be output during a period when reading is not performed. In this case, the control unit 408 controls the illumination timing based on the change in the level of this signal. The timing information can be applied as long as it can be the illumination timing by the illumination unit 31 according to the readout operation by the readout unit 244g.

(Modification of Embodiment 1)
FIG. 5 is a diagram for explaining the exposure and readout timing of the image sensor at the time of photographing by the endoscope system 1 according to the modification of the first embodiment. In the first embodiment described above, it has been described that the image information is generated by the rolling shutter method in which exposure or reading is performed at different timings for each line. However, in this modification, a global that performs exposure or reading simultaneously on all lines is described. Image information is generated by a shutter method.

Also in the present modification, the start timing (start time T read) at which the reading unit 244g starts reading is output as timing information to the processing device 4 via the signal line. Thereby, even when the reading start timing by the reading unit 244g of the image pickup device 244 is different, the read start timing according to the image pickup device or the drive mode can be acquired.

The control unit 408 controls the illumination control unit 32 based on the acquired timing information, so that the first to n-th lines are controlled regardless of the difference in readout start timing depending on the specifications of the image sensor and the drive mode. Illumination light can be irradiated during a period T exposure during which illumination light is irradiated in all lines.

According to the above-described modification, even when image information is generated by the global shutter method, the deviation between the readout timing of the image sensor and the light emission timing of the light source is eliminated regardless of the specification and drive mode of the image sensor. can do.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. FIG. 6 is a block diagram showing a schematic configuration of the endoscope system 1a according to the second embodiment of the present invention. In addition, the same code | symbol is attached | subjected and demonstrated to the structure same as the structure mentioned above. In the first embodiment described above, timing information is output from the image sensor 244 to the processing device 4 via a predetermined signal line. However, in the second embodiment, the timing information is superimposed on the image signal from the image sensor 244. Timing information is output to the processing device 4.

The endoscope system 1a according to the second embodiment includes a superimposition unit 244k that performs signal processing between the P / S conversion unit 244c and the correction unit 244j. The superimposing unit 244k superimposes timing information on the image signal subjected to the correction processing by the correcting unit 244j.

7A and 7B are diagrams for explaining an example of the data structure of the image signal output by the endoscope 2 according to the second embodiment of the present invention. FIG. 7A is a diagram illustrating frame data 6 corresponding to pixel data of one frame. FIG. 7B is a diagram illustrating line data 7 corresponding to pixel data of one line (line L in FIG. 7A).

As shown in FIG. 7A, frame data 6 of one frame (one image) acquired by the image sensor 244 is stored in a start code area 61, a header area 62, an image data area 63, a footer area 64, and an end code area 65. Divided.

The start code area 61 and the end code area 65 are assigned control codes indicating the beginning and the end of the line data 7, respectively.

The header area 62 includes additional information such as information stored in the image data area 63 (for example, a line number, a frame number, and an error correction code paired with the header information).

In the image data area 63, information corresponding to an electrical signal (imaging signal) generated by the imaging element 244 (sensor unit 244a) is stored. The image data area 63 includes an effective pixel area 631 that is image data of effective pixels, a horizontal blanking area 632 that is a margin area provided at the head of the horizontal line of the effective pixel area 631, and the effective pixel area 631 and the horizontal blanking area. The first vertical blanking area 633 is set on the upper end side of the ranking area 632, and the second vertical blanking area 634 is set on the lower end side of the effective pixel area 631 and the horizontal blanking area 632.

In the footer area 64, for example, an error correction code paired with information stored in the image data area 63 is assigned.

As shown in FIG. 7B, the line data 7 corresponding to the pixel data of one line is provided with a start code 71 at the head and a header 72, image data 73, a footer 74, and an end code 75 in order. FIG. 7B shows the line data of the line L having the first vertical blanking area 633 of the image data area 63, and therefore blanking data is given to the area of the image data 73.

In the second embodiment, the start timing (start time) at which the reading unit 244g starts reading the line data 7 after the superimposition unit 244k performs correction processing such as image correction and color correction for pixel defects. T timing) D is superimposed on the blanking data. That is, in the second embodiment, the superimposing unit 244k performs a superimposition process in which the timing information D is stored in the first vertical blanking region 633, whereby the timing signal including the timing information D is superimposed on the image signal. The data is output from 244 to the processing device 4. Thereby, even when the reading start timing by the reading unit 244g of the image pickup device 244 is different, the read start timing according to the image pickup device or the drive mode can be acquired.

The control unit 408 controls the illumination control unit 32 based on the timing information extracted from the image signal by the image processing unit 402 performing signal processing. As a result, the illumination light is irradiated during the period T exposure during which the illumination light is irradiated on all the lines from the first line to the n-th line, regardless of the difference in the readout start timing depending on the specifications of the image sensor and the drive mode. can do.

According to the second embodiment described above, as in the first embodiment, the image sensor 244 uses the superimposition unit 244k to generate an image signal by using the start timing (start time Tread) at which the reading unit 244g starts reading as timing information. When the reading start timing by the reading unit 244g of the image sensor 244 is different because the control unit 408 controls the illumination control unit 32 based on the acquired timing information. Even so, the difference between the readout timing of the image sensor and the light emission timing of the light source can be eliminated. Thereby, an in-vivo image with uniform brightness and an in-vivo image without color mixture can be obtained regardless of the specifications of the image sensor and the drive mode.

Further, according to the second embodiment described above, timing information is superimposed on the image signal and output to the processing device 4, so that the timing information is output to the processing device 4 without increasing the number of signal lines. It becomes possible. Thereby, the diameter of the aggregate cable 245 due to an increase in signal lines can be prevented.

In the second embodiment described above, the timing information is given to the first vertical blanking area. However, the timing information may be given to the second vertical blanking area or to the horizontal blanking area. Alternatively, it may be given to the header area, or may be given to the surplus data area for adjusting the timing of the effective pixel area.

(Modification 1 of Embodiment 2)
In the second embodiment described above, the timing information has been described as being stored in the image data area 63. However, in the first modification, the timing information is stored in an arbitrary position (data) of the line data 7. For example, it may be stored in a line data header (for example, the first line header 72) having the first vertical blanking region 633. Further, it can be provided at an arbitrary position in the header 72. Specifically, the timing at which arbitrary data (for example, data representing the beginning of a frame) has an arbitrary value is specified, and timing information is stored at the specified position.

(Modification 2 of Embodiment 2)
FIG. 8 is a diagram for explaining the exposure and readout timing of the image sensor at the time of photographing by the endoscope system 1a according to the second modification of the second embodiment. In the second embodiment described above, the timing for starting reading (start time T read for the first line) is described as timing information. However, in the second modification, timing information is included in each header of all horizontal lines. Is stored. This timing information includes information on the clock at which the reading is started in the line next to the line to which the timing information is added.

(Modification 3 of Embodiment 2)
FIG. 9 is a diagram for explaining the exposure and readout timing of the image sensor at the time of photographing by the endoscope system 1a according to the third modification of the second embodiment. In the second embodiment described above, the start timing (start time T read of the first line) is described as timing information. However, as in the third modification, the first line is read. Even when the start timing (start time T read for the first line) and the end timing (end time T read-end for the n-th line) to finish reading are used as timing information. Good.

In the rolling shutter system, since the timing of reading start varies depending on the line, the period from the start of reading the first line of the effective pixel region to the completion of reading of the last line of the effective pixel region is important. As in Modification 3, by adding an end timing (n-th line start time T read-end) to the readout of the n-th line to the timing information, the readout timing of the image sensor and the light emission timing of the light source The deviation can be solved more reliably.

(Modification 4 of Embodiment 2)
FIG. 10 is a diagram for explaining the exposure and readout timing of the image sensor at the time of photographing by the endoscope system 1a according to the fourth modification of the second embodiment. In Modification 3 described above, the start timing for starting the reading of the first line (start time T read for the first line) and the end timing for ending reading of the nth line (start time T read for the nth line). -end) is used as the timing information, but as in Modification 4, the period from the start of reading the first line to the end of reading the n-th line (read time corresponding to one frame) T read-length) may be used as timing information.

(Modification 5 of Embodiment 2)
When the charge accumulation period of the image sensor is controlled by controlling the lighting period of the light source without using the electronic shutter, the charge accumulation period of the photodiode of the sensor unit 244a (the blanking period described above) is the mth time. Since this is a period (light source lighting period) from the read timing to the (m + 1) th read start timing, the light source control timing of the illumination control unit 32 is obtained by acquiring timing information as in the first and second embodiments. Can be determined.

However, when an electronic shutter is used, the charge accumulation period of the photodiode is a period from the timing of the electronic shutter after the m-th reading to the m + 1-th reading start timing. In the period in which the electric charge accumulated by the electronic shutter is discarded, the read data is not affected regardless of the incident light. For this reason, by outputting timing information (information relating to the accumulation period) to the processing device 4 using the charge accumulation timing of the electronic shutter as a read start timing, for example, illumination light is emitted in the above-described period T exposure, and uniform illumination light is emitted. Irradiation can be performed.

(Modification 6 of Embodiment 2)
FIG. 11 is a diagram for explaining the exposure and readout timing of the image sensor at the time of photographing by the endoscope system according to the sixth modification of the second embodiment. In the second embodiment described above, the purpose is to understand the readout period or charge accumulation time of all the effective pixel areas. However, in the case of performing clipping or binning in the CMOS sensor, the readout target pixels (for example, FIG. 11) in the effective pixel area. In this case, the region R) can be subjected to timing information, so that the period during which pixels that are not to be read are exposed can be ignored and illumination light can be uniformly irradiated to the pixels to be read.

(Modification 7 of Embodiment 2)
In

Modifications

3 and 4 described above, the read start timing of the first line and the read end timing of the nth line are used as timing information. However, the read start timing and read end timing of each line may be used as timing information. Good. In this case, timing information is given to the header of each line data. Thereby, when a light source with high temporal control accuracy is used, brightness (light quantity, etc.) can be controlled according to the charge accumulation period for each line.

(Embodiment 3)
Next, a third embodiment of the present invention will be described. FIG. 12 is a block diagram illustrating a schematic configuration of the endoscope system 1b according to the third embodiment of the present invention. In addition, the same code | symbol is attached | subjected and demonstrated to the structure same as the structure mentioned above. In the first and second embodiments described above, the processing device 4 is described as controlling the entire endoscope system, but the endoscope 2 may be configured to control the entire endoscope system.

In the endoscope system 1b shown in FIG. 12, the imaging element 244 of the endoscope 2 described above further includes a reference clock generation unit 244l. Further, the endoscope system 1b does not include the above-described synchronization signal generation unit 405. In the third embodiment, a reference clock generation unit 244l is provided in the image sensor 244, and the read start timing and the illumination light irradiation timing are controlled by the clock generated by the reference clock generation unit 244l. In other words, in the endoscope system 1b, the timing between the reading by the reading unit 244g and the emission of the illumination light by the illumination control unit 32 is controlled based on the clock generated by the reference clock generation unit 244l. In the third embodiment, the light source device 3 operates based on the clock generated by the reference clock generation unit 244l, and the reference clock generation unit 409 operates the clock for operating the internal configuration of the processing device 4 such as the image processing unit 402. Is generated.

According to the third embodiment, since the read start timing and the illumination light irradiation timing are controlled by the clock generated by the reference clock generation unit 244l, the clock generated by the reference clock generation unit 244l of the image sensor 244 is used. Even if there is a frequency deviation in the clock generated by the reference clock generation unit 409 of the control device 4, the read start timing and the illumination light irradiation timing can be matched with high accuracy. Note that the reference clock generation unit 244l is not limited to being provided inside the imaging device, but may be provided anywhere in the endoscope 2.

In the first to third embodiments, the read start time T read as timing information has been described as a delay period from when the vertical synchronization signal rises to when the read unit 244g starts reading. In addition to the above timing, any data in the header or horizontal blanking area in the image signal may be used as the reference timing, and the delay time from the reference timing to the read start time may be used as the timing information. If the reference timing is clear, any position (data) can be applied as the reference timing.

In the first to third embodiments, it is preferable that the readout start timing of the first line is the first horizontal line of the line having the effective pixel region, from the viewpoint of acquiring the image of the effective pixel region.

In the first to third embodiments, the timing information may be acquired at the time of calibration of the endoscope 2 such as white balance. Timing information is acquired at the time of white balance, and the timing of reading start by the reading unit 244g and the irradiation timing of illumination light by the illumination control unit 32 are combined to acquire timing information at the time of observation and calculate the delay time. It is possible to perform imaging processing by irradiating illumination light at an optimal timing without performing processing such as performing processing, and processing performed by the endoscope 2 or the processing device 4 during observation can be reduced.

In the first to third embodiments, the imaging control unit 244e has been described as including the timing information generation unit 2441. However, the timing information generation unit is provided by individually providing the imaging element 244 with a timing information generation unit. The timing information may be output to the processing device 4.

In the second embodiment, the superimposing unit 244k is provided in the image sensor 244, and the data on which the timing information is superimposed by the superimposing unit 244k is output to the processing device 4. However, the AFE such as the correcting unit 244j is used. The unit 244b may perform superimposition processing.

In the first to third embodiments, the control unit 408 of the control device 4 has been described as controlling the driving of the light source device 3 based on the acquired timing information. However, the light source device 3 has a control unit. The control unit may acquire timing information and drive based on the timing information.

As described above, the imaging system and the imaging apparatus according to the present invention are useful for eliminating the deviation between the readout timing of the imaging device and the light emission timing of the light source, regardless of the external environment.

1, 1a, 1b Endoscope system 2 Endoscope 3 Light source device 4 Processing device (control device)
DESCRIPTION OF SYMBOLS 31 Illumination part 31a Light source part 31b Light source driver 31c Rotation filter 31d Drive part 31e Drive driver 32 Illumination control part 244 Image sensor (imaging device)

244a Sensor unit

244b AFE unit 244c P / S conversion unit 244d Timing generator 244e Imaging control unit 244f Light receiving unit

244g Reading unit

244k Superimposition unit 244l, 409 Reference clock generation unit 245 Aggregate cable 401 S / P conversion unit 402 Image processing unit 405 Synchronization Signal generation unit 408 Control unit (timing control unit)
2441 Timing information generator

Claims

An illumination unit that emits illumination light;
An illumination control unit for controlling emission of the illumination light by the illumination unit;
A light receiving unit provided with a plurality of pixels that photoelectrically convert light received by each to generate an electrical signal, a reading unit that reads out the electrical signals generated by the plurality of pixels as image information, and an input synchronization signal An imaging control unit that generates a readout signal based on the readout unit and outputs the readout signal to the readout unit; a timing information generation unit that generates timing information related to illumination timing by the illumination unit according to a readout operation by the readout unit; An imaging device having:
A timing control unit that acquires the timing information generated by the timing information generation unit, and controls emission of illumination light by the illumination control unit based on the acquired timing information;
An imaging system comprising:
The imaging system according to claim 1, wherein the timing information includes a delay time that is a processing time from when the imaging device receives the synchronization signal until the reading unit starts reading.
2. The imaging system according to claim 1, wherein the imaging device includes a superimposing unit that superimposes a timing signal including the timing information on an image signal including the image information output from the reading unit.
4. The imaging system according to claim 3, wherein the superimposing unit superimposes the timing information on an information area other than information on an effective pixel area among information included in the image signal.
3. The imaging system according to claim 2, wherein the delay time is a time from a rising or falling edge of a vertical synchronizing signal of the inputted synchronizing signals to a reading start by the reading unit.
The imaging system according to claim 1, wherein the timing information includes a time from a read start timing to a read end timing in one frame.
The imaging system according to claim 1, wherein the timing information includes information indicating a timing at which exposure of the pixel is started by electronic shutter control.
2. The imaging system according to claim 1, wherein the illumination control unit controls the illumination unit to sequentially emit red light, green light, and blue light at predetermined timings.
2. The imaging system according to claim 1, wherein the timing information generation unit generates the timing information during white balance adjustment.
The image sensor is
A signal processing unit that performs signal processing on the electrical signal output by the reading unit;
A first clock generation unit that generates a clock signal serving as a reference of operation timing for operating the imaging element and the illumination unit;
Have
An image processing unit that performs predetermined image processing on the image information output by the image sensor;
A second clock generation unit that generates a clock signal serving as a reference of operation timing for the image processing unit to operate;
The imaging system according to claim 1, further comprising:
An imaging device that captures an in-vivo image of a subject illuminated by illumination light emitted from a light source device,
A sensor unit having a light receiving unit provided with a plurality of pixels that photoelectrically convert light received by each to generate an electrical signal, and a readout unit that reads out the electrical signals generated by the plurality of pixels as image information;
An imaging control unit that generates a readout signal based on an input synchronization signal and outputs the readout signal to the readout unit;
A timing information generating unit that generates timing information related to illumination timing by the light source device according to a reading operation by the reading unit;
An imaging apparatus comprising: