JP5618624B2 - Endoscope system - Google Patents

Description

  The present invention relates to an endoscope system.

  As an imaging device, for example, there is a CCD type image sensor that transfers a charge signal photoelectrically converted by a photoelectric conversion device constituting each pixel by a CCD (Charge Coupled Device) and amplifies the transferred charge signal in a subsequent stage. On the other hand, there is a CMOS image sensor that can be mass-produced by applying a manufacturing process of CMOS (Complementary Metal Oxide Semiconductor) logic, is inexpensive, and consumes less power. This CMOS image sensor can read out electrical signals that have been photo-converted for each unit cell in any order. Recently, technologies for improving photosensitivity and reducing noise have been established, and as a high-resolution image sensor. Is being used.

  When a mechanical shutter is not used, this CMOS type image sensor is driven by a rolling shutter system different from the shutter system of the CCD type image sensor, and starts to sequentially expose a large number of two-dimensionally arranged pixels for each pixel row. In this method, charges accumulated in the pixels are sequentially read out in units of rows. Therefore, when there is a time difference in the charge accumulation period at the top and bottom of the frame and the subject is moving, the exposure time is shifted for each scanning row, resulting in an unnatural image in which the subject flows at the top and bottom of the screen.

  FIG. 16 is a timing chart showing an example of exposure / read timing of an image sensor using a rolling shutter. In the figure, for example, in a CMOS sensor having each pixel row of Line (1) to Line (n) (n is an integer of 2 or more) to be a scanning line, after resetting each pixel (0 to m) on the line. Exposure to the photodiode is started, and after a predetermined exposure time has elapsed, the charge accumulated in each photodiode is transferred and a charge signal is output. Such operations are sequentially delayed from Line (1) to Line (n). That is, scanning and reading of the lines of the light receiving portions (photodiodes) arranged in the main scanning direction are sequentially performed in the sub-scanning direction to read out the charges to generate an n-row m-column image (hereinafter also referred to as a normal scanning mode). .

  For this reason, the time difference between the top and bottom of the frame is hardly a problem for a stationary subject. For example, when the subject S moves to the right in the horizontal direction in FIG. 17, a movement amount Δ is generated in Line (n) in the last line in FIG. 17, and the subject S is inclined like a trapezoid Sa indicated by a thin line connecting the left and right ends of each line. Thus, in an endoscopic device using a CMOS image sensor, when the entire screen is moved, the screen is projected so as to flow. Since the endoscope insertion part is often moved, it becomes easy to feel uncomfortable.

  In order to solve this problem, Patent Document 1 proposes a technique for reducing image distortion by image processing. However, in the case of an endoscope apparatus, it is difficult to estimate an accurate movement amount of a subject by image processing, and the technique disclosed in Patent Document 1 displays images without sufficiently correcting screen distortion. Furthermore, in the case of an endoscopic device, the captured image signal must be transmitted to the control device with a thin and long cable, the signal attenuation in the high frequency band is large, and the transmission speed is limited. There is a disadvantage that the time difference between the beginning and end of the screen becomes large and the screen flow amount increases.

  Patent Document 2 proposes a technique for eliminating screen distortion by aligning the exposure timing of all pixels by mechanical shutter and all pixel reset. In this configuration, the synchronism of the exposure period is ensured by accumulating charges only during the time when reading is not performed using the mechanical shutter. However, there is a limit to the amount of light that can be received in the observation by the endoscope apparatus, and the above method cannot store a sufficient charge and cannot secure the dynamic range required for the observation image.

JP 2009-141717 A JP 2006-191236 A

  The present invention has been made in view of the above situation, and when acquiring image information from a CMOS image sensor using a rolling shutter method, the distortion of a captured image caused by a temporal shift in imaging timing is made inconspicuous. An object of the present invention is to provide an endoscope system that can reduce a sense of incongruity.

The present invention has the following configuration.
(1) an endoscope comprising an imaging unit, an external controller signal is input of captured images generated by the imaging unit the endoscope is detachably connected, is connected to the external control device An endoscope system comprising a display device for displaying image information ,
It is orthogonal to the main scanning direction that the imaging unit reads out a plurality of light receiving units arranged two-dimensionally and a charge signal stored in each light receiving unit along a main scanning direction that is an arrangement direction of the light receiving units. A scanning drive unit that performs scanning and reading that is repeated a plurality of times in the sub-scanning direction.
The scan driver, the main scanning reading of lines of the light receiving portions arranged in the scanning direction are sequentially performed respectively in the sub-scanning direction, a first imaging mode to read out the charge signals from all of the light receiving portion,
The scan reading of the line of the light receiving portion, sequentially performs while interlaced predetermined line in the sub-scanning direction, by sequentially performed while jumping from skipped a line predetermined line, all lines constituting the captured image and the horizontal scanning A second imaging mode for reading out the charge signal from the light receiving unit ;
Is switchable,
When the external control device acquires still image information by the imaging unit, it switches to the first imaging mode, and when it acquires moving image information, it switches to the second imaging mode and is displayed on the display device. An endoscope system for generating shaggy at a screen end of the captured image.
(2) An endoscope including an imaging unit, an external control device to which the endoscope is detachably connected and a signal of a captured image generated by the imaging unit is input, and the external control device An endoscope system comprising a display device for displaying image information,
It is orthogonal to the main scanning direction that the imaging unit reads out a plurality of light receiving units arranged two-dimensionally and a charge signal stored in each light receiving unit along a main scanning direction that is an arrangement direction of the light receiving units. A scanning drive unit that performs scanning and reading that is repeated a plurality of times in the sub-scanning direction.
In the first imaging mode, the scan driving unit sequentially scans and reads the lines of the light receiving units arranged in the main scanning direction in the sub-scanning direction, and reads the charge signals from all the light receiving units. Imaging modes,
Scanning and reading the lines of the light receiving unit is performed by repeating the horizontal scanning of two adjacent light receiving unit lines and then jumping over the next two lines to read the charge signals from all the light receiving units. A second imaging mode;
Is switchable,
When the external control device acquires still image information by the imaging unit, it switches to the first imaging mode, and when it acquires moving image information, it switches to the second imaging mode and is displayed on the display device. An endoscope system for generating shaggy at a screen end of the captured image.

  According to the endoscope system of the present invention, when acquiring still image information, the charge signals of the lines of the respective light receiving units are sequentially read in the sub-scanning direction in the first imaging mode, and when acquiring moving image information, the second image signal is acquired. By reading out the charge signal of the line of each light receiving unit in the imaging mode while skipping the line, it is possible to suppress the distortion of the captured image due to the time lag of the imaging timing. Thereby, even if the endoscope insertion part is moved while gazing at the observation screen, an image without a sense of incongruity can be obtained.

It is a figure for demonstrating embodiment of this invention, and is a notional block block diagram of an endoscope system. It is an external view as an example of the endoscope system shown in FIG. It is a circuit diagram which shows the structure of a solid-state image sensor. It is a figure which shows the exposure and read-out timing of the image pick-up element by 1 line interlace scanning mode. FIG. 5A is an example of a captured image, and FIG. 5B is a diagram illustrating a captured image generated according to the readout order of charge signals in the one-line interlaced scanning mode illustrated in FIG. 4. It is a figure which shows the captured image produced | generated by the 1 line interlace scanning mode shown in FIG. It is a figure which shows the exposure and read-out timing of an image pick-up element by 2 line interlace scanning mode. It is a figure which shows the captured image produced | generated according to the read-out order of the charge signal by 2 line interlace scanning mode shown in FIG. It is a figure which shows the captured image produced | generated by 2 line interlace scanning mode shown in FIG. It is a typical top view showing a part of light-receiving part arranged by Bayer. FIG. 8 is a schematic plan view showing color components detected by each light receiving unit in the 2-line interlaced scanning mode shown in FIG. 7. FIG. 8 is a schematic plan view showing color components detected by each light receiving unit arranged in the order of readout lines of charge signals in the two-line interlaced scanning mode shown in FIG. 7. It is a figure which shows the exposure and read-out timing of the image pick-up element by 2 line lump 2 line interlace scanning mode. It is a figure which shows the captured image produced | generated by the 2 line batch 2 line interlaced scanning mode shown in FIG. FIG. 14 is a schematic plan view showing color components detected by each light receiving unit in the 2-line batch 2-line interlaced scanning mode shown in FIG. 13. It is a timing chart which shows the exposure and read-out timing of the conventional image pick-up element using a rolling shutter. It is a figure which shows the captured image produced | generated by the exposure / reading timing shown in FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram for explaining an embodiment of the present invention, and is a conceptual block diagram of an endoscope system. FIG. 2 is an external view as an example of the endoscope system shown in FIG.

  As shown in FIGS. 1 and 2, an endoscope system 100 includes an endoscope 11, a control device 13 that is an external control device to which the endoscope 11 is connected, and image information that is connected to the control device 13. And a display device 15 for displaying. Furthermore, an input unit 17 that receives an input operation is connected to the control device 13. The endoscope 11 includes an illumination optical system that emits illumination light from the distal end of the endoscope insertion portion 19, an image pickup device that is an image pickup unit that picks up an image of an observation area, and an objective lens unit (none of which is shown). It is an electronic endoscope which has the imaging part 21 comprised by these.

  The endoscope 11 includes an endoscope insertion portion 19 whose distal end is inserted into a subject, and a bending operation and observation of the distal end of the endoscope insertion portion 19 that are connected to the proximal end portion of the endoscope insertion portion 19. An operation unit 25 that performs an operation for the operation, a universal cable 27 extending from the operation unit 25, and a connector unit 29A that is provided at the distal end of the universal cable 27 and detachably connects the endoscope 11 to the control device 13. 29B (see FIG. 2).

  The endoscope insertion portion 19 includes a flexible soft portion 31, a bending portion 33, and a tip portion (hereinafter also referred to as an endoscope tip portion) 35. Although not shown, the endoscope distal end portion 35 has an irradiation port for irradiating light to the observation region and an imaging element such as a CMOS (Complementary Metal-Oxide Semiconductor) image sensor for acquiring image information of the observation region. The objective lens unit is disposed on the light receiving surface of the image sensor.

  The bending portion 33 is provided between the flexible portion 31 and the endoscope distal end portion 35, and can be bent by a turning operation of an angle knob 41 disposed in the operation portion 25. The bending portion 33 can be bent in an arbitrary direction and an arbitrary angle according to the part of the subject in which the endoscope 11 is used, and the observation direction of the irradiation port of the endoscope distal end portion 35 and the imaging element can be changed. , Can be directed to a desired observation site.

  The endoscope distal end portion 35 is provided with a motion detection portion 37 for detecting that the endoscope distal end portion 35 has moved. As the small motion detection unit 37 that can be disposed at the endoscope distal end portion 35, for example, a small acceleration sensor is exemplified, and can be manufactured by a MEMS (Micro Electro Mechanical Systems) technique. In addition, the operation unit 25 is provided with a changeover switch 39 that is an operation unit for manually switching a scanning mode, which will be described later, between a normal scanning mode and an interlaced scanning mode. Furthermore, an operation detection unit 47 that detects that the angle knob 41 has been operated to bend the bending portion 33 is provided.

  As shown in FIG. 1, the control device 13 includes a light source device 43 that is an illuminating unit that generates illumination light to be supplied to the irradiation port of the endoscope distal end portion 35, and an endoscope that performs image processing on image signals from the imaging device. A processor 45 that is a mirror image processing apparatus is provided, and is connected to the endoscope 11 via connector portions 29A and 29B. Further, the display device 15 and the input unit 17 are connected to the processor 45. The processor 45 performs image processing on the imaging signal transmitted from the endoscope 11 based on an instruction from the operation unit 25 or the input unit 17 of the endoscope 11, generates a display image, and outputs the display image to the display device 15. Supply.

  The endoscope 11 includes an imaging unit 21 including an imaging device, an analog signal processing unit 49, an A / D conversion unit 51, a signal conversion unit 53, a drive unit 55, a motion detection unit 37, and an operation unit. A changeover switch 39 and an operation detection unit 47 are provided. The processor 45 connected to the endoscope 11 via the universal cable 27 includes a digital signal processing unit 57 and a control unit 59, to which the display device 15 and the input unit 17 are connected.

  The analog signal processing unit 49 performs predetermined analog signal processing on the imaging signal obtained by the imaging unit 21. The A / D conversion unit 51 converts the analog signal processed by the analog signal processing unit 49 into a digital signal. The output of the A / D converter 51 is converted to serial data by so-called RAW image data by the signal converter 53 through an encoding process, and is sent to the digital signal processor 57 of the processor 45. The RAW image data is digital image data that has been digitized in the form of the imaging signal from the imaging unit 21. The digital signal processing unit 57 decodes the serial data input from the signal conversion unit 53 and reproduces RAW image data before being encoded. The RAW image data is subjected to various processes and displayed on the display device 15 as a display image.

  FIG. 3 is a circuit diagram showing a configuration of a solid-state imaging device included in the imaging unit 21. The imaging device 61 includes a pixel unit 65 in which unit pixels 63 as light receiving units are arranged in a two-dimensional matrix, and a pixel unit 65. And a correlated double sampling (CDS) circuit 67 that performs processing such as noise suppression processing on the electric charge signal (signal based on the accumulated charge accumulated in the light receiving portion) and the vertical scanning of the pixel portion 65. A control signal is sent to the vertical scanning circuit 69 that controls the reset operation of the pixel unit 65, the horizontal scanning circuit 71 that controls scanning in the horizontal direction, the output circuit 73 that outputs the read charge signal, and the circuits 67 to 71. And a control circuit (scan driver) 75 for controlling timings for vertical and horizontal scanning and sampling.

  The unit pixel 63 includes one photodiode D1, a reset transistor M1, a drive (amplification) transistor M2, and a pixel selection transistor M3. Each unit pixel 63 is connected to a vertical scanning line (row selection line) L1 and a horizontal scanning line (column selection line) L2, and is sequentially scanned by a vertical scanning circuit 69 and a horizontal scanning circuit 71.

  The control circuit 75 controls the control signal input to the vertical scanning circuit 69 and the horizontal scanning circuit 71 to scan the rows and columns of the pixel portion 65 and the vertical scanning circuit 69 to reset the charges accumulated in the photodiode D1. A control signal to be input and a control signal to be input to the CDS circuit 67 in order to control connection between the pixel portion 65 and the CDS circuit 67 are generated.

  The CDS circuit 67 is provided for each column selection line L2, and the horizontal scanning circuit 71 outputs the charge signal of each unit pixel 63 connected to the row selection line L1 selected by the vertical scanning circuit 69. Sequentially output according to the horizontal scanning signal. The horizontal scanning circuit 71 controls on / off of the column selection transistor M4 provided between the CDS circuit 67 and the output bus line L3 connected to the output circuit 73 by a horizontal scanning signal. The output circuit 73 amplifies and outputs the charge signal sequentially transferred from the CDS circuit 67 to the output bus line L3. Hereinafter, a signal read from the photodiode D1 is referred to as a charge signal, and a signal output from the output circuit 73 is referred to as a captured image signal.

  Although illustration is omitted, the image pickup device 61 is a solid-state image pickup device of a single plate color image pickup system including a color filter (for example, a primary color filter having a Bayer array) composed of a plurality of color segments.

  The pixel unit 65 has a square array structure in which unit pixels 63 are arranged in a grid pattern in the row direction and the column direction on the surface of the semiconductor substrate when the vertical direction is the column direction and the horizontal direction is the row direction. Further, a so-called honeycomb pixel array configuration in which the odd-numbered unit pixels 63 and the even-numbered unit pixels 63 are arranged by being shifted by ½ pitch may be employed.

Next, a charge signal extraction method in the imaging unit 21 having the above configuration will be described with reference to FIGS.
The imaging element 61 of the imaging unit 21 can selectively set a normal scanning mode that is a first imaging mode and an interlaced scanning mode that is a second imaging mode, according to a command from the control unit 59. The normal scanning mode is a mode in which scanning readout of the lines of the unit pixels 63 arranged in the main scanning direction is sequentially performed in the sub scanning direction orthogonal to the main scanning direction to read out the charge signal (see FIG. 16). The interlaced scanning mode is a mode in which scanning / reading of the lines of the unit pixel 63 is repeatedly performed sequentially while skipping a predetermined line in the sub-scanning direction, and all lines are scanned horizontally to read out a charge signal.

  The control unit 59 is a detection signal of the motion detection unit 37 that detects the movement of the distal end portion 35 of the endoscope insertion unit 19, a detection signal of the operation detection unit 47 that detects that the angle knob 41 is operated, or a changeover switch. When a switching signal to the interlaced scanning mode is input from 39, a command signal is output to the drive unit 55, and the control circuit 75 of the imaging unit 21 is switched from the normal scanning mode to the interlaced scanning mode. Thus, an image is generated in the normal scanning mode when acquiring still image information where the subject is stationary, and in an interlaced scanning mode when acquiring moving image information where the subject moves.

Here, the interlaced scanning mode will be described in detail.
FIG. 4 is a diagram illustrating an example of reset, exposure, and readout timings of the image sensor using the rolling shutter in the one-line interlaced scanning mode. As shown in the figure, for example, a reset signal is output from the vertical scanning circuit 69 to the reset transistor M1 (see FIG. 3) of Line (1), and is stored in each photodiode D1 arranged in Line (1). The charged charge is reset and exposure is started. Thereafter, one line is skipped in the vertical direction (sub-scanning direction), Line (3) is similarly reset, and exposure is started. Further, one line is skipped in the vertical direction, Line (5) is reset, and exposure is started repeatedly.

  Then, when there is no jump destination, the process returns to Line (2) again, and after resetting Line (2) and starting exposure, the line (4) is reset by jumping one line in the vertical direction. The above reset processing is repeated until Line (n) (n is the total number of lines). As a result, the exposure is started from the state where the charge signal accumulated in the light receiving portion 63 of each line is once swept out.

  Then, after a predetermined exposure time, in order to output the charge signal accumulated in the light receiving unit 63 to the CDS circuit 67 (see FIG. 3), a selection signal to the pixel selection transistor M3 input through the row selection line L1, The charge signal of each light receiving portion 63 is changed from Line (1) → Line (3) →... Line (n−1) → Line (by the readout signal input from the column selection transistor M3 through the column selection line L2. 2) → Line (4) →... → Line (n) in this order and output to the output bus line L3 (see FIG. 3).

  As shown in FIG. 5A, the output order of the charge signals is as follows. When the captured image is, for example, the letter “A”, the scanning is performed in the main scanning direction from the image origin P0 as described above, and the sub scanning is performed. When scanning a line that skips one line in the direction is repeated, if the signal output to the output bus line L3 is converted into image data as it is, an image divided into two in the sub-scanning direction as shown in FIG. It becomes data.

  According to the above-described interlaced scanning mode, as shown in FIG. 6, each line is moved in the horizontal direction in the order of reading and the display of each line La on the screen is schematically shown in FIG. The amount of deformation is visually less than the screen shown in FIG. In the display content of FIG. 6, since adjacent lines have a relatively large time difference, discontinuous points occur for each interlaced line, and shaggy occurs at the edge of the screen. However, human visual characteristics are sensitive to the movement of the entire screen, but are insensitive to details, and this level of shaggy stimulation is small and difficult to recognize. For this reason, a subject that moves in the horizontal direction is observed in a state where the deformation in the horizontal direction is relaxed globally.

  Further, in this interlaced scanning mode, the feeling of screen fluctuation can be reduced when moving images are displayed. That is, as shown in FIG. 5B, each divided region (even line, odd line) has global information of the entire screen although the resolution is low, and is one screen than in the normal scanning mode. Reading can be speeded up equivalently. Thus, an image with less noticeable screen distortion can be easily obtained without increasing the signal transmission frequency (reading data rate: the number of read lines per unit time) to the endoscope system 100.

(Modification 1)
In this modification, an interlaced scanning mode that skips two lines is employed instead of the interlaced scanning mode that skips one line.
FIG. 7 is a diagram illustrating an example of reset, exposure, and readout timings of the image sensor using the rolling shutter in the 2-line interlaced scanning mode. As shown in FIG. 7, for example, a reset signal is output from the vertical scanning circuit 69 to the reset transistor M1 (see FIG. 3) of Line (1), and is stored in each photodiode D1 arranged in Line (1). The charged charge is reset and exposure is started. Thereafter, two lines are skipped in the vertical direction, Line (4) is similarly reset, and exposure is started. Further, the resetting of the line skipped by two lines in the vertical direction is repeated.

  Then, when there are no more jump destinations, the process returns to Line (2) again, Line (2) is reset and exposure is started. Then, two lines are skipped in the vertical direction to reset Line (5). This is repeated, and when there is no jump destination, the process returns to Line (3) again. After resetting Line (3) and starting exposure, the line (6) is reset by jumping two lines in the vertical direction. The above reset is repeated until Line (n) (n is the total number of lines). As a result, the exposure is started in a state where the charge signal accumulated in the light receiving portion 63 of each line is swept out.

  Next, after a predetermined exposure time, in order to output the charge signal accumulated in the light receiving unit 63 to the CDS circuit (see FIG. 3), the selection signal to the pixel selection transistor M3 input through the row selection line L1, and the column The charge signal of each light receiving portion 63 is changed from Line (1) → Line (4) →... → Line (2) → Line (5) by the readout signal input from the selection transistor M3 through the column selection line L2. → ... → Line (3) → Line (6) →... → Line (n) are output to the output bus line L3 (see FIG. 3).

  In this way, the signal output to the output bus line L3 by repeatedly scanning the image of FIG. 5A from the image origin P0 in the main scanning direction and scanning the line that has jumped two lines in the sub-scanning direction, When converted into image data as it is, the image data is divided into three in the sub-scanning direction as shown in FIG.

  As a result, as shown schematically in FIG. 9 on the screen, the amount of deformation is visibly smaller than the screen by one-line interlaced scanning shown in FIG. 6, and a moving image is displayed. The feeling of screen fluctuation can be reduced.

  Here, a synchronization process for interpolating a signal of a color component different from the color component for each color component read from each light receiving unit 63 in this modification will be described with reference to FIGS. explain. 10 is a schematic plan view showing a part of the light receiving units arranged in the Bayer array, FIG. 11 is a schematic plan view showing color components detected by the light receiving units in the two-line interlaced scanning mode, and FIG. It is a typical top view which shows the color component which each light-receiving part detected. This synchronization processing is not performed in the image sensor 61 but is performed by the digital signal processing unit 57 of the processor 45 which is an external control device.

  As shown in FIG. 10, the image sensor 61 (pixel unit 65) of this configuration example includes a light receiving unit 63G that detects a G (Green) color component, and a light receiving unit 63B that detects a B (Blue) color component. Are repeatedly arranged in the row direction, a light receiving unit 63R for detecting the R (Red) color component, and a light receiving unit 63G for detecting the G (Green) color component are repeatedly arranged in the row direction. The line thus formed is a so-called Bayer array in which the lines are alternately and repeatedly arranged in the column direction.

  When the image pickup device 61 arranged in the Bayer array is used to capture an image in the two-line interlaced scanning mode, each color component detected by each of the light receiving portions 63G, 63B, and 63R moves in the horizontal direction in the drawing in consideration of the exposure time difference (see FIG. In the example shown, if only one light receiving part is moved), the result is as shown in FIG. 11 in a pseudo manner, similar to the screen deformation shown in FIG. When these are arranged in the order of charge signal readout lines, they are arranged as shown in FIG. That is, the data block (Line (1), Line (4),...) Read in the first round shown in FIG. 12A and the data read in the second round shown in FIG. Each of the blocks (Line (2), Line (5),...) And the data blocks (Line (3), Line (6),...) Read in the third round shown in FIG. , Both are Bayer arrays.

  The synchronization processing is performed by interpolating the color component signals of one light receiving unit 63 with reference to information held by four unit light receiving units 63 in units of two lines as unit blocks 81. Is called. Specifically, as shown in FIG. 12A, the pixel information of the light receiving unit 63G to be interpolated has one R and B and two G in the unit block 81, so (R, G , B) = (R / k, (G + G) / (2k), B / k). Here, k is a coefficient for gradation conversion. For example, the coefficient k when converting 4096 gradation data to 256 gradation is 16 (4096/256).

  Next, the unit block 81 is shifted to the right by one pixel (one light receiving unit 63), and pixel information of the position of the next light receiving unit 63B to be interpolated is obtained in the same manner as described above. Hereinafter, the pixel information of each light receiving unit 63 is obtained while the unit block 81 is shifted rightward by one pixel. The charge signal of each light receiving unit 63 read out in the second and third rounds is processed in exactly the same manner with reference to the information held in each unit block 81 to obtain pixel information. Such data processing is processed by the digital signal processing unit 57 shown in FIG. 1, and captured image data is generated from the signals of the obtained three color components (R, G, B), and subjected to appropriate processing. This is displayed on the display device 15 later.

  Strictly speaking, since there is a time difference between the skipped lines, if this time difference is large, a synchronization process is performed based on imaging data different in time, and a color different from the actual is estimated. There is a fear. However, according to the interlace scanning mode of this configuration example, the time difference between the lines is short for the interlaced lines (for example, Line (1), Line (4), Line (7),...)). Therefore, temporal continuity is substantially ensured.

(Modification 2)
In this modified example, a two-line batch two-line interlaced scanning mode in which reading in units of two lines is skipped by two lines is employed as the interlaced scanning mode.
FIG. 13 is a diagram illustrating an example of reset, exposure, and readout timings of the image sensor using the rolling shutter in the 2-line batch 2-line interlace scanning mode. As shown in the figure, for example, reset signals are sequentially output from the vertical scanning circuit 69 to the reset transistor M1 (see FIG. 3) of Line (1) and Line (2), so that Line (1) and Line (2). The charge accumulated in the photodiodes D1 arranged in (1) is reset, and exposure is started. Thereafter, two lines are skipped in the vertical direction, and Line (5) and Line (6) are sequentially reset in the same manner to start exposure. Further, the reset is repeated by skipping two lines in the vertical direction.

  Then, when there are no more jump destinations, the process returns to Line (3) again, Line (3) and Line (4) are reset sequentially, and then the lines (7) and Line (8) are sequentially jumped by two lines in the vertical direction. Reset. The above reset is repeated until Line (n) (n is the total number of lines), and exposure is started in a state where the charge signal accumulated in the light receiving unit 63 of each line is swept out.

  Then, after a predetermined exposure time, the charge signal accumulated in the light receiving unit 63 is converted into Line (1), Line (2) → Line (5), Line (6) →... → Line (3), Line ( 4) Output to the output bus line L3 in the order of Line (7), Line (8) →... → Line (n−1), Line (n).

  When the captured image signal output to the output bus line L3 is repeatedly output as it is by repeating the output of two lines by two lines in this way, the image data is divided into two in the sub-scanning direction (not shown). The image at this time is as shown in FIG. 14 schematically showing the display on the screen, the amount of deformation of the screen is visually reduced, and the feeling of screen fluctuation is reduced when displaying moving images. it can.

  FIG. 15 is a schematic plan view showing the color components detected by the light receiving sections in the 2-line batch 2-line interlaced scanning mode, and the charge signals of the light receiving sections 63B, 63G, 63G, 63R read out in units of 2 lines. Is a Bayer array. Therefore, four adjacent light receiving portions 63B, 63G, 63G, and 63R are set as a unit block 81, and the information of the unit block 81 is referred to and interpolation is performed by interpolating the color component signals of one light receiving portion. Processing can be performed. Since the synchronization processing procedure is the same as the procedure described in the first modification, the description thereof will be omitted.

  The switching between the normal scanning mode and the interlaced scanning mode is performed when the motion detection unit 37 detects the movement (change speed exceeding a predetermined value) of the distal end portion 35 of the endoscope insertion unit 19 or when the operation detection unit 47 performs the switching. When the operation of the angle knob 41 is detected, the normal scanning mode is automatically switched to the interlace scanning mode. Thereby, the distal end portion 35 (imaging unit 21) of the endoscope 11 is in a normal scanning mode when acquiring still image information that is relatively stopped with respect to the subject, and the imaging unit 21 and the subject are relatively relative to each other. When acquiring moving image information, the image is captured in the interlaced scanning mode to reduce the feeling of screen fluctuation when displaying moving images.

  Alternatively, the surgeon operates the changeover switch 39 to switch the scanning mode, so that the normal scanning mode or the interlaced scanning mode optimum for imaging can be arbitrarily selected according to the situation. Further, a switching command may be input from the input unit 17 to the control unit 59 to switch between the normal scanning mode and the interlaced scanning mode.

  In addition to the above, the scan mode can be switched by, for example, estimating the motion of the observation target by comparing the odd and even lines of the captured image, and switching according to the presence or absence of the motion and the length of the movement amount. Good. In addition, when estimating the motion of the observation target, it is preferable to estimate using the adjacent line when the correlation between the odd-numbered line and the even-numbered line is strong, and using the pixels in the same line when the correlation is weak. . This makes it possible to always perform an accurate switching operation regardless of the magnitude of the movement. Alternatively, the movement of the observation target can be estimated and switched by comparing the current frame with the previous frame.

  As described above, the present invention is not limited to the above-described embodiments, and modifications and applications by those skilled in the art based on the description of the specification and well-known techniques are also within the scope of the present invention. It is included in the range to calculate.

As described above, the following items are disclosed in this specification.
(1) An endoscope system including: an endoscope including an imaging unit; and an external control device to which the endoscope is detachably connected and a signal of a captured image generated by the imaging unit is input. There,
It is orthogonal to the main scanning direction that the imaging unit reads out a plurality of light receiving units arranged two-dimensionally and a charge signal stored in each light receiving unit along a main scanning direction that is an arrangement direction of the light receiving units. A scanning drive unit that performs scanning and reading that is repeated a plurality of times in the sub-scanning direction.
In the first imaging mode, the scan driver sequentially scans and reads the lines of the light receiving units arranged in the main scanning direction in the sub-scanning direction, and reads the charge signals from all the light receiving units,
In the second imaging mode, scanning and reading of the lines of the light receiving unit are sequentially performed while skipping a predetermined line in the sub-scanning direction, and are sequentially performed while skipping the predetermined line from the skipped line. An endoscope system that horizontally scans all lines to read out the charge signal from the light receiving unit.
According to this endoscope system, as the imaging mode, the first imaging mode in which scanning and reading of the lines of the light receiving units arranged in the main scanning direction are sequentially performed in the sub scanning direction and the predetermined line in the sub scanning direction are sequentially skipped. Since it is possible to switch to the second imaging mode to be performed, the charge signal is read out in the first imaging mode when acquiring still image information, and the charge signal is read out in the second imaging mode when acquiring moving image information. For example, it is possible to suppress the distortion of the captured image that occurs due to the temporal shift in the imaging timing. Thereby, even if the endoscope insertion part is moved while gazing at the observation screen, an image without a sense of incongruity can be obtained. Further, unlike the prior art, since image distortion is not suppressed by estimating the amount of image distortion, there is no image formation due to an estimation error, and changes to the image (artifact) are minimized. Furthermore, since a mechanical shutter for ensuring simultaneity is not used, good moving picture information can be obtained only by changing the reading procedure of the light receiving unit without changing the optical system.

(2) The endoscope system according to (1),
In the second imaging mode, a first readout scanning for selecting the unscanned first line, performing horizontal scanning, and reading out the charge signal from each light receiving unit corresponding to the first line; A second scanning line that selects a second unscanned line that skips a predetermined line in the sub-scanning direction from the first line, performs horizontal scanning, and reads the charge signal from each light receiving unit corresponding to the second line; Is repeated to the lower end in the sub-scanning direction while maintaining the distance between the first line and the second line at the predetermined line, and then the unscanned line is selected from the upper end in the sub-scanning direction. An endoscope system which is a mode in which repeating one readout scan and the second readout scan to the lower end in the sub-scanning direction is executed a plurality of times.
According to this endoscope system, after the second imaging mode selects an unscanned line and performs horizontal scanning, it skips a predetermined line in the sub-scanning direction and selects an unscanned line and performs horizontal scanning. By repeatedly executing a plurality of times up to the lower end in the sub-scanning direction, one frame can be divided into a plurality of areas for reading, and screen reading can be equivalently accelerated. Thereby, even if the endoscope insertion portion is moved while gazing at the observation screen, an image with less noticeable screen distortion can be obtained.

(3) The endoscope system according to (1) or (2),
The plurality of light receiving portions arranged in a square lattice shape are covered with a color filter arranged in a Bayer array to detect light of different color components,
In the second imaging mode, the predetermined number of lines jumping in the sub-scanning direction is two lines,
The external control device reads out the charge signal read from each light receiving unit corresponding to two lines of a specific first line and a second line selected from the first line in the sub-scanning direction. Using synchronization means for performing a synchronization process for interpolating a signal of a color component different from the color component for each color component read from each light receiving unit,
An endoscope system that generates captured image data from a signal of a color component of a charge signal read from each light receiving unit and a color component obtained by the synchronization process in the second imaging mode.
According to this endoscope system, the second imaging mode skips two lines in the sub-scanning direction, scans and reads the lines of the light receiving unit, and the synchronization unit receives each of the two lines of light received continuously. Since the captured image data is generated by performing the synchronization process that interpolates the color components using the charge signals of the image capturing section, a color image in which the distortion of the moving image caused by the time lag of the imaging timing is suppressed can be obtained. It is done.

(4) An endoscope system including: an endoscope including an imaging unit; and an external control device to which the endoscope is detachably connected and a signal of a captured image generated by the imaging unit is input. There,
It is orthogonal to the main scanning direction that the imaging unit reads out a plurality of light receiving units arranged two-dimensionally and a charge signal stored in each light receiving unit along a main scanning direction that is an arrangement direction of the light receiving units. A scanning drive unit that performs scanning and reading that is repeated a plurality of times in the sub-scanning direction.
In the first imaging mode, the scan driver sequentially scans and reads the lines of the light receiving units arranged in the main scanning direction in the sub-scanning direction, and reads the charge signals from all the light receiving units,
In the second imaging mode, the scanning and reading of the lines of the light receiving unit are repeated by horizontally scanning the adjacent light receiving unit lines continuously for two lines and then skipping the next two lines. An endoscope system for reading out the charge signal from the rear.
According to this endoscope system, since the second imaging mode performs horizontal scanning of the charge signals of the respective two light receiving sections in succession and then scans and reads out the next two lines, the time of imaging timing is obtained. Therefore, it is possible to suppress the distortion of the captured image that occurs due to a general shift.

(5) The endoscope system according to (4),
The plurality of light receiving portions arranged in a square lattice shape are covered with a color filter arranged in a Bayer array to detect light of different color components,
Each light receiving unit corresponding to the two lines using the charge signal read from each light receiving unit corresponding to two lines of the specific line and the next line selected following the specific line by the external control device Synchronization means for performing a synchronization process for interpolating a signal of a color component different from the color component with respect to a charge signal that is a color component for each color read from
An endoscope system that generates captured image data from a color component of a charge signal read from each light receiving unit and a signal of a color component obtained by the synchronization processing in the second imaging mode.
According to this endoscope system, the synchronization unit performs the synchronization process of interpolating the color components using the read out charge signals of the light receiving units of the two continuous lines, and generates captured image data. As a result, a color image in which image distortion due to temporal shift in imaging timing is suppressed is obtained.

(6) The endoscope system according to any one of (1) to (5),
An endoscope system in which the external control device switches to the first imaging mode when acquiring still image information by the imaging unit, and switches to the second imaging mode when acquiring moving image information.
According to this endoscope system, since the external control device switches to the first imaging mode when acquiring still image information and to the second imaging mode when acquiring moving image information, there is no screen distortion. In addition to obtaining a still image, even in a moving image, distortion of a captured image caused by a temporal shift in image capturing timing is suppressed, and an image with no sense of incongruity is obtained.

(7) The endoscope system according to (6),
A motion detection unit that detects a motion of the imaging unit of the endoscope and outputs a detection signal to the external control device;
An endoscope system in which the external control device switches to the second imaging mode when a change speed obtained from detection information from a motion detection unit of the endoscope exceeds a predetermined value.
According to this endoscope system, when the motion detection unit detects the movement of the imaging unit of the endoscope, the mode is switched to the second imaging mode. For example, when inserting the endoscope insertion unit into the body cavity, The motion detection unit detects the motion of the imaging unit and automatically switches to the second imaging mode, and automatically suppresses the distortion of the moving image caused by the time lag in the imaging timing. Is obtained.

(8) The endoscope system according to (6),
The endoscope is formed on the distal end side of an endoscope insertion portion to be inserted into a subject, a bendable bending portion, a bending operation portion for bending the bending portion, and a bending operation portion from the bending operation portion. An operation detection unit for detecting an operation,
An endoscope system that switches to the second imaging mode when the external control device detects an operation of the bending operation unit by the operation detection unit.
According to this endoscope system, when the operation detection unit detects that the bending operation unit has been operated in order to bend the bending unit, the endoscope is switched to the second imaging mode. Secondly, when moving in the direction of the image, second, it is automatically switched to the imaging mode, and distortion of the moving image caused by the time lag of the imaging timing can be automatically suppressed, and the imaging unit A good image can be obtained even if the image is moving.

(9) The endoscope system according to (6),
The endoscope has an operation unit that switches between the first imaging mode and the second imaging mode,
An endoscope system that switches between the first imaging mode and the second imaging mode when the external control device detects an operation from an operation unit of the endoscope.
According to this endoscope system, when a switching operation to the first imaging mode or the second imaging mode is performed by the operation unit, the first imaging mode or the second imaging mode is performed according to the operation content. Therefore, readout scanning can be performed in an arbitrary mode optimal for imaging, and a clear image can always be obtained.

(10) The endoscope system according to any one of (1) to (9),
An endoscope system in which a signal line connecting the endoscope and the external control device serially transmits a signal of the captured image.
According to this endoscope system, the image signal is transmitted to the external control device by serial communication that transmits data with different weights in time series. Therefore, the higher the transmission speed and the longer the signal line, the more the influence of noise. It becomes easy to receive. Even in such a case, it is possible to suppress the occurrence of image distortion without increasing the readout speed of each line and without shortening the signal line, and a good image can be obtained even with a moving subject. .

11 Endoscope body 13 Control device (external control device)
19 Endoscope insertion part 21 Imaging part 27 Universal cable (signal line)
33 Bending part 37 Motion detection part 39 Changeover switch (operation part)
41 Angle knob (bending operation part)
47 operation detection unit 55 drive unit 57 digital signal processing unit (synchronization means)
63 Light receiver (unit pixel)
100 Endoscope system

Claims (10)

An endoscope including an imaging unit, an external control device to which the endoscope is detachably connected and a signal of a captured image generated by the imaging unit is input, and image information connected to the external control device An endoscope system including a display device for displaying ,
It is orthogonal to the main scanning direction that the imaging unit reads out a plurality of light receiving units arranged two-dimensionally and a charge signal stored in each light receiving unit along a main scanning direction that is an arrangement direction of the light receiving units. A scanning drive unit that performs scanning and reading that is repeated a plurality of times in the sub-scanning direction.
The scan driver, the main scanning reading of lines of the light receiving portions arranged in the scanning direction are sequentially performed respectively in the sub-scanning direction, a first imaging mode to read out the charge signals from all of the light receiving portion,
The scan reading of the line of the light receiving portion, sequentially performs while interlaced predetermined line in the sub-scanning direction, by sequentially performed while jumping from skipped a line predetermined line, all lines constituting the captured image and the horizontal scanning A second imaging mode for reading out the charge signal from the light receiving unit ;
Is switchable,
When the external control device acquires still image information by the imaging unit, it switches to the first imaging mode, and when it acquires moving image information, it switches to the second imaging mode and is displayed on the display device. An endoscope system for generating shaggy at a screen end of the captured image. The endoscope system according to claim 1,
A motion detection unit that detects a motion of the imaging unit of the endoscope and outputs a detection signal to the external control device;
An endoscope system in which the external control device switches to the second imaging mode when a change speed obtained from detection information from a motion detection unit of the endoscope exceeds a predetermined value. The endoscope system according to claim 1,
The endoscope is formed on the distal end side of an endoscope insertion portion to be inserted into a subject, a bendable bending portion, a bending operation portion for bending the bending portion, and a bending operation portion from the bending operation portion. An operation detection unit for detecting an operation,
An endoscope system that switches to the second imaging mode when the external control device detects an operation of the bending operation unit by the operation detection unit. The endoscope system according to any one of claims 1 to 3,
In the second imaging mode, a first readout scanning for selecting the unscanned first line, performing horizontal scanning, and reading out the charge signal from each light receiving unit corresponding to the first line; A second scanning line that selects a second unscanned line that skips a predetermined line in the sub-scanning direction from the first line, performs horizontal scanning, and reads the charge signal from each light receiving unit corresponding to the second line; Is repeated to the lower end in the sub-scanning direction while maintaining the distance between the first line and the second line at the predetermined line, and then the unscanned line is selected from the upper end in the sub-scanning direction. An endoscope system which is a mode in which repeating one readout scan and the second readout scan to the lower end in the sub-scanning direction is executed a plurality of times. The endoscope system according to any one of claims 1 to 4,
The plurality of light receiving portions arranged in a square lattice shape are covered with a color filter arranged in a Bayer array to detect light of different color components,
In the second imaging mode, the predetermined number of lines jumping in the sub-scanning direction is two lines,
The external control device reads out the charge signal read from each light receiving unit corresponding to two lines of a specific first line and a second line selected from the first line in the sub-scanning direction. Using synchronization means for performing a synchronization process for interpolating a signal of a color component different from the color component for each color component read from each light receiving unit,
An endoscope system that generates captured image data from a signal of a color component of a charge signal read from each light receiving unit and a color component obtained by the synchronization process in the second imaging mode. An endoscope including an imaging unit, an external control device to which the endoscope is detachably connected and a signal of a captured image generated by the imaging unit is input, and image information connected to the external control device An endoscope system including a display device for displaying,
It is orthogonal to the main scanning direction that the imaging unit reads out a plurality of light receiving units arranged two-dimensionally and a charge signal stored in each light receiving unit along a main scanning direction that is an arrangement direction of the light receiving units. A scanning drive unit that performs scanning and reading that is repeated a plurality of times in the sub-scanning direction.
The scan driving unit sequentially scans and reads the lines of the light receiving units arranged in the main scanning direction in the sub-scanning direction, and reads out the charge signals from all the light receiving units;
Scanning and reading the lines of the light receiving unit is performed by repeating the horizontal scanning of two adjacent light receiving unit lines and then jumping over the next two lines to read the charge signals from all the light receiving units. A second imaging mode;
Is switchable,
When the external control device acquires still image information by the imaging unit, it switches to the first imaging mode, and when it acquires moving image information, it switches to the second imaging mode and is displayed on the display device. An endoscope system for generating shaggy at a screen end of the captured image. The endoscope system according to claim 6, wherein
A motion detection unit that detects a motion of the imaging unit of the endoscope and outputs a detection signal to the external control device;
An endoscope system in which the external control device switches to the second imaging mode when a change speed obtained from detection information from a motion detection unit of the endoscope exceeds a predetermined value. The endoscope system according to claim 6, wherein
The endoscope is formed on the distal end side of an endoscope insertion portion to be inserted into a subject, a bendable bending portion, a bending operation portion for bending the bending portion, and a bending operation portion from the bending operation portion. An operation detection unit for detecting an operation,
An endoscope system that switches to the second imaging mode when the external control device detects an operation of the bending operation unit by the operation detection unit. The endoscope system according to claim 6, wherein
The plurality of light receiving portions arranged in a square lattice shape are covered with a color filter arranged in a Bayer array to detect light of different color components,
Each light receiving unit corresponding to the two lines using the charge signal read from each light receiving unit corresponding to two lines of the specific line and the next line selected following the specific line by the external control device Synchronization means for performing a synchronization process for interpolating a signal of a color component different from the color component with respect to a charge signal that is a color component for each color read from
An endoscope system that generates captured image data from a color component of a charge signal read from each light receiving unit and a signal of a color component obtained by the synchronization processing in the second imaging mode. The endoscope system according to any one of claims 1 to 9,
An endoscope system in which a signal line connecting the endoscope and the external control device serially transmits a signal of the captured image.

Images