JP2007166351A - Imaging device and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To eliminate a concern of producing a coarse image in right and left two-image display, when gain control is independently performed right and left in digital processing, and when the gain of the image on the dark side is enhanced. <P>SOLUTION: In a wide-angle distortion correction camera system 10, a CMOS sensor 11 is scanned to select each pixel of matrix-shaped disposition on a column-by-column basis, and driven to output each pixel signal in the selected column through a signal line 114. In signal processing, the imaging range of the CMOS sensor 11 is divided into two regions in the column selection direction, and for each of the two divided regions, luminance information is detected by a camera signal processing circuit 13. Based on the above detection result, the signal level in each region is controlled for each of the two divided regions. Thus, each brightness of the display screen based on the imaging result of each divided region is adjusted to an optimal brightness. Further, in a vertical/horizontal conversion & distortion correction circuit 14, a vertical scanning signal is converted into a horizontal scanning signal. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method, and more particularly to an imaging apparatus such as a camera system and an imaging method thereof.

  In an imaging apparatus such as a camera system, for example, an in-vehicle camera system such as a blind corner camera system, image light incident through an optical system including a reflecting mirror, a prism lens, or a fisheye lens is received by one imaging element, and the imaging is performed. A captured image by the element is divided into two in the left-right direction by camera signal processing, and two images obtained by the left-right two-division are displayed on one display.

  In this way, in a camera system that performs two-divided image display, when the difference in the luminance level of the entire screen in the left and right regions becomes large, it must be matched to the luminance level of either the left or right screen. In this case, the brightness of the other screen is saturated and it becomes difficult to see the image, or conversely, the image becomes dark due to insufficient brightness.

  Therefore, conventionally, in the digital signal processing circuit at the latter stage of the image sensor, gain difference between the left and right two screens is adjusted by digitally controlling the gain of the left and right two screen images ( For example, see Patent Document 1).

JP 2002-330338 A

  However, in the above-described prior art in which gain control is performed by digital processing independently on the left and right, there is a concern that a coarse image may be obtained when gain on a dark side is increased due to a problem of digital quantization accuracy.

  Therefore, the present invention does not perform gain control by digital processing, but by performing gain control independently on the left and right by analog processing, an imaging device and an imaging capable of adjusting the brightness of the left and right two screens to optimum brightness, respectively. It aims to provide a method.

  In order to achieve the above object, in the present invention, in an imaging apparatus using a solid-state imaging device in which pixels including photoelectric conversion elements are two-dimensionally arranged in a matrix, each pixel arranged in a matrix with respect to the solid-state imaging device Are sequentially selected for each column, while driving to output a signal of each pixel of the selected column through a signal line wired for each row, while brightness information is obtained for each of a plurality of regions in the column selection direction of the solid-state imaging device. And detecting the signal level of each of the plurality of regions based on the detection result, and setting the scanning direction of the image signal output from the solid-state imaging device to a scanning direction corresponding to a standard television signal. The structure to convert is taken.

  In the imaging apparatus having the above-described configuration, the solid-state imaging device is driven to output the signals of the pixels in the selected column through the signal lines wired for the rows while sequentially selecting the pixels arranged in a matrix for each column. Thus, in signal processing, the imaging region of the solid-state imaging device can be divided into a plurality of regions in the column selection direction (that is, the left-right direction). Then, by controlling the signal level of each area for each of the plurality of areas, the brightness of the display screen based on the imaging result of each area can be adjusted to an optimum brightness. The image signal output from the solid-state image sensor is converted into a scan direction corresponding to a standard television signal so that a captured image obtained by scanning in the vertical direction in the solid-state image sensor can be converted into a normal display device. Can be displayed.

  According to the present invention, since the gain control of each region can be performed by analog processing in the solid-state imaging device, the brightness of the left and right two screens can be optimized without causing problems as in the case of performing digital processing. The brightness can be adjusted.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing an outline of a configuration of an imaging apparatus according to an embodiment of the present invention. Here, taking the case of a wide-angle distortion correction camera system as an example of the imaging apparatus, the left and right two-division zoom function in the wide-angle distortion correction camera system will be described. Here, the left and right divided zoom means that only the left and right edges of an image captured by an image sensor equipped with a fisheye lens in one camera are zoom-displayed by signal processing.

  As shown in FIG. 1, a wide-angle distortion correction camera system 10 according to the present embodiment includes an image sensor 11, a sensor driving circuit 12, a camera signal processing circuit 13, a vertical / horizontal conversion & distortion correction circuit 14, an NTSC conversion circuit 15, and a display. The apparatus 16 has a configuration.

  In this wide-angle distortion correction camera system 10, by mounting the fisheye lens 17, it is possible to capture an image in the direction of 180 degrees in the vertical and horizontal directions with one image sensor 11. Then, by cutting out an image in the left-right direction from the captured image and correcting the distortion, it is possible to realize the same function as a vehicle-mounted camera system, for example, a blind corner camera system.

  The image pickup device 11 is an XY address type solid-state image pickup device, for example, a CMOS sensor, which can read signals independently for each pixel and can perform shutter control independent for each line and each pixel called a so-called random shutter. (Hereinafter referred to as “CMOS sensor 11”). The sensor drive circuit 12 includes a timing generator that generates various timing signals, and drives the CMOS sensor 11 based on the various timing signals generated by the timing generator.

(CMOS sensor)
FIG. 2 is a schematic configuration diagram illustrating an example of the configuration of the CMOS sensor 11. In FIG. 2, pixels 111 including photoelectric conversion elements are two-dimensionally arranged in a matrix to form a pixel array unit 112. Although the pixel 111 includes one or a plurality of transistors in addition to the photoelectric conversion element, the circuit configuration of the pixel 111 is not limited here. Therefore, detailed description of the pixel circuit is omitted.

  With respect to the matrix-like pixel arrangement of the pixel array unit 112, one or a plurality of pixel drive lines 113 are wired for each column and connected to each pixel 111, and a row signal line 114 is wired for each row. It is connected to each pixel 111. One end of each pixel drive line 113 is connected to an output end corresponding to each column of the column selection circuit 115.

  The column selection circuit 115 is configured by a shift register, an address decoder, and the like, and selectively scans each pixel 111 of the pixel array unit 112 in order in a column unit via the pixel drive line 113. That is, the column selection circuit 115 selects each pixel 111 in units of columns by scanning the pixel array unit 112 in the vertical direction.

  Although not shown here, the column selection circuit 115 sequentially selects the pixels 111 in units of columns and performs a read operation for reading a signal of each pixel 111 in the selected column, and the read scan. Electronic shutter scanning system for performing an electronic shutter operation that discards (resets) charges accumulated so far in the photoelectric conversion elements of the pixels 111 in the same column by a time corresponding to the shutter speed before reading scanning by the system It has composition which has.

  The signal charge accumulation time (exposure time) in the pixel 111 is determined by the number of columns of readout scanning intervals by the readout scanning system. However, when the unnecessary charge of the photoelectric conversion element is reset by the shutter scanning by the electronic shutter scanning system after the reading scanning by the reading scanning system, the signal of the pixel 111 is detected by the next reading scanning by the reading scanning system from that timing. The period until the timing of reading out is the signal charge accumulation time in the pixel 111. That is, the electronic shutter operation is an operation that resets signal charges accumulated in the photoelectric conversion elements and newly starts accumulation of signal charges.

  By selecting each column in order by the column selection circuit 115, a signal is output from each pixel of the selected column to the row signal line 114. This signal is supplied to the row signal processing circuit 116 via the row signal line 114. The row signal processing circuit 116 receives a signal supplied from the pixel 111 of each column through the row signal line 114 for each pixel row, and removes a fixed pattern noise peculiar to the pixel with respect to the signal. Signal processing such as Sampling (correlated double sampling) is performed, and the processed signal is temporarily held.

  A column selection switch 118 is connected between the output terminal of each row of the row signal processing circuit 116 and the signal output line 117. The row selection circuit 119 includes a shift register, an address decoder, and the like, and selectively drives the column selection switch 118. When the column selection switch 118 is turned on (closed) by the selection driving by the row selection circuit 119, a signal temporarily held in the row unit in the row signal processing circuit 116 is output as a signal via the column selection switch 118. The signal is output to the line 117 and further output to the outside of the sensor via the output amplifier 120.

  By the way, in general, a solid-state imaging device represented by a CMOS sensor is a column signal line in which signals of each pixel in a selected row are wired for each column while sequentially selecting each pixel 111 in a matrix arrangement in a row unit. It is the structure which outputs via. On the other hand, the CMOS sensor 11 according to this example, as is apparent from the description of the configuration described above, performs the signal selection of each pixel in the selected column while scanning and selecting each pixel 11 in the matrix arrangement in order. A configuration is employed in which output is performed via row signal lines 114 wired for each.

  By adopting such a configuration, the CMOS sensor 11 according to the present example divides the imaging region that is the region of the pixel array unit 112 into a plurality of regions in the column selection direction (horizontal direction / left-right direction) in signal processing. Is possible. Here, since the wide-angle distortion correction camera system 10 has a left-right two-division zoom function, the imaging region is assumed to be divided into left and right.

  As described above, in the case where the imaging region of the CMOS sensor 11 is divided into left and right parts, in the present embodiment, for example, the exposure time can be set for each divided region by controlling the shutter speed of the electronic shutter for each divided region. It is characterized by doing. Details of the control of the shutter speed will be described later.

(Camera signal processing circuit)
FIG. 3 is a block diagram illustrating an example of the configuration of the camera signal processing circuit 13. As shown in FIG. 3, the camera signal processing circuit 13 includes an A / D conversion unit 131, a signal processing unit 132, luminance detection units 133 and 134, and a microcomputer (microcomputer) 135, and is output from the CMOS sensor 11. The analog signal is converted into a digital signal by the A / D conversion unit 131, and the signal processing is performed digitally.

  The signal processing unit 132 performs signal processing such as primary color separation, white balance, and gamma correction on the imaging signal from the CMOS sensor 11. The luminance detection units 133 and 134 are provided corresponding to the divided regions divided into left and right parts, and by integrating (detecting) the luminance signal component for a certain period for each divided region, the luminance for each divided region. Luminance information (brightness information) corresponding to (brightness) is obtained and provided to the microcomputer 135.

(Shutter speed setting)
The microcomputer 135 corresponds to the control means in the claims, and controls the entire system including exposure control for each divided area of the CMOS sensor 11. In the exposure control, control for setting a shutter speed control amount that determines the shutter speed at the time of the electronic shutter operation is performed on the timing generator in the sensor drive circuit 12 according to the luminance information for each divided area given from the luminance detectors 133 and 134. Against. Then, the timing generator sets a different shutter speed for each of the two divided areas by driving and controlling the electronic shutter scanning system in the column selection circuit 115 at a timing according to the control of the microcomputer 135.

  FIG. 4 is a conceptual diagram when different shutter speeds are set for each divided region. Here, the number of scanning lines (columns) selectively scanned by the column selection circuit 115 is N. Then, scanning is started sequentially for each column from the left side of the imaging region at a shutter speed determined by the shutter speed setting value 1, and when the number of scanning lines becomes N / 2, the shutter speed is determined by the shutter speed setting value 2.

  In this way, by switching the shutter speed for each of the two divided areas, the exposure time (signal charge accumulation time) can be made different for each divided area. When the exposure time is different for each divided area, the signal level of the pixel 111, that is, the luminance level is different for each divided area. That is, when the shutter speed increases, the luminance level increases because the amount of light received by the pixel 111 increases, and conversely, when the shutter speed decreases, the luminance level decreases because the amount of light received by the pixel 111 decreases.

  Therefore, under the control of the microcomputer 135, the brightness of the left and right two screens is controlled by controlling the shutter speed for each of the two divided areas based on the luminance information for each divided area detected by the luminance detectors 133 and 134. Each can be adjusted to the optimum brightness. Thereby, in the left and right two-screen display, the right and left two images can be displayed without any sense of incongruity.

(Vertical horizontal conversion & distortion correction circuit)
The vertical / horizontal conversion & distortion correction circuit 14 corresponds to a standard television signal for a video signal that has passed through the camera signal processing circuit 13, that is, a video signal obtained by scanning in the vertical direction by the column selection circuit 115 in the CMOS sensor 11. The video signal obtained by the scanning direction, that is, the horizontal scanning is converted, and at the same time, the distortion correction process and the left / right divided zoom process (resolution conversion process) are performed.

  The distortion correction process and the left and right divided zoom process can be realized by using a known method. Here, description of the vertical / horizontal conversion & distortion correction circuit 14 including distortion correction and zoom processing is omitted, and a vertical / horizontal conversion circuit 14A that performs vertical / horizontal conversion only by simple writing / reading control of the frame memory will be described.

  FIG. 5 is a block diagram showing an example of the configuration of the vertical / horizontal conversion circuit 14A. As shown in FIG. 5, the vertical / horizontal conversion circuit 14 </ b> A according to this example has a configuration including a timing controller 141, a write address generation unit 142, a read address generation unit 143, and two frame memories 144 and 145.

  The write address generation unit 142 generates a write address under the control of the timing controller 141, and sequentially writes image data obtained by scanning in the vertical direction to the frame memory 144/145 based on the write address. Further, the read address generation unit 143 generates a read address under the control of the timing controller 141, and sequentially outputs image data obtained by scanning in the horizontal direction from the frame memory 144/145 in accordance with the read address.

  The timing controller 141 controls the two frame memories 144 and 145 for each frame, alternately controls writing / reading to / from the frame memories 144 and 145, and starts from the frame memory 144/145 on which no image data is written. Image data obtained by scanning in the horizontal direction is read out.

  The vertical / horizontal conversion processing in the vertical / horizontal conversion circuit 14A will be described with reference to FIG. Here, an image of horizontal M pixels × vertical N pixels is considered as an input image, and an image of horizontal K pixels × vertical L pixels (K <M, L <N) is considered as an output image.

  First, M lines (M columns) of data are written in a line-sequential manner to the frame memory 144/145 in the vertical direction, and data for M × N pixels are stored in the frame memory 144/145. The written image data is read out in the next frame. In that case, the reading direction of the image data from the frame memory 144/145 is changed from the vertical direction at the time of writing to the horizontal direction, the data for L lines is read line by line, and the output image for K × L pixels is read. obtain.

  Here, as an example, assuming that the input stage (CMOS sensor 11) is 1.3 million pixels, the effective pixel area is generally 1280 × 960. When the graphics display standard of the output stage (display device 16) is considered to be VGA, it is 640 × 480.

  In this example, only simple image clipping is considered, but it is possible to simultaneously perform enlargement / reduction processing by performing resolution conversion processing when performing vertical / horizontal conversion. Is the same.

  FIG. 7 shows an image conversion flow. In FIG. 7, the images (A) to (D) are output images (A) to (D) of the CMOS sensor 11, the camera signal processing circuit 13, the vertical / horizontal conversion & distortion correction circuit 14 and the NTSC conversion circuit 15 in FIG. ) Respectively. As indicated by solid arrows in FIG. 7, the scanning directions of the images (A) and (B) of the CMOS sensor 11 and the camera signal processing circuit 13 are vertical directions, and the vertical horizontal conversion & distortion correction circuit 14 and NTSC conversion circuit. The scanning direction of each of the 15 images (C) and (D) is the horizontal direction.

  Returning to FIG. The NTSC conversion circuit 15 converts the video signal that has been subjected to the vertical / horizontal conversion processing, distortion correction processing, and left / right split zoom processing by the vertical / horizontal conversion & distortion correction circuit 14 into an NTSC television signal for display. Supply to device 16. The display device 16 is, for example, a liquid crystal display, and zooms and displays a left and right divided image obtained by dividing an image captured by one CMOS sensor 11 based on a television signal supplied from the NTSC conversion circuit 15.

  As described above, in the wide-angle distortion correction camera system 10, the signals of the pixels 111 in the selected column are output through the row signal line 114 while sequentially selecting the pixels 111 arranged in a matrix with respect to the CMOS sensor 11 for each column. In the signal processing, the imaging region (region of the pixel array unit 112) of the CMOS sensor 11 is divided into a plurality of regions (in this example, two in the horizontal direction / left-right direction) in signal processing. Can be divided into (regions).

  Then, the luminance information is detected by the two luminance detectors 133 and 134 for each of the two divided areas, and the signal level of each area is controlled for each of the two divided areas based on the detection result. The brightness of the display screen based on the imaging result can be adjusted to an optimum brightness. That is, gain control of each divided region can be performed by analog processing in the CMOS sensor 11. Therefore, the brightness of the left and right two screens can be adjusted to the optimum brightness without causing problems as in the case of the prior art performed by digital processing.

  The image signal output from the CMOS sensor 11 is converted into a video signal obtained by scanning in the scanning direction corresponding to a standard television signal, that is, in the horizontal direction, by the vertical / horizontal conversion & distortion correction circuit 14 in the scanning direction. By performing the distortion correction process and the left and right divided zoom process, the captured image obtained by the vertical scanning in the CMOS sensor 11 can be displayed on the normal display device 16. As a result, it is possible to display a two-screen image having a sense of incongruity in which the brightness of the left and right two images is uniform.

  In the above embodiment, the case where the present invention is applied to the wide-angle distortion correction camera system 10 using the fisheye lens 17 has been described as an example. However, the present invention is not limited to this application example, and the reflection is performed instead of the fisheye lens 17. The present invention can be similarly applied to a camera system using a mirror or a prism lens.

  In the above embodiment, the imaging region of the CMOS sensor 11, that is, the region of the pixel array unit 112 is divided into two in the column selection direction in signal processing, and the signal level of each region is controlled for each divided region. The number of divisions is not limited to two.

1 is a block diagram illustrating an outline of a configuration of an imaging apparatus according to an embodiment of the present invention. It is a schematic block diagram which shows an example of a structure of a CMOS sensor. It is a block diagram which shows an example of a structure of a camera signal processing circuit. It is a conceptual diagram in the case of setting a different shutter speed for each divided region. It is a block diagram which shows an example of a structure of a vertical / horizontal conversion circuit. It is explanatory drawing of a vertical / horizontal conversion process. It is a figure which shows an image conversion flow.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Wide-angle distortion correction camera system, 11 ... CMOS sensor (imaging element), 12 ... Sensor drive circuit, 13 ... Camera signal processing circuit, 14 ... Vertical horizontal conversion & distortion correction circuit, 14A ... Vertical horizontal conversion circuit, 15 ... NTSC Conversion circuit, 16 ... display device, 17 ... fisheye lens, 131 ... A / D conversion unit, 132 ... signal processing unit, 133, 134 ... luminance detection unit, 135 ... microcomputer, 141 ... timing controller, 142 ... write address generation unit, 143: Read address generation unit, 144, 145 ... Frame memory

Claims (4)

The pixels including the photoelectric conversion elements are two-dimensionally arranged in a matrix, and each pixel in the matrix is sequentially scanned and selected for each column, and the signal of each pixel in the selected column is passed through a signal line wired for each row. An output solid-state imaging device;
A plurality of detection means for detecting luminance information for each of a plurality of regions in a column selection direction of the solid-state imaging device;
Control means for controlling the signal level of each area for each of the plurality of areas based on the detection results of the plurality of detecting means;
An imaging apparatus comprising: conversion means for converting a scanning direction of an image signal output from the solid-state imaging device into a scanning direction corresponding to a standard television signal. The imaging apparatus according to claim 1, wherein the control unit controls a shutter speed for each of the plurality of regions. The solid-state imaging device has an output amplifier that amplifies a signal output from each pixel through the signal line,
The imaging apparatus according to claim 1, wherein the control unit controls a gain of the output amplifier for each of the plurality of regions. An imaging method for an imaging apparatus using a solid-state imaging device in which pixels including photoelectric conversion elements are two-dimensionally arranged in a matrix,
The solid-state image sensor is driven to output a signal of each pixel in a selected column through a signal line wired for each row while sequentially selecting each pixel in a matrix arrangement for each column.
Luminance information is detected for each of a plurality of regions in the column selection direction of the solid-state imaging device, and the signal level of each region is controlled for each of the plurality of regions based on the detection result, and is output from the solid-state imaging device. An imaging method comprising: converting a scanning direction of an image signal into a scanning direction corresponding to a standard television signal.

Images