JP4114633B2 - Image processing apparatus and method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method for processing a subject image picked up by an image pickup device, and in particular, to improve luminance unevenness at an image end caused by an image pickup optical system.

  There is known an image processing apparatus that improves the sensitivity of the entire image even if the resolution is sacrificed by adding the outputs of adjacent pixels (see, for example, Patent Document 1).

Prior art documents related to the invention of this application include the following.
JP 09-182090 A

  By the way, the optical system used for imaging has various sizes from a large aperture to a small aperture, and a zoom lens (a variable magnification optical system) may be used. When an optical system such as a small-aperture lens or a zoom lens is used, the sensitivity at the edge of the image may decrease due to “vignetting” at the edge of the optical system, resulting in uneven brightness.

  When the above-described conventional sensitivity improvement technology is applied to an image processing apparatus that processes an image captured by such a small-aperture lens or a zoom lens, the conventional technology increases the sensitivity of the entire image. However, there is a problem that the sensitivity at the center of the image is also increased, and the luminance unevenness of the image is still not solved.

  An image processing apparatus that forms a subject image on a light-receiving surface of an image pickup device in which a plurality of pixels are developed on a two-dimensional plane through an optical system and processes the subject image picked up by the image pickup device. An area (addition area) for performing addition processing on the subject image is set according to the characteristics, and pixels existing around the pixel in the output of one pixel for each pixel of the image sensor in the addition area on the subject image Is added to generate a new subject image.

  According to the present invention, it is possible to generate a bright image without luminance unevenness in the entire subject image.

<< First Embodiment of the Invention >>
FIG. 1 shows the configuration of the first embodiment. The optical system 1 is an optical system including a zoom lens (magnification changing optical system) that can change the magnification according to a magnification setting signal and a lens holder, and forms a subject image on the light receiving surface of the image sensor 2. The image pickup device 2 is a semiconductor device having a CMOS structure including a plurality of light receiving elements (hereinafter also referred to as pixels) developed on a two-dimensional plane and a read circuit for reading output voltages of the respective light receiving elements. The output voltage of the light receiving element specified by the signal is output.

  The adder 3 adds the output signals of the light receiving elements designated by the read address signal of the addition pattern generation device 6 until the addition reset signal is output from the addition pattern generation device 6. That is, the adder 3 adds the output of the light receiving element existing in the vicinity of the light receiving element to the output of one light receiving element of the imaging element 2 to generate a new image.

  In addition, in the addition process of the adder 3, not only the addition but also the weighting may be performed for each light receiving element. In this case, the weighting element may be output from the addition pattern generation device 6, or the weighting element may be automatically generated by the adder 3 when the weighting element is fixed.

  The AD converter 4 performs analog-digital conversion to digitally process the output of the adder 3. A known circuit may be used for the AD converter 4.

  The processing device 5 performs detection of a vehicle existing ahead, detection of road conditions, and the like based on the subject image signal after digital conversion. As this detection method, a conventionally known method may be used. The processing device 5 also outputs the type of processing executed in the image processing device to the addition pattern generation device 6. For example, in order to detect a white line drawn on a road, vertical edge detection is performed by performing vertical Sobel filtering on 3 × 3 pixel data, a vertical edge detection signal is output to the addition pattern generation device 6. . The processing device 5 further outputs a signal for setting the magnification of the image to the magnification setting device 8 in order to capture an area necessary for processing with an optimum size.

  The addition pattern generation device 6 inputs the type of processing to be executed in the image processing device from the processing device 5 and generates a combination pattern (hereinafter referred to as addition pattern) of addition pixels according to the type of processing, and also on the subject image. An area for performing addition processing by the adder 3 (hereinafter referred to as an addition area) is input from the addition area setting device 7 and a read address signal for designating a light receiving element corresponding to the addition pattern in the addition area is input to the imaging element. Output to 2. The addition pattern generation device 6 also outputs an addition reset signal for initializing the adder 3 to the adder 3.

  Here, the combination pattern (addition pattern) of the addition pixels corresponding to the type of processing is, for example, a pattern in which three light receiving element outputs continuous in the horizontal direction are added, or four light receiving elements arranged vertically every other line. A vertical or horizontal pattern such as a pattern in which outputs are added is obtained so that an optimum result is obtained for the processing purpose.

Further, the addition pattern may be developed on a two-dimensional plane. In the image pickup device 2 shown in FIG. 2 in which a plurality of light receiving elements (pixels) are developed on a two-dimensional plane, addition is performed by adding the outputs of four adjacent pixels and using the added value as it is as the output of the original pixel. It is good also as a pattern. For example, in a pattern in which the outputs of four adjacent pixels (X11, X12, X21, X22) are added, the outputs of the four pixels X11, X12, X21, X22 are x11, x12, x21, x22, respectively. The output of the four pixels after the addition process is
x11 = x11 + x12 + x21 + x22,
x12 = x11 + x12 + x21 + x22,
x21 = x11 + x12 + x21 + x22,
x22 = x11 + x12 + x21 + x22 (1)
It becomes. In this addition pattern, the outputs of the four adjacent pixels (X13, X14, X23, X24) are then added.

As another example of developing the addition pattern on a two-dimensional plane, an addition pattern in which outputs of four pixels adjacent to an arbitrary pixel are added to obtain the output of the pixel can be considered. For example, taking the pixel X11 as an example, the outputs of the pixels X12, X21, and X22 existing around the pixel X11 are added to the output of the pixel X11.
x11 = x11 + x12 + x21 + x22 (2)
Similarly, for the pixels X12, X21, X22,.
x12 = x12 + x13 + x22 + x23,
x21 = x21 + x22 + x31 + x32,
x22 = x22 + x23 + x32 + x33,
... (3)

  The addition pattern developed on the two-dimensional plane is not limited to the above-described “□” (rectangular) type. be able to.

  The addition area setting device 7 sets an area (addition area) for performing the addition processing by the adder 3 on the subject image in accordance with the magnification setting signal from the magnification setting apparatus 8, and sets the addition area setting to the addition pattern generation device 6. Output a signal. For example, as shown in FIG. 3A, when photographing is performed with a small-aperture lens or a zoom lens, the sensitivity of the four corners of the image is reduced due to “vignetting” at the end of the lens, and luminance unevenness occurs. In particular, with a zoom lens, when wide angle shooting is performed with a reduced magnification, the four corners of an image become dark and uneven brightness occurs. Therefore, a bright and high-contrast image can be obtained by performing the above-described addition processing in a region where luminance unevenness occurs at the four corners of the image. In a zoom lens, as the magnification is reduced to a wider angle, the area of occurrence of uneven brightness at the four corners of the image becomes wider. No bright image can be obtained.

  The addition area may be enlarged according to the magnification of the optical system 1 and may be optimized by changing the addition pattern and / or the number of addition pixels according to the magnification of the optical system 1. In addition, in the case of performing addition processing in which weight is added for each pixel output, optimization may be performed by changing the weight for each pixel according to the magnification of the optical system 1.

  The magnification setting device 8 sets the magnification of the image according to the processing of the processing device 5 in order to capture an area necessary for the processing executed by the processing device 5 with an optimal size, and sends the magnification setting signal to the optical system 1. And output to the addition area setting device 7.

  According to this embodiment, an addition area corresponding to the magnification of the zoom lens is set for the original image having uneven brightness at the four corners captured by the optical system 1 of the zoom lens shown in FIG. Since the addition process is performed, luminance unevenness at the four corners of the image is eliminated as shown in FIG. 3B, and a bright and high-contrast image is obtained.

<< Second Embodiment of the Invention >>
In the first embodiment described above, an example in which the addition area is set according to the set magnification of the zoom lens (variable magnification optical system) has been described, but the second area where the addition area is set according to the aperture amount of the optical system. An embodiment will be described.

  FIG. 4 shows the configuration of the second embodiment. In addition, the same code | symbol is attached | subjected to the thing similar to the apparatus and apparatus shown in FIG. 1, and it demonstrates centering around difference. The optical system 1 ′ is an optical system composed of a lens and a lens holder whose aperture amount can be changed according to an aperture setting signal, and forms a subject image on the light receiving surface of the image sensor 2.

  The processing device 5 'performs detection of a vehicle existing ahead, detection of road conditions, and the like based on the imaging signal after digital conversion. As this detection method, a conventionally known method may be used. The processing device 5 ′ also outputs the type of processing to be executed in the image processing device to the addition pattern generation device 6. For example, in order to detect a white line drawn on a road, vertical edge detection is performed by performing vertical Sobel filtering on 3 × 3 pixel data, a vertical edge detection signal is output to the addition pattern generation device 6. . The processing device 5 ′ further outputs a signal for setting an aperture amount to the aperture setting device 9 in order to capture an image necessary for processing with an optimum brightness.

  The aperture setting device 9 sets an aperture amount according to the processing of the processing device 5 ′ in order to photograph an image necessary for processing executed by the processing device 5 ′ with optimum brightness, and sends an aperture setting signal to the optical system. 1 ′ and the addition area setting device 7 ′.

  The addition area setting device 7 ′ sets an area (addition area) for performing addition processing by the adder 3 on the subject image in accordance with the aperture setting signal from the aperture setting apparatus 9, and adds the addition area to the addition pattern generation device 6. Outputs a setting signal. When a wide-angle lens is used to shoot with the aperture open, the four corners of the image become dark and uneven brightness occurs. The region where such uneven brightness occurs is set as an addition region and the above-described addition processing is performed. As a result, a bright and high-contrast image can be obtained. The larger the aperture amount, the smaller the area of uneven brightness at the four corners of the image. Therefore, the larger the aperture amount, the smaller the addition area, so that a bright image with no uneven brightness can be generated regardless of the aperture amount. be able to.

  The addition area may be reduced in accordance with the aperture amount of the optical system 1 ′, and may be optimized by changing the addition pattern and / or the number of added pixels in accordance with the aperture amount of the optical system 1 ′. In addition, in the case of performing addition processing in which weight is added for each pixel output, optimization may be performed by changing the weight for each pixel according to the aperture amount of the optical system 1 ′.

  As described above, even if luminance unevenness occurs due to the characteristics of the optical system, and the region where the luminance unevenness occurs varies depending on the optical characteristics of the optical system such as the magnification and the aperture amount, the addition processing is performed according to the region where the luminance unevenness occurs. By changing the region where the image processing is performed, it is possible to generate a bright image with no luminance unevenness in the entire subject image. In particular, it is effective when shooting with an imaging device using a lens with a small aperture diameter in which uneven brightness appears remarkably, and a small optical system can be used, so that the imaging device can be reduced in size and cost. .

  The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the adder 3 constitutes addition means, and the addition area setting device 7 constitutes addition area setting means. In addition, as long as the characteristic function of this invention is not impaired, each component is not limited to the said structure.

  In the embodiment described above, the adder is arranged immediately after the image sensor. However, an A / D converter is arranged between the image sensor and the adder to convert the image signal into a digital signal. The above-described addition processing may be performed. Furthermore, the above-described embodiments can be applied in combination.

  The present invention is also applicable to an imaging apparatus for a robot that detects the position of an object.

It is a figure which shows the structure of 1st Embodiment. It is a figure which shows the light receiving element (pixel) arrangement | sequence of an image pick-up element. It is a figure which shows the example of an image before and after an addition process. It is a figure which shows the structure of 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 1 'Optical system 2 Image pick-up element 3 Adder 4 A / D converter 5, 5' Processing apparatus 6 Addition pattern production | generation apparatus 7, 7 'Addition area setting apparatus 8 Magnification setting apparatus 9 Aperture setting apparatus

Claims (12)

An image processing device that forms a subject image via an optical system on a light receiving surface of an image sensor in which a plurality of pixels are developed on a two-dimensional plane, and processes the subject image captured by the image sensor,
For each pixel of the image sensor, an adding means for adding the output of a pixel existing around the pixel to the output of one pixel (hereinafter referred to as addition processing);
Addition region setting means for setting a region (hereinafter referred to as an addition region) for performing the addition processing on the subject image according to the optical characteristics of the optical system;
An image processing apparatus that performs the addition process in the addition area on the subject image to generate a new subject image. The image processing apparatus according to claim 1.
The image processing apparatus according to claim 1, wherein the optical system is a variable magnification optical system, and the addition region setting unit sets an addition region corresponding to a magnification of the variable magnification optical system. The image processing apparatus according to claim 2,
The image processing apparatus according to claim 1, wherein the adding means sets the number of added pixels according to the magnification of the zoom optical system. The image processing apparatus according to claim 2,
The addition unit has a combination pattern of a plurality of addition pixels (hereinafter referred to as an addition pattern), and performs the addition process by selecting an addition pattern corresponding to the magnification of the zoom optical system. Processing equipment. The image processing apparatus according to claim 2,
The image processing apparatus according to claim 1, wherein the adding unit assigns a weight for each pixel in accordance with a magnification of the zoom optical system and performs the adding process. The image processing apparatus according to claim 1.
The image processing apparatus according to claim 1, wherein the optical system is an optical system having a variable aperture, and the addition area setting unit sets an addition area according to an aperture amount of the optical system. The image processing apparatus according to claim 6.
The image processing apparatus according to claim 1, wherein the adding means sets the number of pixels to be added according to a diaphragm amount of the optical system. The image processing apparatus according to claim 6.
The addition means has a combination pattern of a plurality of addition pixels (hereinafter referred to as an addition pattern), and performs the addition process by selecting an addition pattern corresponding to the aperture amount of the optical system. apparatus. The image processing apparatus according to claim 6.
The image processing apparatus according to claim 1, wherein the adding unit assigns a weight for each pixel according to a diaphragm amount of the optical system and performs the adding process. An image processing method of forming a subject image via an optical system on a light receiving surface of an image sensor in which a plurality of pixels are developed on a two-dimensional plane, and processing the subject image captured by the image sensor,
An area (hereinafter, referred to as “addition process”) in which an addition process of adding the output of a pixel existing around the pixel to the output of one pixel for each pixel of the image sensor is performed on the subject image according to the optical characteristics of the optical system And an addition region), and a new subject image is generated by performing the addition process on the addition region on the subject image. An image processing method of forming a subject image via an optical system on a light receiving surface of an image sensor in which a plurality of pixels are developed on a two-dimensional plane, and processing the subject image captured by the image sensor,
An area (hereinafter, referred to as “additional processing”) that performs addition processing on the subject image in accordance with the magnification of the optical system to add the output of a pixel existing around the pixel to the output of one pixel for each pixel of the image sensor. An image processing method characterized in that a new subject image is generated by setting an addition region) and performing the addition process on the subject region on the subject image. An image processing method of forming a subject image via an optical system on a light receiving surface of an image sensor in which a plurality of pixels are developed on a two-dimensional plane, and processing the subject image captured by the image sensor,
A region (hereinafter, referred to as “additional processing”) that performs addition processing for adding the output of a pixel existing around the pixel to the output of one pixel for each pixel of the image sensor on the subject image according to the aperture amount of the optical system And an addition region), and a new subject image is generated by performing the addition process on the addition region on the subject image.

Images