JP2013126169A5

JP2013126169A5 -

Info

Publication number: JP2013126169A5
Application number: JP2011274875A
Authority: JP
Filing date: 2011-12-15
Publication date: 2014-12-04
Anticipated expiration: 2031-12-15

Claims

An image processing apparatus that processes an imaging signal output from an imaging element,
When each of a series of images included in the imaging signal is input as a target image, a plurality of pixels constituting the target image are selected as target pixels, and a plurality of in-plane pixels extracted from the target image An in-plane correlation evaluation value representing a correlation between each of the patterns and the target pixel is calculated, and the correlation between the target pixel and the target pixel is the highest among the plurality of in-plane pixel patterns based on the in-plane correlation evaluation value. A main in-plane pattern detection unit for detecting a high in-plane pixel addition pattern;
A plurality of auxiliary in-plane pixel patterns located around the in-plane pixel addition pattern in the attention image are extracted, and a correlation between each of the extracted plurality of auxiliary in-plane pixel patterns and the attention pixel is calculated. An auxiliary in-plane correlation evaluation value to be expressed is calculated, and an auxiliary in-plane pixel addition pattern having the highest correlation with the target pixel is detected from the plurality of auxiliary in-plane pixel patterns based on the auxiliary in-plane correlation evaluation value. An auxiliary in-plane pattern detector,
Calculating an inter-surface correlation evaluation value representing a correlation between each of a plurality of reference pixel patterns extracted from a reference image temporally adjacent to the target image in the series of images and the target pixel; A main reference pattern detection unit that detects a main reference pixel addition pattern having the highest correlation with the target pixel from the plurality of reference pixel patterns based on the inter-surface correlation evaluation value;
A pixel addition unit that generates a correction pixel by adding the pixel of the in-plane pixel addition pattern, the pixel of the main reference pixel addition pattern, and the pixel of the auxiliary in-plane pixel addition pattern each time the target pixel is selected An image processing apparatus comprising:

The image processing apparatus according to claim 1,
Each of the plurality of in-plane pixel patterns includes the target pixel and peripheral pixels centered on the target pixel,
The main surface pattern detection unit
A pixel selector that selects pixels of the plurality of in-plane pixel patterns from the image of interest;
The difference between the maximum pixel value and the minimum pixel value among the pixel values of a plurality of pixels constituting each of the plurality of in-plane pixel patterns is calculated as the in-plane correlation evaluation value, and the plurality of in-plane pixel patterns And an arithmetic unit that selects a pixel pattern having the smallest in-plane correlation evaluation value as the in-plane pixel addition pattern.

The image processing apparatus according to claim 2,
The main reference pattern detection unit
A pixel selection unit that selects pixels of the plurality of reference pixel patterns from the reference image;
The difference between the maximum pixel value and the minimum pixel value of the pixel values of the pixel constituting each of the plurality of reference pixel patterns and the target pixel is calculated as the inter-surface correlation evaluation value, and the plurality of references An image processing apparatus comprising: a calculation unit that selects, as the main reference pixel addition pattern, a pixel pattern having a minimum size of the inter-surface correlation evaluation value from among the pixel patterns.

The image processing apparatus according to claim 3,
The reference image is
A forward reference image that is temporally ahead of the image of interest;
A backward reference image that is located behind the target image in terms of time,
The plurality of reference pixel patterns are:
A plurality of forward reference pixel patterns extracted from the forward reference image;
A plurality of back reference pixel patterns extracted from the back reference image,
The inter-surface correlation evaluation value is
A first inter-surface correlation evaluation value representing a correlation between each of the plurality of forward reference pixel patterns and the target pixel;
A second inter-surface correlation evaluation value representing a correlation between each of the plurality of back reference pixel patterns and the target pixel;
The main reference pattern detection unit calculates a first reference pixel addition pattern having the highest correlation with the target pixel from the plurality of forward reference pixel patterns based on the first inter-surface correlation evaluation value. And selecting a second reference pixel addition pattern having the highest correlation with the target pixel from among the plurality of back reference pixel patterns based on the second inter-surface correlation evaluation value. An image processing apparatus that is selected as another part of the image processing apparatus.

The image processing apparatus according to any one of claims 1 to 4,
The auxiliary in-plane pattern detector is
A pixel selection unit for selecting pixels of the plurality of auxiliary in-plane pixel patterns from the target image;
Calculating the difference between the maximum pixel value and the minimum pixel value of the pixel values of the pixels constituting each of the plurality of auxiliary in-plane pixel patterns and the target pixel as the auxiliary in-plane correlation evaluation value, An image processing apparatus comprising: an arithmetic unit that selects, from among a plurality of auxiliary in-plane pixel patterns, a pixel pattern having the smallest auxiliary in-plane correlation evaluation value as the auxiliary in-plane pixel addition pattern. .

The image processing apparatus according to any one of claims 1 to 5,
Correlation determining unit that determines whether or not the auxiliary in-plane pixel addition pattern is a pixel pattern to be added by the pixel addition unit according to the correlation between the auxiliary in-plane pixel addition pattern and the target pixel. Further comprising
When the correlation determination unit determines that the auxiliary in-plane pixel addition pattern is a pixel pattern to be added, the pixel addition unit determines whether the pixel in the in-plane pixel addition pattern and the main reference pixel addition pattern When the pixel of the auxiliary in-plane pixel addition pattern is added to the pixel, and the correlation determination unit determines that the auxiliary in-plane pixel addition pattern is not a pixel pattern to be added, the in-plane pixel addition Do not add the pixel in the auxiliary in-plane pixel addition pattern to the pixel in the pattern and the pixel in the main reference pixel addition pattern,
An image processing apparatus.

The image processing apparatus according to any one of claims 1 to 5,
A plurality of auxiliary reference pixel patterns located around the main reference pixel addition pattern in the reference image are extracted, and an auxiliary representing a correlation between each of the extracted plurality of auxiliary reference pixel patterns and the target pixel Auxiliary reference pattern detection for calculating an inter-surface correlation evaluation value and detecting an auxiliary reference pixel addition pattern having the highest correlation with the target pixel from the plurality of auxiliary reference pixel patterns based on the auxiliary inter-surface correlation evaluation value And
The auxiliary reference pixel addition pattern should be added to the in-plane pixel addition pattern and the main reference pixel addition pattern by the pixel addition unit according to the correlation between the auxiliary reference pixel addition pattern and the target pixel. A correlation determination unit for determining whether or not the pixel pattern,
The pixel addition unit, when the correlation determination unit determines that the auxiliary reference pixel addition pattern is a pixel pattern to be added, a pixel of the in-plane pixel addition pattern and a pixel of the main reference pixel addition pattern The correction processing pixel is generated by adding pixels of the auxiliary reference pixel addition pattern to the image processing apparatus.

The image processing apparatus according to claim 7,
Between the first auxiliary planes representing the correlation between each of the plurality of first auxiliary reference pixel patterns extracted from the sub-reference images temporally adjacent to the reference image in the series of images and the target pixel. A correlation evaluation value is calculated, and a first auxiliary reference pixel addition pattern having the highest correlation with the target pixel is detected from the plurality of first auxiliary reference pixel patterns based on the correlation evaluation value between the first auxiliary planes. A first auxiliary auxiliary reference pattern detection unit;
The correlation determination unit is configured to convert the first auxiliary reference pixel addition pattern from the in-plane pixel addition pattern and the main pixel according to the correlation between the first auxiliary reference pixel addition pattern and the target pixel. Determine whether the pixel pattern is to be added to the reference pixel addition pattern,
When the correlation determination unit determines that the first auxiliary reference pixel addition pattern is a pixel pattern to be added, the pixel addition unit determines whether the pixel in the in-plane pixel addition pattern and the main reference pixel addition pattern An image processing apparatus, wherein the correction pixel is generated by adding a pixel of the first auxiliary reference pixel addition pattern to a pixel.

The image processing apparatus according to claim 8 ,
The reference image is positioned to be shifted forward or backward in time by one frame image with respect to the target image,
2. The image processing apparatus according to claim 1, wherein the sub-reference image is shifted to the one side by an amount corresponding to two frame images with respect to the target image.

The image processing apparatus according to claim 8, wherein:
A plurality of second auxiliary reference pixel patterns located around the first auxiliary reference pixel addition pattern in the sub-reference image are extracted, and each of the extracted second auxiliary reference pixel patterns and the target pixel are extracted. A correlation evaluation value between the second auxiliary planes representing a correlation between the second auxiliary planes, and the correlation between the target pixel and the second auxiliary reference pixel pattern is the highest among the plurality of second auxiliary reference pixel patterns based on the correlation evaluation value between the second auxiliary planes. A second auxiliary auxiliary reference pattern detection unit for detecting a high second auxiliary reference pixel addition pattern;
The correlation discriminating unit determines whether the second auxiliary reference pixel addition pattern is the pixel addition unit and the in-plane pixel addition pattern and the main pixel according to the correlation between the second auxiliary reference pixel addition pattern and the target pixel. Determine whether the pixel pattern is to be added to the reference pixel addition pattern,
When the correlation determination unit determines that the second auxiliary reference pixel addition pattern is a pixel pattern to be added, the pixel addition unit determines whether the pixel in the in-plane pixel addition pattern and the main reference pixel addition pattern An image processing apparatus, wherein the correction pixel is generated by adding a pixel of the second auxiliary reference pixel addition pattern to a pixel.

An image processing apparatus according to any one of claims 1 to 10,
The image pickup device includes a color filter array in which N types of color filters (N is a positive integer of 2 or more) that respectively transmit light of colors in N different wavelength ranges are periodically and planarly arranged. An imaging surface for detecting an optical image transmitted through the color filter array,
Each pixel constituting the series of images has any one of the colors in the N wavelength ranges,
The pixel addition unit generates the correction pixel by adding the pixels of the same color.

12. The image processing apparatus according to claim 11, wherein the color filter array is a Bayer array.

The image processing device according to any one of claims 1 to 12,
The plurality of reference pixel patterns are:
A pixel pattern having the same pixel arrangement as the in-plane pixel addition pattern;
An image processing apparatus comprising: a pixel pattern obtained by shifting a pixel pattern having the same pixel arrangement in the reference image.

An image processing apparatus according to any one of claims 1 to 13,
A noise component included in the imaging signal is attenuated to generate a noise reduction signal, and the noise reduction signal is supplied to the main surface pattern detection unit, the auxiliary surface pattern detection unit, and the main reference pattern detection unit. A noise reduction circuit,
The noise reduction circuit is:
A delay circuit that delays the imaging signal and outputs a signal of a plurality of pixels in a local pixel region in each image in parallel;
A multiplying unit that individually multiplies signals of a plurality of pixels in the local pixel region output in parallel by the delay circuit and generates a plurality of weighted pixels;
A sum calculating unit for calculating a sum of the plurality of weighted pixels;
An image processing apparatus comprising: a coefficient variable setting unit that sets the weighting coefficient to a value according to subject illuminance information.

15. The image processing apparatus according to claim 14, wherein the coefficient variable setting unit sets the weighting coefficient to a first predetermined value when the value of the subject illuminance information is greater than a threshold value, and the subject illuminance information When the value of is less than or equal to the threshold value, the weighting coefficient is set to a second predetermined value that is larger than the first predetermined value.

The image processing apparatus according to any one of claims 1 to 15,
The pixel addition unit generates the correction pixel by multiplying the addition result by the pixel addition unit by a sensitization magnification,
The image processing apparatus according to claim 1, wherein the value of the sensitization magnification is variably set.

An imaging apparatus comprising: the image processing apparatus according to claim 1; and the imaging element.

The image processing device according to claim 16, the imaging device,
An imaging apparatus comprising: a control unit that sets a value of the sensitization magnification according to an output of the pixel addition unit.

The imaging apparatus according to claim 18, wherein
An aperture stop disposed in front of the image sensor;
A signal amplifier for amplifying the imaging signal;
The control unit sets at least one of an aperture amount of the aperture stop, an amplification gain of the signal amplifier, and an exposure time of the image sensor according to an output of the pixel addition unit. Imaging device.

20. The imaging apparatus according to claim 19, wherein the control unit calculates a converted value of subject illuminance based on a signal amplitude, an amplification gain, the exposure time, and the sensitization magnification output from the pixel addition unit. An image pickup apparatus, wherein the aperture amount, the amplification gain, the exposure time, and the sensitization magnification are individually set according to the converted value of the subject illuminance.

The image processing device according to claim 16 , the imaging device,
A photometric unit that detects the illuminance of the subject by measuring the optical image incident on the image sensor;
An imaging apparatus comprising: a control unit that sets the value of the sensitization magnification according to the detected value of the subject illuminance.

The imaging device according to claim 21,
The controller is
An aperture stop disposed in front of the image sensor;
A signal amplifier for amplifying the imaging signal;
The control unit individually sets the aperture amount of the aperture stop, the amplification gain of the signal amplifier, the exposure time of the image sensor, and the sensitization magnification according to the detection value of the subject illuminance. An imaging device.

The imaging device according to any one of claims 18 to 22, wherein the imaging device is a solid-state imaging device.

An image processing method for processing an imaging signal output from an imaging element,
When each of a series of images included in the imaging signal is input as a target image, a plurality of pixels constituting the target image are selected as target pixels, and a plurality of in-plane pixels extracted from the target image Calculating an in-plane correlation evaluation value representing a correlation between each of the patterns and the target pixel;
Detecting an in-plane pixel addition pattern having the highest correlation with the target pixel from the plurality of in-plane pixel patterns based on the in-plane correlation evaluation value;
A plurality of auxiliary in-plane pixel patterns located around the in-plane pixel addition pattern in the attention image are extracted, and a correlation between each of the extracted plurality of auxiliary in-plane pixel patterns and the attention pixel is calculated. Calculating an auxiliary in-plane correlation evaluation value to represent;
Detecting an auxiliary in-plane pixel addition pattern having the highest correlation with the target pixel from the plurality of auxiliary in-plane pixel patterns based on the auxiliary in-plane correlation evaluation value;
Calculating an inter-surface correlation evaluation value representing a correlation between each of a plurality of reference pixel patterns extracted from a reference image temporally adjacent to the target image in the series of images and the target pixel. When,
Detecting a main reference pixel addition pattern having the highest correlation with the target pixel from the plurality of reference pixel patterns based on the inter-surface correlation evaluation value;
A step of adding a pixel of the in-plane pixel addition pattern, a pixel of the main reference pixel addition pattern, and a pixel of the auxiliary in-plane pixel addition pattern to generate a correction pixel each time the target pixel is selected; An image processing method comprising:

A computer program that is read from a recording medium and causes a processor to execute the steps of the image processing method according to claim 24.

26. A computer-readable recording medium on which the computer program according to claim 25 is recorded.