JP2006279812A - Brightness signal processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform an appropriate contour emphasis, while improving an S/N ratio in a low illuminance, or suppressing occurrence of a false contour signal on the borderline where a color is drastically changed in an RGB Bayer arrangement. <P>SOLUTION: Image signal data based on an image sensor output having a color filter of the Bayer arrangement are converted into a plurality of line outputs in a horizontal direction by a plural line output converter 202 for outputting. Detectors 207 to 210 detect a brightness signal level, etc. in pixel unit or in units of a predetermined region, and set a contour emphasis correction coefficient. A contour emphasis corrector 204 performs a contour emphasis processing in response to the contour emphasis correction coefficient in pixel unit or in units of a predetermined region to output a brightness signal. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a luminance signal processing apparatus such as a video signal processing apparatus capable of obtaining a high-resolution image by obtaining a broadband digital luminance signal from a video signal output from an image sensor developed for a digital camera, for example. Is.

  In recent years, there has been a remarkable shift from analog technology to digital technology in the camera industry. In particular, digital still cameras that do not require film or development are booming, and camera-equipped mobile phones dominate.

  At present, for example, as a digital camera, a camera that employs signal processing corresponding to a sensor equipped with a primary color filter from the viewpoint of emphasizing color reproducibility is mainly used, and a digital that emphasizes resolution and has excellent color S / N. An image pickup apparatus that requires signal processing and can perform noise removal while suppressing a decrease in resolution has been proposed (see, for example, Patent Document 1).

  A luminance signal processing apparatus in this type of imaging apparatus is configured as shown in FIG. 9, for example. In the figure, reference numeral 501 denotes a black level correction, a gamma correction, and an image signal data obtained by removing noise, amplifying, and A / D converting the sensor output from an image sensor having an RGB (Red Green Blue) Bayer array filter. And a pre-processing unit that performs gain correction and the like, 502 is a multi-line output conversion unit that converts the sensor output read from the sensor into a multi-line output in the horizontal direction, and 503 is generated due to the Bayer array of the sensor and is output to the sensor output A low-pass filter that removes the superimposed color carrier component and generates luminance signal data. 504 is a contour enhancement that performs high-frequency component extraction and addition from the luminance signal data after removal of the color carrier component to enhance the contour of the luminance signal data. A correction unit 505 is a post-processing unit that adjusts the bias level and gain of the luminance signal data.

  In the luminance signal processing apparatus, low-pass filter processing is performed at the beginning of processing, high-frequency color carrier components are removed and output to generate a main signal of the luminance signal, and contour enhancement correction is performed using the signal. Here, the luminance signal processing is performed in units of frames, and the degree of contour enhancement is uniform for all areas in one screen.

In Patent Document 1, noise reduction is performed by subtracting a high frequency component.
JP 2001-189944 A

  However, in the conventional luminance signal processing apparatus as described above, since the degree of contour emphasis is uniform for all regions in one screen, increasing the degree of contour emphasis decreases, for example, the S / N in a low illuminance part. Or a false contour signal is generated from the residual color carrier component at the boundary where the color change in the RGB Bayer array is severe, resulting in image quality deterioration, and it is difficult to perform appropriate contour enhancement. Had a point.

  In view of the above points, the present invention is suitable for improving the S / N in a low illuminance part or suppressing the generation of a false contour signal at a boundary where the color change in the RGB Bayer array is severe. The purpose is to perform edge enhancement.

In order to solve the above problems, the invention of claim 1
A luminance signal processing device that outputs luminance signal data based on image signal data based on a sensor signal from a two-dimensional image sensor,
A preprocessing unit that performs at least one of black level adjustment, gamma correction, and gain correction on the image signal data;
For the image signal data output from the pre-processing unit, a multi-line output conversion unit that simultaneously outputs image signal data for a plurality of lines,
A color carrier component removal filter unit that removes a color carrier component based on the image signal data output from the multiple line output conversion unit;
An outline enhancement correction unit that performs outline enhancement correction based on the image signal data output from the color carrier component removal filter unit;
A detection unit that detects at least one of a low-frequency luminance signal level, an arbitrary color level, a color difference edge level, and a red / green / blue saturation level based on the image signal data output from the multiple line output conversion unit;
With
The contour enhancement degree by the contour enhancement correction unit is configured to be set for each pixel according to the detection result by the detection unit.

The invention of claim 2
The luminance signal processing apparatus according to claim 1,
The contour enhancement correction unit
A horizontal high-pass bandpass filter that extracts high-frequency components from the image signal data of the horizontal line;
A vertical high-pass bandpass filter that extracts high-frequency components from the image signal data of the line in the vertical direction;
An adder that obtains contour enhancement data by adding the output of the horizontal high-pass bandpass filter and the output of the vertical bandpass filter;
And having
Furthermore, it has at least one of a coring processing unit that performs coring processing on contour enhancement data, a multiplier that amplifies contour enhancement data, and a nonlinear amplification unit that amplifies contour enhancement data with nonlinear characteristics. ,
That at least one of the coring level of the coring processing unit, the gain of the multiplier, and the non-linear characteristic of the non-linear amplifying unit is configured to be set according to a detection result by the detecting unit. Features.

The invention of claim 3
The luminance signal processing device according to claim 2,
The lower the luminance signal level, the arbitrary color level, the color difference edge level, or the red / green / blue saturation level detected by the detection unit, the smaller the degree of contour enhancement by the contour enhancement correction unit is set. It is characterized by that.

The invention of claim 4
The contour emphasis correction unit performs the contour emphasis correction only to the specified region according to the detection result by the detection unit in response to the region specification that specifies a predetermined region in one screen. It is comprised so that it may perform.

  As described above, when generating the edge emphasis correction signal from the main signal of the luminance signal from which the color carrier component is removed, the characteristics of the edge emphasis correction signal can be changed adaptively for each pixel. In low-light areas, the contour enhancement signal is suppressed to improve the S / N characteristics, and the contour enhancement signal is suppressed even at the boundary where the color change in the RGB Bayer array is severe to reduce the generation of false contour signals due to residual color carrier components. In addition, it is possible to weaken the edge enhancement correction at an arbitrarily set color level and suppress the enhancement correction for noise generated by the saturation signal of the sensor.

  According to the present invention, for example, the S / N characteristic is improved in a low illuminance part while performing appropriate edge enhancement, or the generation of a false edge signal is suppressed even at a boundary where the color change in the RGB Bayer array is severe. In addition, it is possible to suppress enhancement correction with respect to noise or the like in an area of an arbitrarily set color level or a saturation signal area of a sensor.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  When an image signal is output from an image sensor having a Bayer array color filter 101 as shown in FIG. 1, luminance signal data is generated by the luminance signal processing device 102. More specifically, for example, the imaging apparatus including the luminance signal processing apparatus 102 includes an image sensor 601, a timing generator 602 (TG) that generates a driving pulse of the image sensor 601, and an output of the image sensor 601 as shown in FIG. A CDS / AGC circuit 603 for removing noise and controlling gain, an analog-digital converter 604 (ADC), a digital signal processing circuit 605 (DSP), a memory circuit 606 for storing image data and various data, and a camera A microcomputer 607 (CPU) to be controlled is provided.

  The digital signal processing circuit 605 functions as the luminance signal processing device 102 by executing a predetermined program. More specifically, for example, it has a functional configuration as shown in FIG.

  The preprocessing unit 201 performs black level correction, gamma correction, gain correction, and the like on the image signal data A / D converted by the analog-digital converter 604.

  The multiple line output conversion unit 202 converts the image signal data output from the preprocessing unit 201 into a horizontal multiple line output.

  The color carrier removal / high frequency level correction filter unit 203 removes the color carrier component generated due to the Bayer array of the image sensor 601 and superimposed on the image signal, and generates luminance signal data. .

  The contour emphasis correcting unit 204 performs contour emphasis on the luminance signal data by extracting and adding the high-frequency component in the horizontal direction and the vertical direction to the luminance signal data after the color carrier component removal. Yes. The detailed configuration of the contour emphasis correcting unit 204 will be described later.

  The post-processing unit 205 adjusts the bias level and gain of the luminance data.

  The luminance signal level detection unit 207, the arbitrary color level detection unit 208, the color difference edge detection unit 209, and the RGB saturation level detection unit 210 are respectively output from a plurality of RGB Bayer multiple lines output from the multiple line output conversion unit 202. A contour enhancement correction coefficient 1 that detects the luminance signal level, the arbitrary color level, the edge level of the color difference signal, or the saturation level of the RGB signal for each pixel, and controls the characteristics of the contour enhancement correction unit 204 according to those levels. ~ 3 are generated.

  Here, all the detections by the detection units 207 to 210 are not necessarily performed, and only one of them may be performed. Further, when two or more detections are performed, the values of the edge enhancement correction coefficients 1 to 3 determined by each are averaged or weighted average, or one of them is preferentially used. Also good.

  The designated area generating unit 211 in one screen sets the contour enhancement correction coefficients 1 to 3 to the contour enhancement correction unit 204 by the detection units 207 to 210 in units of pixels or predetermined regions in accordance with a designation signal (not shown). An effective range signal indicating whether to enable each is output.

  The AND circuit 212 validates only the contour enhancement correction coefficients 1 to 3 selected by the in-screen designated area generation unit 211 out of the contour enhancement correction coefficients 1 to 3 output from the detection units 207 to 210. An input to the emphasis correction unit 204 is made.

  More specifically, the contour emphasis correcting unit 204 is configured as shown in FIG. 4, for example.

  The horizontal high-pass bandpass filter 301 extracts a high-frequency component in the horizontal direction from the main signal of the luminance signal data.

  The vertical high-frequency bandpass filter 302 extracts a high-frequency component in the vertical direction from the main signal (a plurality of lines) of the luminance signal data.

  The adder 303 obtains a high-frequency component (high-frequency extraction signal) in a two-dimensional direction by adding the high-frequency components in the horizontal and vertical directions extracted by the bandpass filters 301 and 302. .

  The coring processing unit 304 performs coring processing with a characteristic corresponding to the contour enhancement correction coefficient 1 output from the detection units 207 to 210 (via the AND circuit 212). A horizontal filter that restricts the high range is used.

  The multiplier 305 amplifies the high-frequency extraction signal output from the coring processing unit 304 with a gain corresponding to the contour enhancement correction coefficient 2 output from the detection units 207 to 210.

  The non-linear processing unit 306 adjusts the level of the high-frequency extraction signal output from the multiplier 305 with non-linear characteristics corresponding to the contour enhancement correction coefficient 3 output from the detection units 207 to 210.

  Note that the coring processing unit 304, the coring processing unit 304, and the multiplier 305 are not necessarily provided in the order shown in FIG.

  An operation of the imaging apparatus including the luminance signal processing apparatus 102 configured as described above will be described.

  First, when imaging light enters the image sensor 601 through a lens (not shown), it is converted into an electric signal by a photodiode or the like, and an analog continuous signal is obtained by vertical driving and horizontal driving synchronized with a driving pulse from the timing generator 602. Is output from the image sensor 601. The output image signal is subjected to automatic gain control after the 1 / f noise is appropriately reduced by the sample hold circuit (CDS) of the CDS / AGC circuit 603, and input to the analog-digital converter 604 (A / D converter). Then, it is converted into image signal data (RGB data) which is a digital signal. The converted image signal data is input to the digital signal processing circuit 605, and various processes such as luminance processing, color separation, and color matrix processing are performed via the memory circuit 606.

  Specifically, the luminance signal processing is performed as follows. When the color filter array of the image sensor 601 is an RGB Bayer array, the image signal data captured by the digital signal processing circuit 605 is subjected to black level adjustment, gamma adjustment, and color gain adjustment by preprocessing of the preprocessing unit 201. After that, the multiple line output conversion unit 202 uses the line memory or work memory to convert the output order, and sequentially outputs the image signal data of the pixels of the multiple horizontal lines in parallel.

  The output from the multi-line output conversion unit 202 is input to the color carrier removal / high-frequency level correction filter unit 203 while maintaining the Bayer arrangement information, and the components near the Nyquist frequency including the color carrier are removed. The main signal of the luminance signal is generated.

  Thereafter, the contour emphasis correcting unit 204 extracts and adds high frequency components from the luminance signals of a plurality of lines after the color carrier component is removed, and performs contour signal emphasis processing. Further, the post-processing unit 205 adjusts the bias level and gain of the luminance signal in order to optimize the output level of the luminance signal in the 8-bit range.

  In the contour enhancement process, more specifically, after the high-frequency extraction signals output from the horizontal high-frequency bandpass filter 301 and the vertical high-frequency bandpass filter 302 are added by the adder 303, for each pixel, The following correction processing according to the detection results of the detection units 207 to 210 is performed.

  In the coring processing unit 304, for example, as shown in FIG. 5, the positive and negative coring levels are variably controlled according to the contour emphasis correction coefficient 1 output from the detection units 207 to 210. Therefore, when the coring level is increased, as a result, the degree of contour enhancement is reduced, but the S / N of the low illuminance portion is improved.

  Further, in the multiplier 305, the gain is variably controlled, for example, as shown in FIG. Therefore, if the gain is reduced, the degree of contour enhancement is also reduced, but the S / N of the low illuminance part is improved.

  Further, in the nonlinear processing unit 306, the rising slope of the input / output characteristics is variably controlled according to the edge enhancement correction coefficient 3, for example, as shown in FIG. Therefore, even if the inclination is reduced, the degree of contour enhancement is reduced, but the S / N of the low illuminance part is improved. The input / output level relationship is not limited to a straight line as shown in FIG. 7, but may be a curved line as shown in FIG.

  On the other hand, the luminance signal level detection unit 207 detects the luminance signal level of each pixel based on the output from the multi-line output conversion unit 202. For example, the lower the luminance signal level (or the lower the predetermined threshold value) B), the edge enhancement correction coefficients 1 to 3 are outputted so as to reduce the degree of edge enhancement.

  In addition, the arbitrary color level detection unit 208 performs contouring so that the degree of contour emphasis is reduced as the luminance level is lower (or lower than a predetermined threshold), for example, for pixels of arbitrary color set in advance. Emphasis correction coefficients 1 to 3 are output. Specifically, for example, for a blue sky or a skin color, in many cases, a large outline emphasis degree is not necessarily required, and the S / N decrease is easily noticeable. By improving / N, the overall image quality can be improved.

  Similarly, the chrominance edge detection unit 209 detects the chrominance edge level of each pixel. For example, the lower the chrominance edge level (or lower than a predetermined threshold value), the smaller the edge enhancement is. Correction coefficients 1 to 3 are output. As a result, it is possible to suppress the emphasis of the false high-frequency signal generated at the color difference edge portion and improve the image quality.

  For example, the RGB saturation level detection unit 210 detects the RGB saturation level for each pixel and determines that the sensor output is saturated when the input signal is greater than the set saturation level, and reduces the degree of edge enhancement. The edge emphasis correction coefficients 1 to 3 are output to As a result, the false high-frequency signal generated at the edge of the saturated area can be suppressed, and the rough feeling seen in the image sensor at the time of saturation can be hardly exaggerated, thereby improving the image quality.

  The edge enhancement correction coefficients 1 to 3 corresponding to the detection results of the detection units 207 to 210 are pixels or areas corresponding to the designation when the setting effective area is designated by the designated area generating unit 211 within one screen. , The signal passes through the AND circuit 212 and is actually set in the contour emphasis correction unit 204. Thereby, the luminance signal processing as described above can be performed for each shooting scene, each region, and the like.

  As described above, according to the present invention, when generating the edge enhancement correction signal from the main signal of the luminance signal from which the color carrier component has been removed, the characteristic of the edge enhancement correction signal is changed to the low-frequency luminance signal level for each pixel. It can be controlled and adaptively independently by detection, arbitrary color level detection, edge level detection of color difference data, and saturation level detection of RGB Bayer data. For example, in low light areas, the contour enhancement signal is suppressed and the S / N characteristics are improved. In addition, it suppresses the edge enhancement signal even at the boundary where the color change in the RGB Bayer arrangement is severe, reduces the false edge signal generation due to the residual color carrier component, weakens the edge enhancement correction at the arbitrarily set color level, and the sensor saturation signal It can be optimally controlled within one screen to suppress enhancement correction for noise generated in.

  The luminance signal processing apparatus according to the present invention, for example, improves the S / N characteristic in a low illuminance part while performing appropriate edge enhancement, or generates a false edge signal even at a boundary where the color change in the RGB Bayer array is severe. For example, an image sensor developed for a digital camera has an effect of suppressing generation and suppressing enhancement correction for noise in an area of an arbitrarily set color level or a saturation signal area of the sensor. It is useful as a luminance signal processing device such as a video signal processing device that can obtain a high resolution image by obtaining a wideband digital luminance signal from the video signal output from the video signal.

It is explanatory drawing which shows the example of the color filter in embodiment of this invention. 2 is a block diagram illustrating a configuration of an imaging apparatus to which the luminance signal processing apparatus 102 is applied. FIG. 2 is a block diagram showing a configuration of a luminance signal processing apparatus 102. FIG. 2 is a block diagram showing a specific configuration of an outline emphasis correction unit 204. FIG. 4 is a graph showing an example of characteristics of the coring processing unit 304. FIG. 3 is a graph showing an example of characteristics of a multiplier 305. FIG. 4 is a graph showing characteristics of the nonlinear processing unit 306. FIG. 6 is a graph showing another example of the characteristic of the nonlinear processing unit 306; It is a block diagram which shows the structure of the conventional luminance signal processing apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Color filter 102 Luminance signal processing apparatus 201 Pre-processing part 202 Multiple line output conversion part 203 Color carrier removal / high-pass level correction filter part 204 Contour emphasis correction part 205 Post-processing part 207 Luminance signal level detection part 208 Arbitrary color level detection part 209 Color difference edge detection unit 210 RGB saturation level detection unit 211 In-screen specified area generation unit 212 AND circuit 301 Horizontal high-frequency bandpass filter 302 Vertical high-frequency bandpass filter 303 Adder 304 Coring processing unit 305 Multiplier 306 Non-linear processing 601 Image sensor 602 Timing generator 603 CDS / AGC circuit 604 Analog-digital converter 605 Digital signal processing circuit 606 Memory circuit 607 Microcomputer

Claims (4)

A luminance signal processing device that outputs luminance signal data based on image signal data based on a sensor signal from a two-dimensional image sensor,
A preprocessing unit that performs at least one of black level adjustment, gamma correction, and gain correction on the image signal data;
For the image signal data output from the pre-processing unit, a multi-line output conversion unit that simultaneously outputs image signal data for a plurality of lines,
A color carrier component removal filter unit that removes a color carrier component based on the image signal data output from the multiple line output conversion unit;
An outline enhancement correction unit that performs outline enhancement correction based on the image signal data output from the color carrier component removal filter unit;
A detection unit that detects at least one of a low-frequency luminance signal level, an arbitrary color level, a color difference edge level, and a red / green / blue saturation level based on image signal data output from the multiple-line output conversion unit;
With
A luminance signal processing apparatus, wherein the degree of contour enhancement by the contour enhancement correction unit is set for each pixel in accordance with a detection result by the detection unit. The luminance signal processing apparatus according to claim 1,
The contour enhancement correction unit
A horizontal high-pass bandpass filter that extracts high-frequency components from the image signal data of the horizontal line;
A vertical high-pass bandpass filter that extracts high-frequency components from the image signal data of the line in the vertical direction;
An adder that obtains contour enhancement data by adding the output of the horizontal high-pass bandpass filter and the output of the vertical bandpass filter;
And having
Furthermore, it has at least one of a coring processing unit that performs coring processing on contour enhancement data, a multiplier that amplifies contour enhancement data, and a nonlinear amplification unit that amplifies contour enhancement data with nonlinear characteristics. ,
That at least one of the coring level of the coring processing unit, the gain of the multiplier, and the non-linear characteristic of the non-linear amplifying unit is configured to be set according to a detection result by the detecting unit. A characteristic luminance signal processing apparatus. The luminance signal processing device according to claim 2,
The lower the luminance signal level, the arbitrary color level, the color difference edge level, or the red / green / blue saturation level detected by the detection unit, the smaller the degree of contour enhancement by the contour enhancement correction unit is set. A luminance signal processing apparatus.   The contour emphasis correction unit performs the contour emphasis correction only to the specified region according to the detection result by the detection unit in response to the region specification that specifies a predetermined region in one screen. A luminance signal processing device configured to perform the processing.

Images