KR101068886B1 - Image signal processing device and image signal synthesis method - Google Patents

Abstract

The present invention relates to the synthesis of two images having different characteristics. An image signal processing apparatus according to the present invention receives a first image signal having a low illuminance characteristic and a second image signal having a high illuminance characteristic of a subject, and a luminance component that extracts a first luminance signal and a second luminance signal corresponding to each other Extraction unit; A reference luminance signal for generating a reference luminance signal having a value between the first luminance signal and a second luminance signal in a region where brightness is darker than a first reference; Generation unit; A mapping unit to redefine the first image signal and the second image signal into a first mapping signal and a second mapping signal by using the reference luminance signal; And a synthesis unit configured to synthesize the first mapping signal and the second mapping signal.

Luminance Extraction, Large Dynamic Range, WDR

Description

Image Signal Processing Apparatus and Image Signal Synthesis Method

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing apparatus having a wideband dynamic range and a method for synthesizing an image signal, and more particularly, to synthesizing two images having different characteristics.

Conventional video signal processing devices have a very limited dynamic range compared to the human eye. For humans, the dynamic range of 120dB or more, and the range from bright to dark, the normal imaging device has a dynamic range of about 60dB due to the limited charge storage capacity. After all, when the dark and bright luminance levels are mixed, it is not easy to express the imaging device at the same time. Therefore, much research has been conducted to realize a dynamic range similar to that of human eyes.

As a method for solving this problem, a method using a method of synthesizing two images having different characteristics, for example, high and low illumination images having different exposure times, respectively, is used.

1 to 2 illustrate synthesis of images according to the prior art.

(A) of FIG. 1 is an image LP including color components obtained by prolonging the exposure time, (b) is an image SP including color components obtained by shortening the exposure time, and (c) Is a graph showing these characteristics.

2 is a graph showing an image obtained by averaging by a conventional method and synthesizing them, and their characteristics.

However, according to the conventional method, there is a problem that the low light region of the synthesized image is dark and the chroma is inferior.

An object of the present invention is to obtain an image having a wide dynamic range without deterioration or damage of image quality.

In addition, an object of the present invention is to extract the luminance components of the low light image and high light image and to synthesize the two images in consideration of the luminance characteristics.

An image signal processing apparatus according to the present invention receives a first image signal having a low illuminance characteristic and a second image signal having a high illuminance characteristic of a subject, and a luminance component that extracts a first luminance signal and a second luminance signal corresponding to each other Extraction unit; A reference luminance signal for generating a reference luminance signal having a value between the first luminance signal and a second luminance signal in a region where brightness is darker than a first reference; Generation unit; A color mapping unit to redefine the first image signal and the second image signal into color signals of a first mapping signal and a second mapping signal by using the reference luminance signal; And a synthesis unit configured to synthesize the first mapping signal and the second mapping signal.

The reference luminance signal may include a second reference for suppressing a sudden characteristic change in a region brighter than the first reference.

The apparatus may further include a weighting factor generating unit configured to generate a weighting factor using the difference between the first reference and the second reference and the reference luminance signal.

The first mapping signal is a value obtained by dividing the reference image signal by a first luminance signal by the first image signal, and the second mapping signal is obtained by dividing the reference image signal by a second luminance signal. It is preferably a value obtained by multiplying two video signals.

The first mapping signal is obtained by subtracting the first luminance signal from the reference image signal and adding the first image signal. The second mapping signal is obtained by subtracting the second luminance signal from the reference image signal. It is preferable that the value is obtained by adding two video signals.

Preferably, the synthesis unit multiplies the first mapping signal by a value obtained by subtracting the weighting factor from 1, multiplies the second mapping signal by the weighting factor, and then adds them.

In addition, the video signal synthesis method according to the present invention comprises the steps of receiving a first video signal having a low illumination characteristics and a second video signal having a high illumination characteristics and extracting a luminance signal from each of these signals; An arbitrary reference is set in the extracted luminance signal, and in a region darker than the reference, a saturation region in which the low luminance characteristic of the luminance signal is used as it is, and the reference and the first and second video signals intersect. Generating a reference luminance signal by connecting a signal; Redefining the first video signal and the second video signal by using the reference luminance signal by color mapping the first and second mapping signals; And synthesizing the first mapping signal and the second mapping signal.

The reference luminance signal may include a second reference for suppressing a sudden characteristic change in a region brighter than the first reference.

The method may further include generating a weighting coefficient using the difference between the first reference and the second reference and the reference luminance signal.

The first mapping signal is a value obtained by dividing the reference image signal by a first luminance signal by the first image signal, and the second mapping signal is obtained by dividing the reference image signal by a second luminance signal. It is preferably a value obtained by multiplying two video signals.

The first mapping signal is obtained by subtracting the first luminance signal from the reference image signal and adding the first image signal. The second mapping signal is obtained by subtracting the second luminance signal from the reference image signal. It is preferable that the value is obtained by adding two video signals.

In the synthesizing of the mapping signals, it is preferable to multiply the first mapping signal by a value obtained by subtracting the weighting factor from 1, multiply the second mapping signal by the weighting factor, and then add them.

According to the present invention, the image processing apparatus can provide a wide dynamic range without deterioration or damage of image quality.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

3 is a block diagram showing an embodiment of an image signal processing apparatus according to the present invention.

Referring to FIG. 3, the image signal processing apparatus includes a pixel unit 10, an image signal acquisition unit 20, a luminance component extraction unit 30, a reference luminance signal generation unit 40, a color mapping unit 50, And a weighting coefficient generator 60 and a synthesizer 70.

The pixel unit 10 includes a light conversion element to receive light to convert optical energy into electrical energy.

The image signal acquisition unit 20 acquires an image signal from the pixel unit 10, obtains a low illumination image, and outputs a high illumination image with the first image acquisition unit 22 that outputs the first image signal LP. It may include a second image acquisition unit 24 to obtain and output as a second image signal (SP). Although only one pixel portion is illustrated in the present embodiment, two pixels may be used to separately configure a high exposure pixel and a low exposure pixel. The low light image and the high light image may be controlled by the exposure time or may be controlled by the area of the light conversion element of the pixel. The low illuminance image refers to a signal having a lower illuminance than a preset image, and the high illuminance image refers to a signal having a higher illuminance than a preset image.

The luminance component extractor 30 extracts a luminance component from the first image signal LP, and extracts and outputs a first luminance signal LPY and a second luminance signal SPY.

The reference luminance signal generator 40 generates and outputs a new reference luminance signal Ymap based on the first luminance signal LPY and the second luminance signal SPY. The reference luminance signal Ymap is generated by setting at least one reference and connecting the points A where the criteria and the two luminance signals meet. In this embodiment, the case in which the above two criteria are taken as an example. The first and second criteria CP and MP may be set to optimal values in consideration of luminance characteristics as a value between the first luminance signal and the second luminance signal. Optimum values can be obtained from experimental data. The first reference CP is for using the low illuminance characteristic of the first luminance signal LPY as it is, and the second reference MP is for suppressing a sudden characteristic change. The second criterion is not necessary but preferably at least one.

The color mapping unit 50 maps and resets the first image signal LP and the second image signal SP to a color signal based on the reference luminance signal Ymap. You can map it like this:

LPmap = [Ymap / LPY] × LP

SPmap = [Ymap / SPY] × SP

In the above equation, LPmap and SPmap are newly mapped first and second video signals, LPY and SPY are first and second luminance signals, and LP and SP are original first and third video signals before mapping. 2 Video signal.

In another embodiment, the mapping may be performed as in Equation 2 below.

LPmap = [Ymap-LPY] + LP

SPmap = [Ymap-SPY] + SP

The first video signal and the second video signal mapped in the above manner are synthesized by the combining unit 70 in consideration of the weighting coefficient.

The weighting coefficient generator 60 obtains the weighting coefficient WGT in the same manner as in Equation 3 based on the reference luminance signal Ymap.

The synthesizing unit 70 synthesizes and outputs the first image signal LPmap and the second image signal SPmap d. In this case, it is preferable that the second image signal SPmap becomes larger as the weighting coefficient is larger to reflect the weighting coefficient. It can be synthesized as in Equation 4 below.

WDR = (1-WGT) × LPmap + WGT × SPmap

According to the above equation, the first video signal LPmap having a good low illuminance characteristic has a higher specific gravity in the low light region smaller than the first reference CP, and the second video signal SPmap having a high illuminance characteristic is better than MP. The specific gravity is increased in the high and high illuminance region, and the intermediate region is composed of a combination of LPmap and SPmap.

5 is an image obtained by synthesizing two image signals LP and SP according to the present invention. It can be seen that the image quality is improved compared with FIG. 2 according to the prior art. Since the color component is mapped and combined using the luminance component defined as the optimal value, the conventional saturation problem can be improved.

6 illustrates an image synthesis method according to the present invention.

Referring to FIG. 6, a luminance signal is first extracted from an input signal (S10), and a reference luminance signal is generated based on the extracted signal (S20). In this case, the input signal may be a signal having a low light characteristic and a signal having a high light characteristic, and the method of generating the reference luminance signal may be set as described above, and an arbitrary reference may be set and the light intensity is low in a region darker than the reference. The signal can be used as it is. When setting two or more references, the point in the lowest light intensity region is the first reference, and the low light signal is used as it is in the first region or less and the value between the low light signal and the high light signal is set in the remaining regions. By connecting the references with each other, a reference luminance signal can be generated.

When the reference luminance signal is generated, the initial input signal is reset based on the reference luminance signal (S40). Equation 1 and Equation 2 described above may be used as a resetting method. When the mapping is completed, the synthesizers may be synthesized (S50) to terminate the synthesis of the image signals, and may obtain an image having a wide dynamic range with improved image quality compared to the conventional one.

In the synthesizing step, an improved image may be obtained by using a weight. As shown, when the reference luminance signal is generated, the weighting coefficient may be set. The weighting coefficient may be set in the same manner as in Equation 3 below. This example also takes the case of two criteria.

When the weighting coefficient is generated, an improved image signal having a wide dynamic range can be obtained by synthesizing the low light signal and the high light signal in the same manner as in Equation 4.

The present invention can be widely applied to an image signal processing apparatus that acquires two image signals and combines them, such as an image pickup device. Therefore, it can be applied to CCD image sensor as well as CMOS image sensor.

1 is a graph illustrating a photograph obtained by acquiring a low light image and a high light image and its signal characteristics;

2 is a photograph and a characteristic graph obtained by synthesizing the image obtained in FIG.

3 is a block diagram of an image signal processing apparatus according to the present invention;

4 is a graph illustrating a method of generating a reference luminance signal;

5 is a picture synthesized according to the present invention the image obtained in FIG.

6 is a flowchart illustrating a method of synthesizing an image according to the present invention.

Claims (13)

The first luminance signal and the first image signal having a lower illuminance characteristic than the preset reference signal among the image signals of the subject and the second image signal having a higher illuminance characteristic than the preset reference signal among the image signals are received. A luminance component extractor for extracting two luminance signals; A reference luminance signal generator for generating a reference luminance signal having a value between the first luminance signal and the second luminance signal in a region where brightness is brighter than a first reference; And And a color mapping unit defining a color signal using the first image signal and the second image signal using the reference luminance signal. The method of claim 1, And the reference luminance signal generating unit generates the reference luminance signal having the first luminance signal value or the sum of the first luminance signal and the second luminance signal in a region darker than the first reference. The method of claim 1, And the reference luminance signal includes a second reference which is a brighter signal than the first reference. The method of claim 3, And a weighting coefficient generator for generating a weighting coefficient using the difference between the first reference and the second reference and the reference luminance signal. The method of claim 1, wherein the color mapping unit Generating a first color signal by multiplying the reference video signal by a value obtained by dividing the first luminance signal by the first video signal; And a second color signal generated by multiplying the reference video signal by a value obtained by dividing the second luminance signal by the second video signal. The method of claim 1, wherein the color mapping unit Subtracting the first luminance signal from the reference image signal and adding the first image signal to generate a first raw signal; And a second color signal generated by subtracting the second luminance signal from the reference video signal and generating the second color signal. The method of claim 4, wherein And a synthesizer configured to multiply the first color signal by a value obtained by subtracting the weight factor from 1, multiply the second color signal by the weight factor, and add the sums thereof. A first luminance signal corresponding to these signals by receiving a first image signal having a lower illuminance characteristic than a preset reference signal among the image signals of the subject and a second image signal having a higher illuminance characteristic than the preset reference signal among the image signals of the subject And extracting a second luminance signal; A reference is set in the extracted luminance signal, the reference having the first luminance signal value in a region darker than the reference, and having a value between the first luminance signal and the second luminance signal in the region brighter than the reference. Generating a luminance signal; And Redefining by mapping the first video signal and the second video signal to a color signal using the reference luminance signal; And synthesizing the mapped signal. The method of claim 8, And the reference luminance signal comprises a second reference which is a brighter signal than the first reference. The method of claim 9, And generating a weighting coefficient using the difference between the first reference and the second reference and the reference luminance signal. The method of claim 8, wherein the redefining The first video signal is multiplied by the first luminance signal divided by the reference video signal, and mapped to the first color signal. And multiplying the reference video signal by a value obtained by dividing the second luminance signal by the second video signal and embedding the second video signal into a second color signal. The method of claim 11, The first color signal is a value obtained by subtracting the first luminance signal from the reference image signal and adding the first image signal. And the second color signal is a value obtained by subtracting the second luminance signal from the reference image signal and adding the second image signal. The method of claim 10, The synthesizing of the redefined signal may include multiplying the first mapping signal by a value obtained by subtracting the weighting factor from 1, multiplying the weighting coefficient by the second mapping signal, and summing them.

Images