KR20090111065A - Apparatus for image composition - Google Patents

Abstract

PURPOSE: An image composition device for synthesizing an efficient image is provided to supply efficient WDR about a camera by supplying an image and synthesize the efficient image. CONSTITUTION: An image composition device for synthesizing an efficient image is as follows. A histogram calculation unit(110) calculates an exposure histogram and exposure brightness signal. An exposure calculation unit(120) calculates the shutter speed. The synthesis parameter calculation unit includes the intensity level translation table calculation unit and color difference weight calculating unit. The intensity level translation table calculation unit calculates the intensity level translation table.

Description

Apparatus for image composition

The present invention relates to an image synthesizing apparatus, and more particularly, to an apparatus for efficient image synthesis of long and short exposures in a long exposure / short exposure image capturing technique used to widen a dynamic range of a camera. It is about.

Various methods for widening the dynamic range of the camera have been proposed, which is attributable to the narrower dynamic range of the camera than the human eye. The human eye has a dynamic range of more than 120dB, while the camera usually exceeds or falls below 60dB.

Accordingly, the attempts to extend the dynamic range of the proposed camera have been largely carried out in two ways.

1. How to increase the performance of the image sensor

2. Combining two images with narrow dynamic range

The first method is to improve the performance of the image sensor itself for a single image, which is performed by sensor manufacturers, but there are technical limitations at present. In other words, technology development is difficult, productivity is low, and price is at a limit.

The second method is a method of obtaining and synthesizing two images of a long exposure image and a short exposure image for a single image. Currently, a camera to which the corresponding technology is applied has been commercialized. In other words, in order to realize wide dynamic range (WDR) by synthesis, sensor makers produce dual scan CCDs that simultaneously produce long and short exposure images instead of the first method. (charge-coupled device) A sensor is commercially available.

The conventional camera configuration to which such a WDR is applied is shown in FIG. 1.

Although it includes detailed elements such as a vertical driver, an analog front end (AFE), a digital signal processor (DSP), and an SDRAM, in detail, the image sensor 10 that receives an image and converts it into an electrical signal and the converted signal It consists of an image processing unit 20 to process.

In order to acquire two images of the long exposure image and the short exposure image for the same image, the image sensor 10 outputs the long exposure image and the short exposure image by operating the shutter twice for the same image. shall. The signal for the long exposure image and the signal for the short exposure image thus output are synthesized by the image processor 20 to obtain a desired image.

The process in detail is as follows. WDR is also required in a general environment, but especially in a dark room where a part of the room is included to capture a brighter outdoor or a bright place such as outside the tunnel. Required when capturing an image.

2 is a view illustrating an image of an outdoor brighter than an indoor ecology including a part of an indoor room in a dark room with a general camera.

Some interior structures appear, but are not clearly visible, while outdoors appear darker than they actually are.

Although the state of the displayed image may be adjusted by adjusting the exposure degree of a general camera, the short exposure image and the long exposure image of FIG. 3 or the image state therebetween may be adjusted, and in any case, the user may not obtain a satisfactory image.

In other words, when the exposure is shortened by shortening the shutter speed as in the short exposure image, the interior becomes dark enough to distinguish the structure, and the outdoor is also darker than the actual exposure, and the shutter speed is slowed down as in the long exposure image. If the exposure is prolonged for a long time, the indoor structure can be clearly distinguished, but the luminance of the outdoor image becomes saturated and appears white.

Accordingly, the WDR-applied camera of FIG. 1 obtains a satisfactory image as shown in FIG. 2 by acquiring two short and long exposure images of FIG. Do it. In this case, conventionally, the synthesized image is processed and output in a batch manner such as dividing the color value in half, which causes a problem as follows.

Due to the fixed or batch processing of the synthesized image, the contrast is reduced or a blurry image appears, which provides an improved image than a conventional camera, but it is insufficient to satisfy a user's desire.

In addition, in order to apply the composite WDR, the long exposure image and the short exposure image should be derived in an optimal state suitable for the corresponding image, and as a result, it is not possible to suggest a different alternative to the control of the shutter speed. The quality of the video is being degraded.

The present invention was created to solve the above problems, and an object of the present invention is to provide a shutter speed corresponding to a real image in order to improve the efficiency of a WDR of a synthesis method requiring a long exposure image and a short exposure image.

In addition, by improving the synthesis of the obtained long-exposure image and the short-exposure image, another object is to solve the problem of lowering the contrast, blurring image generated during the synthesis process.

As a result, it aims to implement a more realistic WDR through this.

In order to achieve the above object, the present invention includes a histogram calculator for receiving a long exposure luminance and a short exposure luminance signal to calculate a long exposure and a short exposure histogram, respectively; And an exposure calculator configured to calculate a long exposure shutter speed and a short exposure shutter speed based on the long exposure / short exposure histogram.

Also, a luminance level conversion table calculator which calculates a long exposure / short exposure luminance level conversion table based on the long exposure / short exposure histogram and a long exposure / short exposure based on the long exposure / short exposure / histogram. There is provided an image synthesizing apparatus further comprising a composition parameter calculator including a color difference weight calculator for calculating color difference weights.

Herein, a luminance synthesizer configured to perform a luminance level conversion on the long exposure luminance signal and the short exposure luminance signal according to the long exposure / short exposure luminance level conversion table, and synthesize the luminance exposure unit, a long exposure color difference signal, and a short exposure color difference signal. Preferably, the apparatus further includes a synthesis unit having a color difference synthesis unit configured to apply the weights, wherein the color difference synthesis unit is disposed at the front of the weight to normalize the input long exposure color difference signal and the short exposure color difference signal. It is preferable to further include a normalization unit and a denormalization unit which denormalizes again after combining by applying the weight.

As described above, the image synthesizing apparatus according to the present invention analyzes the luminance of an image to be captured and performs a shutter speed and composition processing corresponding thereto, thereby providing an image close to reality, which is a camera. It means providing efficient WDR for.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

4 is a block diagram illustrating an image synthesizing apparatus according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the apparatus for synthesizing an image according to the present embodiment receives a long exposure luminance and a short exposure luminance signal and calculates a long exposure and a short exposure histogram, respectively, and the long exposure / short exposure histogram. The exposure calculation unit 120 calculates a long exposure shutter speed and a short exposure shutter speed based on the control.

A histogram is a graph of the frequency distribution, which is used by cameras to measure the exposure of an image. It is a numerical value representing the distribution of contrast values for pixels in an image.

The histogram calculator 110 receives the long exposure brightness and the short exposure brightness for the same image, and calculates the histogram for each of the histograms. The histogram is used to calculate the shutter speed for determining the long exposure and the short exposure. Used. That is, the feedback is made, through which the optimum long exposure / short exposure degree for the same image can be determined. Of course, the shutter speed for the first long exposure brightness and the short exposure brightness input should be set to a predetermined value. In other words, when capturing the same image for the first time (the user performs one imaging for one image but the camera performs two long-exposure / short-exposure images), a long exposure shutter speed and a short exposure shutter speed of a predetermined value are set. An exposure / short exposure video signal is received and the luminance signal is separated and used as an input signal. The input signal is processed into a histogram and used to calculate the shutter speed of the exposure calculator 120. The shutter speed calculated as this is the optimal long exposure / short exposure shutter speed for the corresponding image. For example, in calculating the shutter speed related to the histogram, it may be desirable to prepare a preset table so that the related shutter speed is extracted from the table according to the histogram.

The preset value may take an average value (which may be the camera default setting value) of various imaging environments measured experimentally, or may be a recent long exposure / short exposure shutter speed captured by a user. In addition, both may be supported, in which case the option may be selected by the user.

The exposure calculator 120 calculates and sets a shutter speed so as to optimally extract two long / short exposure images for WDR during imaging. As described above, the histogram calculator 110. Calculate and set the shutter speed using the histogram calculated in.

Looking at the above embodiment, in order to extract the optimal image for the same image or image, a total of four imaging should be performed for the same image. The first two images are taken to correct (optimize) the long exposure / short exposure shutter speed, and the second two images are optimized according to the corrected shutter speed. In this case, the user has to focus on the same image for a long time compared with a camera in general, which may cause user dissatisfaction. Therefore, it is desirable to provide a solution to solve this problem. For example, two images of the same image, that is, a first image without a shutter speed correction and a second image with a shutter speed correction By displaying all of them, the user can utilize them when checking and editing the imaging result. In addition, when the WDR function is not required according to the present embodiment, the user does not perform the shutter speed correction, that is, the user is provided with an option to perform the imaging (the conventional WDR function) only at the preset shutter speed. It is desirable to be able to selectively use the function.

As described above, the method of optimizing and extracting the long-exposure / short-exposure image, which is an object of image synthesis, among the improved WDRs aimed at this embodiment has been described. Next, the method of processing the extracted and acquired long exposure / short exposure image will be described.

5 is a block diagram illustrating a preferred image synthesizing apparatus for processing a long exposure / short exposure image or a conventional conventional long exposure / short exposure image extracted by the embodiment of FIG. 4.

Referring to FIG. 5, the apparatus for synthesizing an image according to the present exemplary embodiment calculates a long exposure / short exposure luminance level conversion table based on the long exposure / short exposure histogram according to the embodiment of FIG. 4. The apparatus further includes a composition parameter calculator 130 including a calculation unit and a color difference weight calculator configured to calculate a long exposure / short exposure color difference weight based on the long exposure / short exposure / histogram.

The luminance level conversion table calculator calculates and creates a table used for luminance level conversion of long exposure / short exposure calculated by the histogram calculator. The video signal can be expressed in RGB, which is expressed in three basic colors, and is generally converted to Ycbcr, which is expressed in luminance and color difference, so as to be suitable for data processing, compression, and transmission. That is, although not shown in the drawing, an electrical signal of an image to be captured is received from an image sensor, and converted into luminance and color difference, and the luminance and color difference are processed separately. The shutter speed described in FIG. 4 considers only luminance because the luminance is more sensitive to human eye characteristics than the color difference.

The synthesis parameter calculator 130 includes a luminance level conversion table calculator that calculates a table corresponding to a parameter for luminance, and a color difference weight calculator that calculates a color difference weight. Means that. The term "luminance level" can be easily regarded as a luminance level. Since the change of the luminance level is sensitive to the human vision, the processing of the brightness level is separated so that the brightness conversion at the time of synthesizing the long-exposure / short-exposure image is different from the real image. To be consistent. That is, the luminance level conversion table calculator is configured to perform the composition of the luminance separately from the color difference when synthesizing the long-exposure / short-exposure image.

The processing of the table rather than the luminance level conversion is because the luminance levels of the long exposure / short exposure images are different. When the long-exposure image and the short-exposure image are synthesized by converting to the optimum luminance level through the luminance level conversion table calculator, the cloudy state of the synthesized image, which has been previously problematic, is improved.

The color difference weight calculator calculates a weight to be applied to the color difference of the long exposure / short exposure calculated by the histogram calculator.

The color difference affects the contrast. According to the conventional simple synthesis method, the color difference acquired from the captured image is used as it is. This method may seem to match the real image at first glance, but in reality the contrast is lowered compared to the real image because the color difference is different from the real image at the time of synthesis.

Therefore, it is necessary to apply a correction to the color difference prior to the synthesis of the long exposure / short exposure, and the correction is made by the weight, and the weight is derived by the color difference weight calculation unit. The chrominance weight calculation unit processes only the chrominance without considering luminance, which is in line with what is intended to be performed by dividing the long-exposure / short-exposure image into luminance and color difference as described above.

As such, the luminance level conversion table and the color difference weight to be used for the synthesis of the long-exposure / short exposure image are processed by a synthesis unit to be connected to an output terminal of the synthesis parameter calculator. An example of the synthesis unit is illustrated in FIG. 6.

For reference, Y is luminance, C is color difference, Y _L is brightness of long exposure image, Y _S is brightness of short exposure image, C _L is color difference of long exposure image, and C _S is brightness of short exposure image. Indicates.

Referring to FIG. 6, a luminance synthesizer configured to perform a luminance level conversion on the long exposure luminance signal and the short exposure luminance signal according to the long exposure / short exposure luminance level conversion table, and synthesize the luminance exposure unit, a long exposure color difference signal, and a short exposure. It can be seen that the synthesizer 140 further includes a color difference synthesizer configured to apply the weight to the color difference signal.

The luminance combining unit converts the luminance signal of the long-exposure image and the luminance signal of the short-exposure image according to the luminance level conversion table calculated by the luminance level conversion table calculator to generate and output the synthesized luminance signal. As shown in FIG. 7, a long exposure brightness level converting unit for converting a brightness level of a long exposure image, a short exposure brightness level converting unit for converting a brightness level of a short exposure image, and an adder for synthesizing the outputs of the two level converting units ( It consists of +).

The long exposure brightness level converting unit and the short exposure brightness level converting unit convert the brightness level based on the brightness level conversion table output from the brightness level conversion table calculator. The brightness level conversion table is provided for each of the long exposure and the short exposure, and may be represented as a table or a graph.

FIG. 9 is a graph illustrating short exposure in the luminance level conversion table, and SCYOMAX (Short Channel Y (luminance) Out MAXimum) is the maximum value of the short exposure image luminance output, and is represented by Equation 1 below. It depends on the ratio RR.

RR: Real Ratio

SCYOMAX: Short Channel Y Out Maximum

Since it has an exponential relationship as shown in FIG. 10 showing the relationship between the maximum exposure brightness (SCYOMAX) and the WDR ratio (RR) of the short-exposure image, the maximum exposure value (SCYOMAX) of the short exposure is large when the RR value is low, and the RR is large. When the value is high, the change range is small. When RR is 1.0, SCYOMAX = 0 under normal illuminance conditions, which means that short exposure is ignored. On the contrary, when RR is very large, back light is severe, so the maximum value of short exposure (SCYOMAX) is increased. Increase the share of short exposures.

9, SCYIMIN (Short Channel Y (luminance) In MINimum) is a short exposure input luminance minimum value and corresponds to the last luminance level (SCLP, Short Channel Last Pixel) of the histogram of the short exposure. It is expressed as

SCYIMIN: Short Channel Y In Minimum

SCLP: Short Channel Last Pixel

SCYIMAX (Short Channel Y (Luminance) In MAXimum) is a short exposure input luminance maximum value, which is a value obtained by dividing the maximum luminance level (L) by RR, which is the actual ratio of WDR, and is expressed by Equation 3 below.

SCYIMAX: Short Channel Y Out Maximum

RR: Real Ratio

L: Max Y Level

When the above description is applied to the short-exposure brightness level converting section, the output brightness is 0 when the input brightness is SCYIMIN or less, and the output brightness is SCYOMAX e when SCYIMAX or more. The output luminance between SCYIMIN and SCYIMAX is linearly mapped to a value between 0 and SCYIMAX.

FIG. 11 is a graph illustrating long exposure in the luminance level conversion table, and LCYOMAX (Long Channel Y (Luminity) Out MAXimum) is the maximum value of the long exposure luminance level output, as shown in Equation 4 below. It is expressed by subtracting SCYOMAX, which is the maximum value of the short exposure luminance level output from (L). For reference, in FIG. 11, L means Max Y Level.

LCYOMAX: Long Channel Y Out Maximum

L: Max Y Level

SCYOMAX: Short Channel Y Out Maximum

LCYIMIN (Long Channel Y (Luminance) In MINimum) is the minimum value of the long exposure luminance input value, which is a value that properly scales down the long exposure histogram starting luminance level (LCFP) and is expressed as in Equation 5 below. do.

LCYIMIN: Long Channel Y In Minimum

LCFP: Long Channel First Pixel

k ₁ = 0.5 to 1.0

LCYIMAX (Long Channel Y (Luminance) In MAXimum) is the maximum value of the long exposure luminance input value, which properly scales up the last luminance level (LCLP) of the long exposure histogram and is expressed as in Equation 6 below.

LCYOMAX: Long Channel Y In Maximum

LCLP: Long Channel Last Pixel

k ₂ = 1.0 to 2.5

Referring to FIG. 9 according to the above description, a value of 0 is output when the input luminance is smaller than LCYIMIN, and an LCYOMAX value is output when the input luminance is larger than LCYIMAX. When the value is between LCYIMIN and LCYIMAX, the convex function is convex upward, and can be expressed by the following equation.

LCYO: Long Channel Y Out

LCYI: Long Channel Y In

k ₃ : 1 ~ 3.0

The color difference synthesis unit illustrated in FIG. 6 applies a predetermined weight to the color difference signal of the long exposure image and the color difference signal of the short exposure image, and generates and outputs a synthesized color difference signal. As shown in FIG. 8, the long exposure / short exposure color difference is shown. A normalizer for normalizing each of the signals, a weighting unit for weighting the normalized color difference signals, an adder (+) for synthesizing the weighted long-exposure / short-exposure color difference signals, and denormalizing the synthesis results It consists of a denormalization part that denormalizes.

As shown in Equation 8, color difference signals are generated by subtracting luminance from R (red) and B (blue) of long-exposure images and short-exposure images, respectively, and the process includes subscript L and short exposure. Similar to the RGB to Ycbcr conversion except for the subscript S, which means.

Each color difference signal generated by Equation 8 is normalized to a corresponding luminance signal (Y _S or Y _L ), multiplied by weights, and then summed again to obtain a synthesized color difference signal. Expression (9) can be expressed.

Cr _SNY , Cb _SNY : Synthesized Color Difference Signal

Y _L , Y _S : Long exposure luminance, Short exposure luminance signal

Y _SYN : Synthesized Luminance Signal

W _L , W _S : Long Exposure Weight, Short Exposure Weight

At this time, the weight is determined by the color difference weight calculator in the synthesis parameter calculator 130, and has the following characteristics.

The weight W _L for the long exposure chrominance signal lowers the weight when the long exposure luminance signal Y _L is large, and ignores the color information because the luminance signal is large because it is a saturated region. On the contrary, the weight W _S for the short exposure color difference signal lowers the weight when the level of the short exposure luminance signal Y _S is low.

In other words, in the long exposure image, the weight difference is increased for the low luminance, and in the short exposure image, the weight difference is increased for the high luminance. This is to prevent the cloudy phenomenon and the lowering of the contrast. Considering the long exposure image for which the acquisition of the low luminance image is aimed at and the short exposure image for the purpose of obtaining the high luminance image acquisition, the processing for the color difference in the luminance region of interest is enhanced. For example, when the indoor and outdoor images coexist, the long-exposure video signal with high weight for low luminance and the short-exposure video signal with low weight for low luminance are synthesized. The result is that the image of the long exposure image will be highlighted, which means that the representation of the dark room will be clearer than the outdoor place.

Meanwhile, in the above-described embodiment, since the long-exposure / short-exposure image is synthesized by separating the luminance and the chrominance separately, the composition is performed by the pixel unit rather than the predetermined region for partitioning the image. The distinction between the boundary lines between objects in the image is unambiguous and can be expressed close to reality. That is, according to the present exemplary embodiment, a method of inducing synthesis between pixels instead of a region replacement method in which a long exposure is saturated, a short exposure is dark, and a short exposure is replaced by a high brightness region and a long exposure low brightness region is used. By adopting this, the problem of ambiguity of boundary division that occurs in the area replacement method does not appear. In addition, it is a matter of course that an image that is closer to the actual synthesis of such pixels can be obtained.

13 shows an example of a conventional general image, a conventional WDR image, and a WDR image according to the present embodiment.

In the normal image, the upper part of the color plate is saturated and appears white, while the interior part, which is the lower part of the color difference, appears black.

In the conventional WDR image, the indoor display is improved compared to the general image, but appears darker than the actual image, is bluish (especially the border line), and the color is not good. In contrast, in the WDR image according to the present embodiment, the luminance of the indoor and the outdoor appears close to the actual image, and if the blurring phenomenon does not appear, it can be seen that the color contrast between the boundary lines is clearly shown.

It is expected to be applicable to all camera technologies requiring WDR. Especially, it will be useful for surveillance cameras that are installed indoors and should capture both indoors and outdoors. In addition, by not requiring a high performance image sensor, application to a small terminal device such as a mobile phone is also expected to be no problem.

1 is a block diagram showing the configuration of a conventional WDR implementation camera.

Figure 2 is a schematic diagram showing an image of an outdoor image brighter than the indoor ecology including a part of the room in a dark room with a normal camera.

Figure 3 is a schematic diagram showing a long exposure / short exposure image and a composite image of a conventional WDR implementation camera.

4 is a block diagram showing an image synthesizing apparatus according to an exemplary embodiment of the present invention.

FIG. 5 is a block diagram illustrating a preferred image synthesizing apparatus for processing a long exposure / short exposure image extracted by FIG. 4 or a general conventional long exposure / short exposure image. FIG.

FIG. 6 is a block diagram illustrating a configuration including a synthesis unit in FIG. 5. FIG.

FIG. 7 is a block diagram showing a luminance combining unit in the combining unit of FIG. 6; FIG.

FIG. 8 is a block diagram illustrating a color difference combining unit in the combining unit of FIG. 6. FIG.

9 is a graph illustrating short exposure in a brightness level conversion table.

Fig. 10 is a graph showing the relationship between the short-exposure video luminance output maximum value SCYOMAX and the WDR ratio RR.

11 is a graph illustrating long exposure in a luminance level conversion table.

12 is a graph showing the characteristics of weights used in the chrominance combining unit.

13 is a picture showing a WDR applied image by a normal image, a conventional WDR applied image, the image synthesizing apparatus according to the present invention.

110 Histogram calculator 120 Exposure calculator

130 Synthesis parameter calculator 140 Synthesis unit

Claims (4)

A histogram calculator configured to receive long exposure luminance and short exposure luminance signals and calculate long exposure and short exposure histograms, respectively; And And an exposure calculator configured to calculate a long exposure shutter speed and a short exposure shutter speed based on the long exposure and short exposure histograms. The method according to claim 1, A luminance level conversion table calculator for calculating a long exposure and a short exposure luminance level conversion table based on the long exposure and short exposure histograms, and a long exposure and short exposure color difference weight based on the long exposure, short exposure, and histograms And a composition parameter calculator including a color difference weight calculation unit configured to calculate a color difference. The method according to claim 2, A luminance synthesizer configured to perform a luminance level conversion on the long exposure luminance signal and the short exposure luminance signal according to the long exposure and short exposure luminance level conversion table, and to combine the weighted values to a long exposure color difference signal and a short exposure color difference signal; And a synthesizer including a color difference synthesizer configured to apply the synthesized image. The method according to claim 3, The chrominance combining unit is arranged at a front end of the weight to normalize the input long-exposure color difference signal and the short-exposure color difference signal, and a denormalization unit for denormalizing again after combining by applying the weight. Image synthesizing apparatus comprising a.

Images