JP2010187350A - Image capturing system, video signal processing program, and image capturing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image capturing system in which tone correction is improved. <P>SOLUTION: A second exposure condition different from a first exposure condition set when acquiring a first video signal is set to acquire a second video signal. The first video signal is separated into a first luminance signal and a first color difference signal, and the second video signal is separated into a second luminance signal and a second color difference signal. Then, a tone changing curve for making an amplification factor of a signal value indicating a region other than a black region larger than an amplification factor of a signal value indicating the black region in the first video signal is created based on the first color difference signal of the first video signal and the second color difference signal of the second video signal to correct the tone of the first video signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging system, a video processing signal program, and an imaging method.

  Currently, in the signal processing of an imaging system such as a digital still camera or a video camera, a process of gradation conversion of a video signal is often performed. As one method of gradation conversion processing, there is a method of performing gradation conversion by calculating a correction amount based on a luminance component and a color difference component of a video signal.

For example, in Patent Document 1, a target area is detected from an input video signal, and a correction table is created based on the frequency of a luminance component and a color difference component in each of the target area and the non-target area, and gradation conversion is performed. Thus, each of the target area and the non-target area can be appropriately subjected to gradation conversion.
JP 2007-124604 A

  However, in the above invention, when there are a dark region including a color difference signal component and a black region including almost no color difference signal component in the video signal, the black region is affected by the signal processing of the imaging system, noise in the dark portion, and the like. Since there is a possibility that no difference occurs in the color difference component between the dark area and the dark area, the same gradation correction is performed in both areas. As a result, black appears in the black area, resulting in a sharp image. The black region is a region including a region showing black and a region showing a color close to black. In addition, the dark region is a region that exhibits a color different from a region that originally exhibits black or a color close to black. The dark area is an area that becomes dark due to, for example, shade and does not show the original color.

  The present invention has been made in view of such problems, and by performing gradation correction by distinguishing dark areas and black areas based on color difference components of a plurality of video signals whose exposures have been changed, It is an object of the present invention to provide an imaging system capable of performing optimum gradation correction.

  An aspect of the present invention is an imaging system that performs gradation conversion and output of a video signal, and is different from an imaging unit that acquires the video signal and a first exposure condition when the first video signal is acquired. An exposure condition setting unit for setting a second exposure condition, and a second video signal obtained by separating the first video signal into a first luminance signal and a first color difference signal and obtained by the second exposure condition On the basis of the first color difference signal and the second color difference signal, and the black portion in the first video signal or the second video signal based on the first color difference signal and the second color difference signal. Based on the gradation conversion curve, a gradation conversion curve setting unit for setting a gradation conversion curve for setting the amplification factor of the signal value indicating a region other than the black region to be larger than the amplification factor of the signal value indicating the region, A gradation correction unit that performs gradation correction of the first video signal or the second video signal.

  Another aspect of the present invention is a video signal processing program that performs gradation conversion of a video signal, and a computer obtains a first acquisition procedure for acquiring a first video signal photographed under a first exposure condition; A second acquisition procedure for acquiring a second video signal imaged under a second exposure condition different from the first exposure condition; a first luminance signal; a first color difference signal; And the first video signal based on the separation procedure for separating the second video signal into the second luminance signal and the second color difference signal, and the first color difference signal and the second color difference signal. Or a gradation conversion curve setting procedure for setting a gradation conversion curve for increasing the amplification factor of the signal value indicating a region other than the black region, rather than the amplification factor of the signal value indicating the black region in the second video signal; Based on the gradation conversion curve, the gradation compensation of the first video signal or the second video signal is performed. And gradation correction procedure for, is the execution.

  Yet another aspect of the present invention is an imaging method for converting a gradation of a video signal and outputting the image signal, wherein the first exposure condition is different from the first exposure condition when the video signal is acquired and the first video signal is acquired. 2 exposure conditions are set, the first video signal is separated into the first luminance signal and the first color difference signal, and the second video signal acquired by the second exposure condition is used as the second luminance signal. And a second color difference signal, a correction coefficient for gradation correction is calculated based on the first color difference signal and the second color difference signal, and the first color difference signal and the second color difference signal are calculated. Based on the above, a gradation conversion curve for increasing the amplification factor of the signal value indicating the region other than the black region is larger than the amplification factor of the signal value indicating the black region in the first video signal or the second video signal. Based on the gradation conversion curve, gradation correction of the first video signal or the second video signal is performed.

  According to these aspects, based on the color difference signals in the first video signal and the second video signal, the amplification factor of the signal value indicating the region other than the black region is higher than the amplification factor of the signal value indicating the black region. By setting a gradation conversion curve that increases the image quality, a dark area and a black area in the video signal can be accurately distinguished, and optimal gradation conversion can be performed.

  According to the present invention, gradation conversion can be improved.

  The configuration of the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram of an imaging system according to the first embodiment.

  The imaging system of the first embodiment includes a lens system 100, a diaphragm 101, a color filter 102, a CCD 103 (imaging unit), an AE sensor (light intensity detection unit) 104, an A / D 105, a buffer 106, An exposure control unit (exposure condition setting unit) 107, a signal processing unit 108, a color difference signal separation unit (separation unit) 109, a correction coefficient calculation unit 110, a correction unit 111, a compression unit 112, and an output unit 113 A control unit 114 and an external I / F unit 115.

  The exposure control unit 107 is connected to the diaphragm 101, the CCD 103, and the AE sensor 104. The signal processing unit 108 is connected to the color difference signal separation unit 109, the correction coefficient calculation unit 110, and the correction unit 111. The color difference signal separation unit 109 is connected to the correction coefficient calculation unit 110. The correction coefficient calculation unit 110 is connected to the correction unit 111. The correction unit 111 is connected to the compression unit 112. A control unit 114 such as a microcomputer is bi-directionally connected to the exposure control unit 107, the signal processing unit 108, the color difference signal separation unit 109, the correction coefficient calculation unit 110, the correction unit 111, and the compression unit 112. Further, the external I / F unit 115 is also connected to the control unit 114 in both directions.

  The lens system 100 includes a plurality of lenses. The diaphragm 101 adjusts the amount of light reaching the light receiving surface of the CCD 103 per unit time.

  The CCD 103 is an RGB primary color single-plate CCD. A color filter 102 is disposed closer to the subject than the CCD 103. In this embodiment, the charge accumulation time (exposure time) is controlled by controlling the shutter speed of the electronic shutter. However, the exposure time may be controlled using a mechanical shutter.

  The lens system 100, the diaphragm 101, the color filter 102, and the CCD 103 are arranged along the optical axis.

  The A / D 105 converts an analog video signal photographed via the CCD 103 into a digital video signal. The digital video signal is transferred to the buffer 106. Further, the video signal in the buffer 106 is transferred to the signal processing unit 108.

  The exposure control unit 107 will be described with reference to FIG. FIG. 2 is a block diagram of the exposure control unit 107.

  The exposure control unit 107 includes a histogram calculation unit 200, an exposure calculation unit (cumulative value calculation unit, exposure value calculation unit) 201, and a parameter control unit 202. The histogram calculation unit 200 is connected to the exposure calculation unit 201, and the exposure calculation unit 201 is connected to the parameter control unit 202. The AE sensor 104 is connected to the histogram calculation unit 200. The control unit 114 is bi-directionally connected to the histogram calculation unit 200, the exposure calculation unit 201, and the parameter control unit 202. The parameter control unit 202 is connected to the diaphragm 101 and the CCD 103.

  The histogram calculation unit 200 calculates a histogram of luminance values detected by the AE sensor 104 during shooting. The histogram calculated by the histogram calculation unit 200 is, for example, as shown in FIG.

The exposure calculation unit 201 calculates the average value M ₁ of the luminance values of the video signal from the histogram calculated by the histogram calculation unit 200. Then, the exposure value EV ₁ ^ISO of the first video signal is calculated by equation (1).

It is assumed that the aperture value when the first video signal is acquired is F ₁ , the exposure time is T ₁ , and the ISO sensitivity is ISO. In addition, bit is bit precision when converting a video signal into a digital signal. i and p are coefficients for converting the video signal into an EV value. Hereinafter, the video signal obtained under the first exposure condition with the aperture value F ₁ and the exposure time T ₁ is referred to as a first video signal.

  Further, the exposure calculation unit 201 integrates the frequency of the number of pixels whose luminance value is equal to or less than a predetermined threshold value (first predetermined value) t as the integrated value A1 based on the histogram. The threshold value t is a preset value.

  When the integrated value A1 is equal to or greater than a predetermined value (second predetermined value), the average value is moved so that the luminance value increases by Δm. The predetermined value is a value set in advance. In the present embodiment, when the integrated value A1 is larger than the predetermined value, it is determined that the captured image includes many black areas or many dark areas. FIG. 3B shows a histogram obtained by moving the histogram in FIG. 3A by Δm. Δm is set so that the integrated value A2 of the frequency of the number of pixels that is equal to or less than the threshold t is smaller than a predetermined value.

When moved by Δm, the relationship between the aperture value F ₂ and the exposure time T ₂ that become the exposure value EV ₁ ^ISO can be expressed as in Expression (2).

  When Expression (2) is transformed, it can be expressed as Expression (3).

The exposure value EV ₂ ^ISO of the expression (3) is an exposure in which the integrated value A2 is equal to or less than a predetermined value, the average value of the luminance value is M ₁ , the aperture value is F ₂ , and the exposure time is T ₂ Value. The exposure value EV ₂ ^ISO is smaller than the exposure value EV ₁ ^ISO . Therefore, even when the same subject is photographed, the image obtained by photographing under the condition of the expression (3) is brighter than the image obtained by photographing under the condition of the expression (1). Become.

The exposure calculation unit 201 calculates the aperture value F ₂ and the exposure time T _{2 at} the exposure value EV ₂ ^ISO based on the formula (3).

The parameter control unit 202 controls the aperture value of the aperture 101 and the shutter speed of the CCD 103 so that the aperture value F ₂ and the exposure time T ₂ calculated by the exposure calculation unit 201 are obtained.

As described above, the exposure control unit 107 sets the second exposure condition such that the aperture F ₂ and the exposure time T ₂ are set for the first video signal, and the aperture value of the aperture 101 and the shutter of the CCD 103. Acquire a video signal shot by controlling the speed. Hereinafter, the video signal obtained under the second exposure condition is referred to as a second video signal. In addition, when the video signal that does not distinguish the first video signal and the second video signal is described, it is referred to as a video signal.

  The signal processing unit 108 reads a single-panel video signal on the buffer 106 based on the control of the control unit 114. Then, known interpolation processing (demosaicing processing), white balance processing, and the like are performed, and RGB full-color video signals are generated at each pixel. Among the generated video signals, the first video signal is transferred to the color difference signal separation unit 109, the correction coefficient calculation unit 110, and the correction unit 111. Further, the second video signal is transferred to the color difference signal separation unit 109 and the correction coefficient calculation unit 110.

  The color difference signal separation unit 109 separates the video signal transferred from the signal processing unit 108 into a luminance signal and a color difference signal. For example, the calculation is performed as shown in the following formula (4).

  Here, RGB represents an RGB value of the video signal, Y represents a luminance signal, and Cb and Cr represent color difference signals. The color difference signal obtained in the color difference signal separation unit 109 is transferred to the correction coefficient calculation unit 110.

  The correction coefficient calculation unit 110 will be described with reference to FIG. FIG. 4 is a block diagram of the correction coefficient calculation unit 110.

  The correction coefficient calculation unit 110 includes a frame memory 301, a movement amount calculation unit 302, a coefficient calculation unit (first alignment unit) 304, and a frame memory 303. The frame memory 301 is connected to the movement amount calculation unit 302. The movement amount calculation unit 302 and the frame memory 303 are connected to the coefficient calculation unit 304. The signal processing unit 108 is connected to the frame memory 301. The color difference signal separation unit 109 is connected to the frame memory 303. The control unit 114 is bi-directionally connected to the movement amount calculation unit 302 and the coefficient calculation unit 304. The coefficient calculation unit 304 is connected to the correction unit 111.

  The frame memory 301 stores the video signal transferred from the signal processing unit 108.

  The frame memory 303 stores the color difference signal transferred from the color difference signal separation unit 109.

  The movement amount calculation unit 302 extracts the first video signal and the second video signal accumulated in the frame memory 301. Then, the movement amount of the second video signal with respect to the first video signal is calculated. As a method for calculating the movement amount, a known motion vector, block matching, or the like is used.

  The coefficient calculation unit 304 extracts a color difference signal from the frame memory 303. Then, based on the movement amount calculated by the movement amount calculation unit 302, coordinate conversion of the second color difference signal with respect to the first color difference signal is performed, and alignment is performed. For example, it is assumed that the movement amount of a in the x-axis direction and b in the y-axis direction is calculated when the horizontal direction when the camera is in the normal position is the x direction and the vertical direction is the y direction. In this case, the second color difference signal is translated by -a in the x-axis direction and by -b in the y-axis direction. As a result, the first color difference signal and the second color difference signal are aligned. In addition to translation, coordinate conversion processing such as rotation, enlargement, and reduction is possible.

The coefficient calculation unit 304 calculates the correction coefficient α for each corresponding pixel after performing alignment. The correction coefficient α is calculated as a ratio between the saturation S ₁ calculated from the first color difference signal and the saturation S ₂ calculated from the second color difference signal, for example, as shown in Equation (5). The

Here, k is a normalization coefficient, C _r1 and C _b1 are first color difference signals, and C _r2 and C _b2 are second color difference signals. However, it is assumed that the exposure value EV ₁ ^ISO is larger than the exposure value EV ₂ ^ISO .

The correction coefficient α is calculated from the difference between the saturation S ₁ calculated from the first chrominance signal and the saturation S ₂ calculated from the second chrominance signal, as shown in Equation (6). Also good.

  The correction coefficient α calculated by the coefficient calculation unit 304 is transferred to the correction unit 111.

  The correction unit 111 will be described with reference to FIG. FIG. 5 is a block diagram of the correction unit 111.

  The correction unit 111 includes a gradation conversion curve creation unit (tone conversion curve setting unit) 401 and a conversion curve application unit (tone correction unit) 403. The gradation conversion curve creation unit 401 is connected to the conversion curve application unit 403. The correction coefficient calculation unit 110 is connected to the gradation conversion curve creation unit 401. The signal processing unit 108 is connected to the conversion curve application unit 403. The control unit 114 is bidirectionally connected to the gradation conversion curve creation unit 401 and the conversion curve application unit 403. The conversion curve application unit 403 is connected to the compression unit 112.

  The tone conversion curve creation unit 401 creates a tone conversion curve using the correction coefficient α transferred by the correction coefficient calculation unit 110. The gradation conversion curve is created using, for example, Expression (7). x is a signal value before gradation correction, and y is a signal value after gradation correction.

  Here, since x is an 8-bit signal, 255 is used as a coefficient for normalizing the gradation conversion curve.

  The conversion curve application unit 403 performs tone correction on each of the RGB values of the first video signal transferred by the signal processing unit 108 using the tone conversion represented by Expression (7).

And the saturation S ₁ of the first video signal, and the saturation S ₂ of the second video signal, if but substantially equal to, the correction coefficient α becomes substantially 1 by equation (5). Therefore, the signal value hardly changes before and after tone correction using the tone conversion curve shown in Expression (7). That is, in a black region where the saturation does not change due to exposure, the amplification factor of the input signal value by gradation correction is small, and the input signal value and the output signal value are almost equal even when gradation correction is performed.

On the other hand, a saturation S ₁ of the first video signal, if the saturation S ₂ of the second video signal, there is a difference, the correction coefficient α becomes 1 or more by Equation (5). Therefore, according to the equation (6), the signal value y after gradation correction is larger than the signal value x before gradation correction, and the signal value in the dark region is amplified. That is, in a dark region where the saturation changes depending on the exposure condition, the amplification factor of the input signal value by gradation correction is large, and the input signal value is amplified and output by gradation correction. The first video signal whose tone has been corrected is transferred to the compression unit 112.

  The compression unit 112 performs a known compression process such as JPEG on the first video signal after gradation correction transferred from the conversion curve application unit 403, and transfers it to the output unit 113.

  The output unit 113 records and stores the compressed signal on a memory card or the like, or displays the first video signal after gradation correction on an external display.

  The external I / F 115 includes a power switch, a shutter button, and an interface for switching various modes during shooting.

  This embodiment is premised on processing by hardware, but is not necessarily limited to such a configuration. For example, a configuration in which the video signal from the CCD 103 is output as raw data to which header information such as the video signal size and shooting conditions (exposure, focal length) and the like is added and processed by software separately is also possible. When processing is performed by software, for example, the CPU included in the control unit 114 executes arithmetic processing based on a control program stored in the memory, whereby the signal processing unit 108 and the like function.

  A case where processing is performed by software will be described with reference to a flowchart of FIG.

  In step S1, header information is read.

  In step S <b> 2, the first video signal is acquired and stored in the buffer 106. Further, when the second video signal is acquired, the second video signal is stored in the buffer 106.

  In step S3 corresponding to the signal processing unit 108, an RGB full-color video signal is created.

  In step S4 corresponding to the color difference signal separation unit 109, the first video signal is separated into a first luminance signal and a first color difference signal. The second video signal is separated into a second luminance signal and a second color difference signal.

  In step S5 corresponding to the correction coefficient calculation unit 110, the correction coefficient α is calculated based on the first color difference signal and the second color difference signal.

  In step S6 corresponding to the correction unit 111, a gradation conversion curve is calculated based on the correction coefficient α, and gradation correction is performed on the first video signal using the gradation conversion curve.

  In this embodiment, tone correction is performed on the first video signal, but tone correction may be performed on the second video signal.

  Further, although the aperture value and shutter speed are set as the exposure conditions and the exposure is controlled, the present invention is not limited to this, and the exposure may be controlled by adjusting the ISO sensitivity, the strobe and the like. Further, the second exposure condition is set by changing the predetermined exposure value, for example, by widening the aperture value by a predetermined value or by decreasing the shutter speed by a predetermined value with respect to the first exposure condition. Also good.

  The effect of 1st Embodiment of this invention is demonstrated.

  A first video signal obtained by photographing under a first exposure condition is separated into a first luminance signal and a first color difference signal. Further, the second video signal obtained by photographing under a second exposure condition different from the first exposure condition is separated into a second luminance signal and a second color difference signal. Based on the first color difference signal and the second color difference signal, a gradation conversion curve is set to increase the amplification factor of the signal value indicating a region other than the black region than the amplification factor of the signal value indicating the black region. . Then, tone correction is performed on the first video signal based on the tone conversion curve. Thereby, the gradation correction can be performed by distinguishing between the black area and the dark area in the first video, and a sharp gradation correction can be performed.

A correction coefficient α is calculated based on the saturation S ₁ calculated from the first color difference signal and the saturation S ₂ calculated from the second color difference signal, and a gradation conversion curve is set based on the correction coefficient α. To do. By setting the gradation conversion curve based on the saturation that changes depending on the exposure condition, the gradation correction can be performed by distinguishing between the black area and the dark area in the first video.

  Further, the movement amount of the second video signal with respect to the first video signal is calculated based on the first color difference signal and the second color difference signal. Then, based on the calculated movement amount, the first color difference signal and the second color difference signal are aligned, and a correction coefficient α is calculated for each aligned pixel. As a result, for example, even when camera shake, subject blur, or the like occurs, an accurate correction coefficient α can be calculated for each pixel. Therefore, the gradation correction of the first video signal can be performed accurately.

Further, imaging is performed under the first exposure condition, and a histogram of the first video signal detected by the AE sensor 104 is calculated. In the histogram, the frequency of the number of pixels whose luminance value is equal to or less than a predetermined threshold value t is integrated as an integrated value A1. Then, when the integrated value A1 is equal to or greater than the predetermined value, a second exposure condition that sets an exposure value EV ₂ ^ISO smaller than the exposure value EV ₁ ^{ISO based} on the _first exposure condition is set. Then, the second video signal is acquired according to the second exposure condition. A gradation in which the dark area and the black area in the first image are accurately distinguished by performing gradation correction based on the second video signal and the first video signal acquired in this way. Correction can be performed.

  Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 7 is a block diagram of an imaging system according to the second embodiment.

  The second embodiment further includes a correction area selection unit 1000. The same code | symbol is attached | subjected about the structure same as 1st Embodiment. In the following, a different part from 1st Embodiment is demonstrated. The correction area selection unit 1000 is connected to the correction coefficient calculation unit 120.

The first color difference signal separated by the color difference signal separation unit 109 is transferred to the correction area selection unit 1000. When the saturation S ₁ calculated from the first color difference signal satisfies Expression (8), it is determined that the area is a correction area. That is, the correction area selection unit 1000 selects an area having a low saturation S ₁ as the correction area.

Th is a preset value (third predetermined value). The region where the saturation S ₁ satisfies the equation (8) is a region including gray with low saturation, that is, a dark region or a black region. Information about the correction area selected by the correction area selection unit 1000 is transferred to the correction coefficient calculation unit 120.

In the second embodiment, it is determined whether the saturation S ₁ calculated from the first color difference signal satisfies Expression (8) and the correction region is selected. ₂ may be calculated to determine whether or not Expression (8) is satisfied.

  The correction coefficient calculation unit 120 is different from the first embodiment in that information regarding the correction area is transferred to the coefficient calculation unit 305 as shown in FIG. The coefficient calculation unit 305 calculates the correction coefficient α for the correction region selected by the correction region selection unit 1000 when calculating the correction coefficient α.

  As a result, the correction unit 111 performs gradation correction only on the correction region selected by the correction region selection unit 1000. Therefore, the area for gradation correction can be reduced, and high-speed processing can be performed.

  The correction area selection unit 1000 is premised on processing by hardware, but can also be configured to perform processing by software as in the first embodiment.

  The case where the process in 2nd Embodiment is performed by software is demonstrated using the flowchart of FIG.

  In step S11, header information is read.

  In step S <b> 12, the first video signal is acquired and stored in the buffer 106. Further, when the second video signal is acquired, the second video signal is stored in the buffer 106.

  In step S13 corresponding to the signal processing unit 108, an RGB full-color video signal is created.

  In step S14 corresponding to the color difference signal separation unit 109, the first video signal is separated into a first luminance signal and a first color difference signal. The second video signal is separated into a second luminance signal and a second color difference signal.

In step S15 corresponds to the correction region selection unit 1000, based on the saturation S ₁ calculated from the first color difference signal, for selecting the correction region.

  In step S16 corresponding to the correction coefficient calculation unit 120, a correction coefficient α is calculated for the selected correction area based on the first color difference signal and the second color difference signal.

  In step S17 corresponding to the correction unit 111, a gradation conversion curve is calculated based on the correction coefficient α, and gradation correction is performed on the first video signal using the gradation conversion curve.

  The effect of 2nd Embodiment of this invention is demonstrated.

An area where the saturation S ₁ calculated from the first color difference signal is determined to be a dark area is selected as a correction area, and the correction coefficient α is calculated only for the correction area. As a result, gradation correction can be performed only on the selected correction region, and high-speed processing can be performed.

  Next, a third embodiment of the present invention will be described with reference to FIG. FIG. 10 is a block diagram of an imaging system according to the third embodiment.

  The third embodiment is different from the first embodiment in a signal processing unit 130, a color difference signal separation unit 131, and a correction coefficient calculation unit 132. The same code | symbol is attached | subjected about the structure same as 1st Embodiment. In the following, a different part from 1st Embodiment is demonstrated.

  The signal processing unit 130 generates a video signal. Of the generated video signals, the first video signal is transferred to the color difference signal separation unit 131 and the correction unit 111. Further, the second video signal is transferred to the color difference signal separation unit 131.

  The color difference signal separation unit 131 separates the video signal transferred from the signal processing unit 108 into a luminance signal and a color difference signal. The separated luminance signal and color difference signal are transferred to the correction coefficient calculation unit 132.

  The correction coefficient calculation unit 132 will be described with reference to FIG. FIG. 11 is a block diagram of the correction coefficient calculation unit 132. The correction coefficient calculation unit 132 includes a frame memory 501, a movement amount calculation unit (second alignment unit) 502, a frame memory 503, and a coefficient calculation unit 504. The frame memory 501 is connected to the movement amount calculation unit 502. The movement amount calculation unit 502 and the frame memory 503 are connected to the coefficient calculation unit 504. The color difference signal separation unit 109 is connected to the frame memory 501 and the frame memory 503. The coefficient calculation unit 504 is connected to the correction unit 111. The control unit 114 is bi-directionally connected to the movement amount calculation unit 502 and the coefficient calculation unit 504.

  The frame memory 501 stores the luminance signal transferred from the color difference signal separation unit 109.

  The frame memory 503 stores the color difference signal transferred from the color difference signal separation unit 109.

  The movement amount calculation unit 502 extracts the first luminance signal and the second luminance signal from the frame memory 501 and calculates the movement amount of the second luminance signal with respect to the first luminance signal. As a method for calculating the movement amount, a known motion vector, block matching, or the like is used.

  The coefficient calculation unit 504 extracts the first color difference signal and the second color difference signal from the frame memory 503, and based on the movement amount calculated by the movement amount calculation unit 502, the second color difference with respect to the first color difference signal. Perform signal alignment.

  The coefficient calculation unit 504 calculates a correction coefficient α for each corresponding pixel after performing alignment.

  Note that, similarly to the first embodiment, a configuration of processing by software is also possible. When processing by software, the same processing as the flowchart shown in FIG. 6 of the first embodiment is performed.

  The effect of the third embodiment of the present invention will be described.

  By calculating the amount of movement using the luminance signal and performing alignment, alignment suitable for human visual characteristics can be performed. Therefore, an ideal correction coefficient α can be calculated, and gradation correction can be performed in accordance with human visual characteristics between dark areas and black areas.

  Next, a fourth embodiment of the present invention will be described with reference to FIG. FIG. 12 is a block diagram of an imaging system according to the fourth embodiment.

  The fourth embodiment is different from the second embodiment in a signal processing unit 140, a color difference signal separation unit 141, and a correction coefficient calculation unit 142. The same code | symbol is attached | subjected about the structure same as 2nd Embodiment. In the following, a different part from 2nd Embodiment is demonstrated.

  The signal processing unit 140 generates a video signal. Among the generated video signals, the first video signal is transferred to the color difference signal separation unit 141 and the correction unit 111. Further, the second video signal is transferred to the color difference signal separation unit 141.

  The color difference signal separation unit 141 separates the video signal transferred from the signal processing unit 140 into a luminance signal and a color difference signal. The separated luminance signal and color difference signal are transferred to the correction coefficient calculation unit 142. Further, the separated first color difference signal is transferred to the correction area selection unit 1000.

  The correction coefficient calculation unit 142 will be described with reference to FIG. FIG. 13 is a block diagram of the correction coefficient calculation unit 142.

  The correction coefficient calculation unit 142 includes a frame memory 501, a movement amount calculation unit 502, a frame memory 503, and a coefficient calculation unit 505. The correction coefficient calculation unit 142 of the fourth embodiment is different from the correction coefficient calculation unit 132 of the third embodiment in the coefficient calculation unit 505. The same code | symbol is attached | subjected about the same structure of 3rd Embodiment. In the following, a configuration different from that of the third embodiment will be described.

  The coefficient calculation unit 505 extracts the first color difference signal and the second color difference signal from the frame memory 503, and based on the movement amount calculated by the movement amount calculation unit 502, the second color difference with respect to the first color difference signal. Perform signal alignment.

  The coefficient calculation unit 505 calculates a correction coefficient α for each corresponding pixel in the region selected by the correction region selection unit 1000 after performing alignment. The calculated correction coefficient α is transferred to the correction unit 111.

  Note that, similarly to the second embodiment, a configuration in which processing is performed by software is also possible. When processing by software, the same processing as the flowchart shown in FIG. 9 of the second embodiment is performed.

  The effect of 4th Embodiment of this invention is demonstrated.

Since alignment is performed using the luminance signal, alignment suitable for human visual characteristics can be performed, and an ideal correction coefficient α can be calculated. A region where the saturation S _{1 of} the first color difference signal is determined to be a dark region is selected as a correction region, and the correction coefficient α is calculated only for the correction region. As a result, gradation correction can be performed only on the selected correction region, and high-speed processing can be performed.

  Next, a fifth embodiment of the present invention will be described with reference to FIG. FIG. 14 is a block diagram of the correction unit 600 of the fifth embodiment.

  The fifth embodiment is different from the first embodiment in the correction unit 600. The same code | symbol is attached | subjected about the structure same as 1st Embodiment. Here, the correction unit 600 will be described.

  The correction unit 600 includes an LUT (lookup table) 601 and a conversion curve application unit 602. The LUT 601 is connected to the correction coefficient calculation unit 110 and the conversion curve application unit 602.

  The LUT 601 sets a gradation conversion curve based on the correction coefficient α calculated by the correction coefficient calculation unit 110. The LUT 601 stores a plurality of gradation conversion curves corresponding to various correction coefficients α. The gradation conversion curve has gradation conversion characteristics that can obtain the same effects as those of the first embodiment. In the LUT 601, when the correction coefficient α is close to 1, the gradation conversion characteristic is selected such that the output signal value does not change much with respect to the input signal value. When the correction coefficient α is greater than 1, the gradation conversion characteristic is dark. A signal whose output signal value is amplified with respect to the input signal value in the region is selected.

  The effect of 5th Embodiment is demonstrated.

  By selecting a gradation conversion curve based on the correction coefficient α by the LUT 601, the same effect as in the first embodiment can be obtained.

  Note that the fifth embodiment may be applied to the imaging system of the second to fourth embodiments.

  Further, the processing in the above-described embodiment may be executed by an apparatus other than the imaging system, for example, a personal computer. In this case, the control program is stored in a storage medium such as a ROM, for example, and the CPU executes arithmetic processing based on the control program, thereby functioning as a color difference signal separation unit or the like.

  It goes without saying that the present invention is not limited to the above-described embodiments, and includes various modifications and improvements that can be made within the scope of the technical idea.

It is a block diagram in the imaging system of a 1st embodiment of the present invention. It is a block diagram of the exposure control part of 1st Embodiment. It is a figure which shows the example of the histogram calculated by the histogram calculation part. It is a block diagram of the correction coefficient calculation part of 1st Embodiment. It is a block diagram of the correction | amendment part of 1st Embodiment. It is a flowchart in the case of performing the process of 1st Embodiment by software. It is a block diagram in the imaging system of 2nd Embodiment of this invention. It is a block diagram of the correction coefficient calculation part of 2nd Embodiment. It is a flowchart in the case of performing the process of 2nd Embodiment by software. It is a block diagram in the imaging system of 3rd Embodiment of this invention. It is a block diagram of the correction coefficient calculation part of 3rd Embodiment. It is a block diagram in the imaging system of 4th Embodiment of this invention. It is a block diagram of the correction coefficient calculation part of 4th Embodiment. It is a block diagram of the correction | amendment part of 5th Embodiment of this invention.

101 Aperture 103 CCD (imaging unit)
104 AE sensor (light intensity detector)
107 Exposure control unit (exposure condition setting unit)
108, 130, 140 Signal processing unit 109, 131, 141 Color difference signal separation unit (separation unit)
110, 120, 132, 142 Correction coefficient calculation unit 111, 600 Correction unit 114 Control unit 200 Histogram calculation unit 201 Exposure calculation unit (cumulative value calculation unit, exposure value calculation unit)
202 Parameter control unit 302, 502 Movement amount calculation unit 304, 305, 504, 505 Coefficient calculation unit (first alignment unit, second alignment unit)
401 gradation conversion curve creation section (gradation conversion curve setting section)
403, 602 Conversion curve application unit (gradation correction unit)
601 LUT part 1000 Correction area selection part

Claims (17)

An imaging system that converts a gradation of a video signal and outputs the image signal,
An imaging unit for obtaining the video signal;
An exposure condition setting unit that sets a second exposure condition different from the first exposure condition at the time of acquiring the first video signal;
The first video signal is separated into a first luminance signal and a first color difference signal, and the second video signal acquired by the second exposure condition is used as a second luminance signal and a second color difference signal. A separation part that separates into
Based on the first color difference signal and the second color difference signal, an amplification factor of a signal value indicating a black area in the first video signal or the second video signal is higher than that of the black area. A gradation conversion curve setting unit for setting a gradation conversion curve for increasing an amplification factor of a signal value indicating a region;
An imaging system comprising: a gradation correction unit that performs gradation correction of the first video signal or the second video signal based on the gradation conversion curve. A correction area selection unit that selects a correction area based on the first color difference signal or the second color difference signal;
The gradation correction unit performs gradation correction of the first video signal or the second video signal on the correction area based on the gradation conversion curve. The imaging system described in 1. A correction coefficient calculation unit that calculates a correction coefficient for gradation correction based on the saturation calculated from the first color difference signal and the saturation calculated from the second color difference signal;
The imaging system according to claim 1, wherein the gradation conversion curve setting unit sets the gradation conversion curve based on the correction coefficient. A first alignment unit configured to align the first video signal and the second video signal;
The correction coefficient calculation unit includes a saturation calculated from the first color difference signal and the second color difference signal after alignment is performed using the first video signal and the second video signal. The imaging system according to claim 3, wherein the correction coefficient is calculated on the basis of the saturation calculated from. A second alignment unit configured to align the first luminance signal and the second luminance signal;
The correction coefficient calculation unit includes the saturation calculated from the first color difference signal and the second color difference signal after alignment is performed using the first luminance signal and the second luminance signal. The imaging system according to claim 3, wherein the correction coefficient is calculated on the basis of the saturation calculated from.   The correction coefficient calculation unit calculates the correction coefficient based on a ratio or difference between saturation calculated from the first color difference signal and saturation calculated from the second color difference signal. The imaging system according to any one of claims 3 to 5. It has a light intensity detector that detects the brightness of the subject,
The exposure condition setting unit
A histogram calculation unit that calculates the luminance of the first video signal under the first exposure condition detected by the light intensity detection unit as a histogram;
In the histogram, a cumulative value calculation unit that calculates a cumulative value of the signal value of the first video signal whose luminance is equal to or less than a first predetermined value;
The exposure value calculation part which calculates an exposure value so that the said cumulative value may be below a 2nd predetermined value, and sets the said 2nd exposure condition, It has any 1 to 6 characterized by the above-mentioned. The imaging system according to one.   The correction area selecting unit sets, as the correction area, an area where the saturation calculated from the first color difference signal or the saturation calculated from the second color difference signal is equal to or less than a third predetermined value. The imaging system according to claim 2.   An electronic apparatus comprising the imaging system according to claim 1. A video signal processing program for gradation conversion of a video signal,
On the computer,
A first acquisition procedure for acquiring a first video signal imaged under a first exposure condition;
A second acquisition procedure for acquiring a second video signal imaged under a second exposure condition different from the first exposure condition;
A separation procedure for separating the first video signal into a first luminance signal and a first color difference signal, and separating the second video signal into a second luminance signal and a second color difference signal;
Based on the first color difference signal and the second color difference signal, an amplification factor of a signal value indicating a black area in the first video signal or the second video signal is higher than that of the black area. A gradation conversion curve setting procedure for setting a gradation conversion curve for increasing an amplification factor of a signal value indicating a region;
A video signal processing program for executing a gradation correction procedure for performing gradation correction of the first video signal or the second video signal based on the gradation conversion curve. In the computer,
Based on the first color difference signal or the second color difference signal, a correction area selection procedure for selecting a correction area is executed,
11. The gradation correction procedure performs gradation correction of the first video signal or the second video signal on the correction area based on the gradation conversion curve. The video signal processing program described in 1. In the computer,
A correction coefficient calculation procedure for calculating a correction coefficient for gradation correction based on the saturation calculated from the first color difference signal and the saturation calculated from the second color difference signal;
12. The video signal processing program according to claim 10, wherein the gradation conversion curve setting procedure sets the gradation conversion curve based on the correction coefficient. In the computer,
Performing a first alignment procedure for aligning the first video signal and the second video signal;
The correction coefficient calculation procedure is based on the first color difference signal and the second color difference signal after alignment is performed using the first video signal and the second video signal. The video signal processing program according to claim 12, wherein the correction coefficient is calculated. In the computer,
Performing a second alignment procedure for aligning the first luminance signal and the second luminance signal;
The correction coefficient calculation procedure is based on the first color difference signal and the second color difference signal after alignment is performed using the first luminance signal and the second luminance signal. The video signal processing program according to claim 12, wherein the correction coefficient is calculated.   The correction coefficient calculation procedure calculates the correction coefficient based on a ratio or difference between a saturation calculated from the first color difference signal and a saturation calculated from the second color difference signal. The video signal processing program according to any one of claims 12 to 14.   The correction area selecting means uses the area where the saturation calculated from the first color difference signal or the saturation calculated from the second color difference signal is equal to or less than a third predetermined value as the correction area. The video signal processing program according to claim 11. An imaging method for converting a gradation of a video signal and outputting it,
Obtaining the video signal;
Setting a second exposure condition different from the first exposure condition at the time of acquiring the first video signal;
The first video signal is separated into a first luminance signal and a first color difference signal, and the second video signal acquired by the second exposure condition is used as a second luminance signal and a second color difference signal. And separated into
Calculating a correction coefficient for gradation correction based on the first color difference signal and the second color difference signal;
Based on the first color difference signal and the second color difference signal, an amplification factor of a signal value indicating a black area in the first video signal or the second video signal is higher than that of the black area. Set the gradation conversion curve to increase the amplification factor of the signal value indicating the area,
An imaging method, wherein gradation correction of the first video signal or the second video signal is performed based on the gradation conversion curve.

Images