JP2006324789A - Method and apparatus for processing video signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make the signal value of the primary color signal outputted from a linear matrix circuit monotonously increase or nearly constant. <P>SOLUTION: For example, a 3×3 matrix is used for arithmetic processing of the linear matrix circuit 12. Non-diagonal components of the matrix have negative values or values smaller than matrix coefficients of diagonal components. Matrix coefficients of the non-diagonal components are multiplied by a gain (k). The value of the gain (k) is set to 0 until any of a plurality of chrominance signals inputted to the linear matrix circuit 12 gets saturated. Thus, the value of the gain (k) is set to 0 and then the non-diagonal components of the matrix are 0, so that the signal value of the primary color signal which is obtained through matrix arithmetic processing and outputted by the linear matrix circuit 12 monotonously increase or is nearly constant. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a video signal processing method and a video signal processing device that adaptively control matrix coefficients of a linear matrix circuit provided in the video signal processing device, for example.

  In recent years, an increase in the number of pixels of an image sensor has been remarkable, and the resolution of the image sensor has reached a level having smooth graininess that cannot be determined by human eyes. Under these circumstances, the demand for image quality is shifting to color reproducibility, noise reduction, and dynamic range expansion. In particular, the present invention provides means and apparatus for improving color reproducibility.

  In order to faithfully reproduce the color of a subject, a technique called linear matrix processing using a linear matrix circuit has been conventionally known. In the linear matrix processing, primary signal RGB signal values are generated by performing matrix conversion on the output signal of the image sensor that is linear in luminance.

  Further, in the imaging apparatus, the diaphragm provided in the optical system of the imaging apparatus is controlled using primary color RGB signal values generated by linear matrix processing. The primary color RGB signal value created by linear matrix processing is detected to obtain a signal value indicating brightness, and the opening / closing of the aperture of the imaging device is controlled according to this signal value, and the amount of light incident on the imaging device is adjusted. Is called.

  FIG. 6 shows an example of spectral characteristics of an image sensor having three color RGB (red, green, blue) color filters. FIG. 7 shows spectral characteristics (color matching functions) of RGB signals that match human visibility characteristics. 6 and 7, the horizontal axis indicates the wavelength of light, and the vertical axis indicates the relative sensitivity.

  Focusing on the spectral characteristics of the R signal in FIG. 7, the relative sensitivity of the R signal is negative in the vicinity of a wavelength of 500 nm. On the other hand, the spectral characteristic of the R signal of the image sensor has no negative region as is apparent from FIG. That is, the spectral sensitivity of the image sensor cannot be made negative, and it is physically impossible to realize ideal spectral characteristics with the image sensor. Therefore, spectral characteristics having positive spectral sensitivity are used as ideal spectral characteristics. As described above, when the spectral characteristic of the image sensor is different from the ideal spectral characteristic, spectral correction is performed by linear matrix processing.

Expression (1) is an example of processing by a linear matrix circuit. In the processing by the linear matrix circuit, for example, a primary calculation process is performed using a 3 × 3 (row × column) matrix. In Expression (1), the input of the linear matrix circuit is indicated as R _in , G _in , and B _in, and the output is indicated as R _out , G _out , and B _out .

Since it is necessary to maintain the white balance even after the calculation according to Expression (1), each coefficient of the matrix (hereinafter referred to as matrix coefficient) is set so as to satisfy the relationship of Expressions (2) to (4) below. .
a + b + c = 1 (2)
d + e + f = 1 (3)
g + h + i = 1 (4)

  A value obtained from an experiment or a value determined from experience is fixedly assigned to the matrix coefficient. In general, in a 3 × 3 matrix, matrix coefficients a, matrix coefficients e, and matrix coefficients i that are matrix coefficients in a diagonal direction (hereinafter collectively referred to as “diagonal components”) are positive values. The Further, matrix coefficients b, matrix coefficients c, matrix coefficients d, matrix coefficients f, matrix coefficients g, matrix coefficients, which are matrix coefficients other than diagonal components (hereinafter collectively referred to as “non-diagonal components”). In many cases, h is a negative value or a value smaller than the value of the matrix coefficient of the diagonal component.

  Expression (5) shows an example of processing by a matrix circuit to which a specific matrix coefficient is assigned. A positive value (for example, 1.06 or 1.20) is set for the matrix coefficient of the diagonal component, and a negative value (for example, −0.09 or −0.17) is set for the matrix coefficient of the non-diagonal component. ) Or a smaller value (for example, 0.03) than the matrix coefficient of the diagonal component.

When calculation is performed using the matrix coefficients exemplified in Expression (5), the following Expressions (6) to (8) are obtained.
R _out = 1.06R _in −0.09G _in + 0.03B _in (6)
G _out = −0.03R _in + 1.20G _in −0.17B _in (7)
_{B out = -0.01R in -0.04G in +} 1.05B in ··· (8)

  In recent years, it has been proposed to adaptively determine a matrix coefficient in accordance with imaging conditions without setting the matrix coefficient to a fixed value, and details thereof are described in Patent Document 1 below.

JP 2004-208079 A

  By the way, when the brightness of light incident on the image sensor exceeds a certain level, the image sensor is saturated. In the case of an imaging device including a color filter, the value of the incident light amount at which the RGB color signal is saturated differs because the sensitivity of the pixel differs with respect to the wavelength. Which RGB color signal is saturated earliest depends on the characteristics of the light source and the subject. For example, when a white subject is photographed with a general light source such as sunlight, the G signal is saturated earliest. When a red subject is photographed, the R signal is saturated most quickly.

For example, the description will be made assuming that the G signal is saturated earliest. The signal value of the color signal G _in the brightness of the incident light is input to the linear matrix circuit as the increase increases, the signal value of an output of the linear matrix circuit color signals G _out is increased monotonically by equation (7) To do. However, the timing of the color signal G _in is saturated (hereinafter, appropriately referred to as the saturation point) signal values of the color signal G _in exceeds but not change, since the brightness of incident light is increased, the saturation point The signal values of the color signal R _in and the color signal B _in that have not reached 1 increase. Since the coefficients of the color signal R _in and the color signal B _in are negative values, the signal value of the color signal G _out obtained by Expression (7) starts to decrease after the saturation point of the color signal G _in .

As described above, the signal value of the color signal _Gout that should originally increase monotonously with the increase in light brightness decreases at the saturation point, so the signal of the color signal _Gout obtained by the equation (7). The value becomes inappropriate, and not only the color of the subject that is actually photographed is not reproduced, but also a color different from the color of the subject may be reproduced.

Further, in the imaging apparatus, a signal value indicating brightness is obtained by detecting the output of the linear matrix circuit, and the opening / closing of the diaphragm is controlled based on the signal value to control the amount of light incident on the imaging element. For example, if the signal value indicating brightness is large, the amount of incident light is large. Therefore, the aperture diameter is reduced to reduce the amount of incident light on the image sensor. Conversely, if the signal value is small, the amount of incident light is small. To increase the amount of light incident on the image sensor. The luminance level Y, which is a signal value indicating brightness, is created by the following equation (9), for example.
Y = 0.3R _out + 0.1B _out + 0.6G _out (9)

As described above, saturated color signal G _in by the brightness of light incident on the imaging element is increased, the signal value of the color signals G _out of should monotonically increase to lead to a decrease. Since the color signal _Gout is multiplied by the largest coefficient in the equation (9), the signal value Y obtained by the equation (9) decreases as the signal value of the color signal _Gout decreases. There is a fear. In other words, the brightness level Y generated from the output of the linear matrix circuit may decrease despite the increase in the brightness of the light incident on the image sensor.

  Since the brightness level Y decreases, the aperture of the diaphragm is increased and control is performed to increase the amount of incident light. As described above, although the brightness of the light incident on the image sensor is actually increased, the aperture of the diaphragm is controlled to be increased, and as a result, the amount of light incident on the image sensor is controlled. There was a problem that it could not be performed properly.

  Accordingly, an object of the present invention is to provide a video signal processing method and a video signal processing apparatus capable of reproducing the color of a subject more faithfully by controlling matrix coefficients.

  In order to solve the above-described problem, according to a first aspect of the present invention, a matrix calculation step for generating a primary color signal by performing matrix calculation on a plurality of input color signals, and a gain k (however, 0 ≦ k ≦ 1) and a calculation step of multiplying a predetermined matrix coefficient used in the matrix calculation step by a gain k. The gain generation step includes a plurality of input color signals. This is a video signal processing method for generating a gain k that is 0 before any of the color signals is saturated.

  According to a second aspect of the present invention, a linear matrix unit that generates a primary color signal by performing a matrix operation on a plurality of input color signals and a gain k (where 0 ≦ k ≦ 1) are generated, A control unit that multiplies a predetermined matrix coefficient of the linear matrix unit by a gain k, and the control unit has a gain k that is set to 0 before any one of the input color signals is saturated. Is a video signal processing device for generating

  According to the present invention, the color of the subject can be reproduced more faithfully. Moreover, according to this invention, the light quantity which injects into an image pick-up element can be controlled appropriately.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In one embodiment, an example in which the present invention is applied to an imaging apparatus will be described.

  FIG. 1 shows an example of a main configuration of an imaging apparatus 1 according to an embodiment of the present invention. The optical system of the image pickup apparatus 1 includes a lens 2, a diaphragm 3, an optical LPF (Low Pass Filter) and an infrared ray removal filter (not shown). The imaging apparatus 1 includes an imaging element 4, a front end 5, a camera signal processing unit 6, a control unit 7, and a drive circuit 8.

  The light condensed by the lens 2 enters the diaphragm 3. The diaphragm 3 adjusts the amount of light collected by the lens 2 and adjusts the amount of light taken into the image sensor 4. The imaging device 4 is configured by, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). On the front surface of the image sensor 4, for example, a three-color color filter in which an R filter that transmits only red light, a G filter that transmits only green light, and a B filter that transmits only blue light are arranged in a checkered pattern. Is provided.

  In the image sensor 4, light incident through the three-color filter is converted into an electric signal by photoelectric conversion, and three types of color signals (R signal, G signal, and B signal) are generated. The color signal generated by the image sensor 4 is supplied to the front end 5.

  In the front end 5, CDS (Correlated Double Sampling) processing (correlated double sampling processing) for removing noise components from the color signal supplied from the image sensor 4, gain control processing for adjusting the signal level, digital signal conversion processing Etc., and image data is obtained.

  Image data output from the front end 5 is supplied to the camera signal processing unit 6. The camera signal processing unit 6 includes, for example, an optical / sensor correction unit 11, a linear matrix circuit 12, a white balance processing unit (WB) 13, a gamma (γ) correction unit 14, a luminance signal (Y) processing unit 15, and color difference signal processing. The unit 16 is included. The camera signal processing unit 6 includes a calculation unit 17 that detects the input of the linear matrix circuit 12 and performs calculation processing, and a calculation unit 18 that detects the output of the linear matrix circuit 12 and performs calculation processing. .

  In the camera signal processing unit 6, various digital signal processes are performed on the color signals constituting the image data supplied from the front end 5. The optical / sensor correction unit 11 performs shading correction for correcting unevenness of brightness generated in the optical system, digital clamp processing for correcting the black level, distortion correction, defective pixel correction, and the like. The linear matrix circuit 12 performs matrix calculation processing to generate RGB primary color signals. The processing of the linear matrix circuit 12 will be described later.

  The white balance processing unit 13 performs processing to adjust the RGB balance and correct the color so as to obtain a correct color. The gamma correction unit 14 performs processing to correct non-linearity of the light emission characteristics of a display device such as a monitor. Done. The color signal output from the gamma correction unit 14 is supplied to a luminance signal (Y) processing unit 15 and a color difference signal (C) processing unit 16. The Y signal processing unit 15 combines the RGB color signals with a predetermined combining ratio to generate a luminance signal, and the C signal processing unit 16 generates a color difference signal Cr and a color difference signal Cb, respectively from the camera signal processing unit 6. Is output.

  Each unit of the imaging apparatus 1 as described above is controlled by the control unit 7 realized by a microcomputer, for example. For example, the control unit 7 sends a control signal S 1 to the drive circuit 8. The drive circuit 8 supplied with the control signal S1 controls the position of the lens 2 and the opening / closing of the diaphragm 3 according to the content of the control signal S1, and adjusts the amount of light taken into the image sensor. The control unit 7 controls the image sensor 4 to control the electronic shutter timing, the front end 5 controls the gain signal S3 to control the gain, and the camera signal processing unit 6 controls the control signal. S4 is sent out.

Further, the detection information S <b> 5 is supplied from the calculation unit 17 to the control unit 7. The detection information S5 is information obtained by detecting color signals input to the linear matrix circuit 12 (hereinafter referred to as color signals R _in , color signals G _in , and color signals B _in ), color signals R _in, the color signal G _in, respectively the information indicating the signal value of the color signal B _in, the color signals R _in, the color signal G _in, the signal value indicating brightness obtained from the color signal B _in (hereinafter, the signal value and the luminance level Y _in the includes information indicating the appropriate designated). The luminance level Y _in is obtained, for example, by calculation using the following formula (10).
Y _in = 0.3 R _in +0.1 B _in +0.6 G _in (10)

Further, the detection information S <b> 7 is supplied from the calculation unit 18 to the control unit 7. The detection information S7 is information obtained by detecting color signals output from the linear matrix circuit 12 (hereinafter referred to as color signals _Rout , color signals _Gout , and color signals _Bout as appropriate). And includes information indicating a signal value indicating brightness obtained from the color signal R _out , the color signal G _out , and the color signal B _out (hereinafter, this signal value is referred to as a luminance level Y _out ). The luminance level Y _out is obtained, for example, by calculation using the following formula (11).
Y _out = 0.3R _out + 0.1B _out + 0.6G _out (11)

  Here, processing performed in the linear matrix circuit 12 and control of the linear matrix circuit 12 by the control unit 7 will be described. The processing described below can be realized not only by hardware but also by software processing. In the present invention, the matrix coefficients of the matrix used in the arithmetic processing by the linear matrix circuit 12 are not fixed values. A predetermined matrix coefficient is multiplied by the gain k generated by the control unit 7. Further, the matrix coefficient can be made variable by adaptively setting the value of the gain k.

  Equation (12) shows an example of processing by the linear matrix circuit 12 in one embodiment of the present invention.

As shown in Equation (12), the matrix coefficient of the non-diagonal component of the matrix is multiplied by the gain k. The value of the gain k is 0 ≦ k ≦ 1. The value of the gain k is controlled to 0 by the saturation point at which any one of the color signal R _in , the color signal G _in , and the color signal B _in input to the linear matrix circuit 12 is saturated. With this control, the amount of light incident on the image sensor increases, and the color signal output from the linear matrix circuit 12 even if any one of the color signals R _in , G _in , and B _in is saturated. R _out , color signal G _out , and color signal B _out can be monotonously increased or maintained substantially constant.

Since the color signal R _out , the color signal G _out and the color signal B _out output from the linear matrix circuit 12 are monotonously increased or substantially constant, the luminance level Y _out detected by the calculation unit 18 also monotonously increases. Accordingly, the luminance level Y _out of the detection information S7 supplied to the control unit 7 increases monotonously, and the control unit 7 sends out a control signal S1 for reducing the aperture of the diaphragm 3. In other words, it is possible to appropriately control the opening / closing of the diaphragm 3 and to appropriately control the amount of light taken into the image sensor 4.

This will be described in more detail. First, the gain k generated by the control unit 7 will be described. The gain k can be obtained by the following equation (13), for example.
k = 1- {max (R, G, B) -threshold} * Gain (13)

FIG. 2 shows an example of the calculation for obtaining the gain k using Expression (13). The calculation shown in FIG. 2 is performed by the control unit 7, for example. In the maximum value calculation process 21 Initially, the color signal R _in included in the detection information S5, supplied to the control unit 7 from the operation unit 17, the color signal G _in, among signal values of the color signal B _in, the maximum signal value Selected. Let the maximum signal value selected be max (R, G, B).

Subsequently, a subtraction process 22 for subtracting the parameter threshold value from max (R, G, B) is performed, and a difference (diff) between max (R, G, B) and the parameter threshold value is obtained. The parameter threshold value is set to an appropriate value according to the luminance level Y _in included in the detection information S5. Expression (14) shows the process performed in the subtraction process 22. In the following formula (14), the parameter threshold is indicated as threshold.
diff = max (R, G, B) −threshold (14)

Subsequently, a zero clip calculation process 23 is performed on the obtained difference (diff). In the 0 clip calculation processing 23, if the difference (diff) is larger than 0, the 0 clip calculation result (diff_cp0) is left as it is, and if the difference (diff) is 0 or less, the 0 clip calculation result (hereinafter referred to as diff_cp0). Is set to 0. Expressions (15) and (16) indicate the processes performed in the 0 clip calculation process 23.
If diff> 0, then diff_cp0 = diff; (15)
If diff ≦ 0, then diff_cp0 = 0; (16)

Subsequently, a multiplication process 24 for multiplying the 0 clip calculation result (diff_cp0) by the parameter gain is performed. The value of the parameter gain is a value between 0 and 1, and is set to an appropriate value according to the luminance level Y _in included in the detection information S5. Expression (17) shows the process performed in the multiplication process 24. In the following equation (17), the parameter gain is indicated as gain, and the processing result of the multiplication processing 24 is indicated as (diff_gn).
diff_gn = diff_cp0 * gain (17)

Subsequently, a one-clip operation process 25 is performed on the result (diff_gn) of the multiplication process 24. In 1-clip operation processing 25, if (diff_gn) is greater than 1, the 1-clip operation result (hereinafter referred to as diff_cp1) is set to 1, and if (diff_gn) is 1 or less, (diff_cp1) is left as it is. Expressions (18) and (19) indicate processes performed in one clip calculation process.
If diff_gn> 0, then diff_cp1 = 1; (18)
If diff_gn ≦ 0, then diff_cp1 = diff_gn; (19)

Subsequently, a subtraction process 26 is performed on the result (diff_cp1) of the one-clip calculation process 25. In the subtraction process 26, an operation of subtracting (diff_cp1) from the constant 1 is performed. Expression (20) shows the subtraction process 26.
k = 1-diff_cp1 (20)

  The value of gain k is calculated by the arithmetic processing described above.

FIG. 3A shows the transition of the value of max (R, G, B) with respect to the luminance level Y _in (horizontal axis) included in the detection information S5. FIG. 3B shows a transition of the value of the gain k with respect to the luminance level Y _in (horizontal axis) included in the detection information S5. 3A and 3B are associated with each point.

The P0 point on the horizontal axis in FIGS. 3A and 3B indicates a saturation point at which any one of the color signal R _in , the color signal G _in , and the color signal B _in input to the linear matrix processing unit is saturated. . Which color signal is saturated depends on the characteristics of the light source and the subject. P1 point indicates the saturation point of the next saturated color signal, and P2 point indicates the saturation point of the last saturated color signal.

For example, when the parameter threshold value is set to Th1 as shown in FIG. 3A, the value of the gain k remains 1 until the A1 point that is a point corresponding to Th1 in FIG. 3B. If the luminance level Y _in further increases after the A1 point, the value of the gain k changes to a value smaller than 1, and the value of the gain k is set to 0 at the P0 point. The slope of the gain k change from the A1 point (gain k = 1) to the P0 point (gain k = 0) can be adjusted by the parameter gain value used in the multiplication process 24 of the calculation process for obtaining the gain k. The

Further, as shown in FIG. 3A, when the parameter threshold value is set to Th2, the value of gain k is set to 1 up to the point A2 corresponding to Th2 in FIG. 3B. A2 points later, when the luminance level Y _in is increased, the value of the gain k is changed to a value smaller than 1, are zero value of the gain k in P0 point. If the parameter threshold value is set to the same value as the saturation value, the gain k changes steeply from the position of the Amax point to the point P0. Thus, by changing the value of the parameter threshold value, a position where the value of the gain k changes from 1 can be set. However, the value of the gain k is set to 0 by the point P0.

The description will be given with reference to equation (12) again. By setting the gain k to 0 by the point P0 that is the saturation point, all the matrix coefficients of the off-diagonal components of the equation (12) after the point P0 are set to 0. Therefore, the following expressions (21) to (23) are established.
R _out = aR _in (21)
G _out = eG _in (22)
B _out = iB _in (23)

  Here, since the matrix coefficient a, the matrix coefficient e, and the matrix coefficient i are generally positive values of 1 or more, the output of the color signal saturated at the point P0 is substantially constant, but regarding the output of other color signals. The monotonic increase will be maintained until saturation. That is, even if a certain color signal is saturated, the output of the color signal from the linear matrix circuit 12 does not decrease as in the prior art.

Furthermore, the color signals R _out the arithmetic unit 18, the color signals G _out, but obtain luminance levels Y _out by detecting a color signal B _out by calculation based on equation (11), the output from the linear matrix circuit 12 is not reduced Therefore, the luminance level Y _out does not decrease. For this reason, the control unit 7 can appropriately control the diaphragm 3 based on the luminance level _Yout, and as a result, can appropriately control the amount of light taken into the image sensor 4.

  Another example of a method for determining the gain k will be described. This other example shows a condition of a value that the gain k can take without setting the gain k to 0 at the P0 point.

The formula of the matrix calculation performed by the linear matrix circuit 12 is expressed by the above-described formula (12). In addition, the luminance level Y _out obtained by detecting the color signal output from the linear matrix circuit 12 is expressed by the above-described equation (11).

Here, the following formula (24) is obtained by substituting the calculation result of formula (12) into formula (11).
Y _out = (0.3a + 0.6 kd + 0.1 kg) R _in + (0.3 kb + 0.6e + 0.1 kh) G _in + (0.3 kc + 0.6 kf + 0.1 i) B _in (24)

Here, one color signal, the here, the color signal G _in is earliest saturated, Y _out after the saturation point of the color signal G _in includes a function determined by the signal values of the color signals R _in the color signal B _in Become.

Since the signal values of the color signal R _in and the color signal B _in are considered to increase as the luminance level Y _in indicating the brightness level of light incident from the lens increases until saturation, the following equation (25) And Formula (26) is materialized.
R _in = l1 * Y _in (25)
B _in = l2 * Y _in (26)

L1 of formula (25) indicates the slope of increase linear color signals R _in to increase with increasing the luminance level Y _in. Similarly, l2 of formula (26) indicates the slope of increase linear color signal B _in to increase with increasing the luminance level Y _in.

By substituting Equation (25) and Equation (26) into Equation (24) and assuming that the color signal G _in is saturated, if the term containing the color signal G _in is a constant C, then Equation (27) is obtained.
Y _out = (0.3a + 0.6 kd + 0.1 kg) l1 * Y _in + (0.3 kc + 0.6 kf + 0.1i) l2 * Y _in + C (27)

In order to always increase Y _out from the equation (27), the condition represented by the following equation (28) must be satisfied.
(0.3a + 0.6kd + 0.1kg) l1 + (0.3kc + 0.6kf + 0.1i) l2 ≧ 0 (28)

  Note that the values of l1 and l2 in Equation (28) are not uniquely determined because they vary depending on the light source and the subject. However, if the light source of the input light input to the imaging apparatus 1 and the reflectance of the subject are known, the value of the gain k can be determined so as to satisfy the condition shown in Expression (28).

  The present invention can be variously modified and applied without departing from the gist of the present invention, and is not limited to the above-described embodiment. For example, although it has been described that three types of color signals are supplied to the matrix circuit 12 of the imaging device 1 in the above-described embodiment, the present invention is applied even when four or more types of color signals are supplied. Can do.

  FIG. 4 shows another example of a color filter provided in the image sensor 4 and shows an example of a color filter 31 of four colors. The color filter 31 includes an R filter that transmits only red light, a B filter that transmits only blue light, a G1 filter that transmits only green light in the first wavelength band, and a second highly correlated with the G1 filter. The G2 filter that transmits only green light in the wavelength band is configured as a minimum unit.

  FIG. 5 shows an example of the spectral sensitivity characteristic of the color filter 31. In FIG. 5, a curve L61 represents the spectral sensitivity of R, and a curve L62 represents the spectral sensitivity of G1. A curve L63 represents the spectral sensitivity of G2, and a curve L64 represents the spectral sensitivity of B. As shown in FIG. 5, the spectral sensitivity curve L62 of G2 has a high correlation with the spectral sensitivity curve L62 of G1. Further, the spectral sensitivity of R, the spectral sensitivity of G (G1, G2), and the spectral sensitivity of B overlap each other in an appropriate range.

  Four color signals (color signal R, color signal G1, color signal G2, and color signal B) are supplied to the front end 5 from the imaging device 4 provided with the four color filters. Color signals that have undergone predetermined signal processing by the front end 5 and the optical / sensor correction unit 11 are input to the linear matrix circuit 12.

In the arithmetic processing by the linear matrix circuit 12 to which four color signals are supplied, a 3 × 4 matrix is used. The following formula (29) shows a 3 × 4 matrix to which the present invention is applied. In the 3 × 4 matrix, the matrix coefficient m ₀ , the matrix coefficient m ₅ , the matrix coefficient m ₆ , and the matrix coefficient m ₁₁ are diagonal components, and the matrix coefficient m ₁ , the matrix coefficient m ₂ , the matrix coefficient m ₃ , and the matrix coefficient m ₄ , matrix coefficient m ₇ , matrix coefficient m ₈ , matrix coefficient m ₉ , and matrix coefficient m ₁₀ are off-diagonal components. The matrix coefficient of the non-diagonal component is multiplied by the gain k. In Expression (29), the color signals input to the linear matrix circuit 12 are indicated as R, G1, G2, and B, and the outputs after the matrix calculation are indicated as R ′, G ′, and B ′.

As shown in Expression (29), the gain k is multiplied by a matrix coefficient other than the matrix coefficient of the diagonal component relating to the main component of each color. For example, the signal value of the color signal R ′ output from the linear matrix circuit 12 is obtained by the following equation (30) when the gain k is not multiplied by a predetermined matrix coefficient of the equation (29).
R ′ = m ₀ R + m ₁ G1 + m ₂ G2 + m ₃ B (30)

In Expression (30), the main component of the output color signal R ′ is R on the right side. The gain k is multiplied by a matrix coefficient m ₁ , a matrix coefficient m ₂ , and a matrix coefficient m ₃ that are matrix coefficients other than the matrix coefficient m _{0 of the} diagonal component related to the color signal R.

The main components of the output color signal G ′ are the color signal G1 and the color signal G2. Accordingly, the gain k is multiplied by the matrix coefficient m ₄ and the matrix coefficient m ₇ which are matrix coefficients other than the matrix coefficient m ₅ and the matrix coefficient m _{6 of the} diagonal components applied to the color signal G1 and the color signal G2.

The main component of the output color signal B ′ is the color signal B. Therefore, the gain k is multiplied by the matrix coefficient m ₈ , the matrix coefficient m ₉ , and the matrix coefficient m ₁₀ that are matrix coefficients other than the matrix coefficient m _{11 of the} diagonal component related to the color signal B.

  The gain k in Expression (29) can be calculated by the same calculation as that described with reference to FIG. That is, the maximum value calculation process 21 selects the maximum signal value from the color signal R, the color signal G1, the color signal G2, and the color signal B, and the subtraction process 22, the 0 clip process 23, the multiplication process 24, and the 1 clip calculation process. 25, the gain k can be calculated by performing the subtraction process 26.

  Similarly to the above-described embodiment, the value of the gain k is set to 0 until the color signal R, the color signal G1, the color signal G2, and the color signal B that are saturated earliest are saturated. By controlling the value of the gain k, the color signal R ′, the color signal G ′, and the color signal B output from the linear matrix circuit 12 even after any one of the four types of input color signals is saturated. 'Can be monotonically increasing or almost constant.

It is a block diagram which shows an example of a structure of the imaging device in one Embodiment of this invention. It is a basic diagram which shows an example of the calculation method of the gain k in one Embodiment of this invention. It is a basic diagram which shows the transition of the gain k with respect to a luminance level. It is a basic diagram which shows the example of a 4 color filter. It is a basic diagram which shows the example of the spectral sensitivity characteristic of a 4 color filter. It is a basic diagram which shows an example of the spectral sensitivity of an image pick-up element. It is a basic diagram which shows an example of a human visual sensitivity characteristic.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 3 Aperture 4 Image sensor 6 Camera signal processing part 7 Control part 17, 18 Calculation part

Claims (4)

A matrix calculation step for generating a primary color signal by performing matrix calculation on a plurality of input color signals;
A gain generation step for generating a gain k (where 0 ≦ k ≦ 1);
A calculation step of multiplying the predetermined matrix coefficient used in the matrix calculation step by the gain k,
The video signal processing method in which the gain generation step generates the gain k that is set to 0 before any one of the input color signals is saturated. In claim 1,
A video signal processing method in which the gain k is multiplied by a matrix coefficient of a non-diagonal component of a matrix in the calculation step. In claim 1,
In the calculation step, the gain k is a non-diagonal component of the matrix and is multiplied by a negative matrix coefficient. A linear matrix unit that generates a primary color signal by performing a matrix operation on a plurality of input color signals;
A control unit that generates a gain k (where 0 ≦ k ≦ 1) and multiplies a predetermined matrix coefficient of the linear matrix unit by the gain k;
The video signal processing apparatus, wherein the control unit generates the gain k that is set to 0 before any one of the input color signals is saturated.

Images