JP2011023774A - Image processing apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of nonlinearly transforming luminance and saturation independently without changing any hue, and capable of improving the image quality when the nonlinear transformation of a relatively dark image is performed. <P>SOLUTION: A transformation coefficient calculation unit 7 calculates a luminance transformation coefficient and a saturation transformation coefficient which should be used for nonlinear transformation processing of each targeted pixel which is a processing target pixel in an image signal. A signal level transforming unit 9 nonlinearly transforms a pixel signal of the targeted pixel on the basis of the luminance transformation coefficient and the saturation transformation coefficient. The signal level transforming unit 9 respectively changes the luminance and the saturation in accordance with the luminance transformation coefficient and the saturation transformation coefficient. Thereby, the luminance and the saturation are separately transformed without changing any hue. A value corresponding to the luminance transformation coefficient and smaller than the luminance transformation coefficient can be obtained as the saturation transformation coefficient. Accordingly, even when performing the nonlinear transformation processing for a dark image, saturation transformation as well as luminance transformation are performed. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that performs nonlinear conversion processing on an image signal.

  Conventionally, an imaging apparatus such as a digital camera includes an image processing apparatus that performs camera signal processing. This type of image processing apparatus is configured to perform nonlinear conversion processing of an image signal. The nonlinear conversion process is generally performed in order to perform gamma processing on an image. The gamma processing is for compensating in advance on the imaging side for non-linearity of the video level with respect to the signal of the receiver. However, the purpose of the nonlinear conversion process is not limited to the above gamma process. For example, the non-linear conversion characteristic of gamma is set in an S-shape, and a specific gradation is compressed or expanded to reduce noise and improve gradation.

  When nonlinear conversion processing is performed on primary color signals that use the three primary colors R, G, and B, if nonlinear signal processing is performed on each of the R, G, and B signals, the ratio of the R, G, and B signals changes. The hue will change.

Therefore, in order to avoid a change in hue, the representative value X is calculated from the R, G, and B signals in the vicinity of the target pixel, and a nonlinear conversion value Gamma (X) with respect to the representative value X is obtained. Further, a value obtained by dividing the nonlinear transformation value Gamma (X) by the representative value X is obtained as a nonlinear transformation coefficient. Then, the non-linear conversion coefficient is integrated with each of the R, G, and B signals. The nonlinear conversion process is expressed as (Equation 1).
Gamma (R) = R x Gamma (X) / X
Gamma (G) = G x Gamma (X) / X
Gamma (B) = B x Gamma (X) / X (Formula 1)

  As a result, the nonlinear conversion process is performed in a state where the signal ratios of R, G, and B are clearly maintained.

In the above nonlinear conversion processing, if the maximum value of R, G, B is used as the representative value X and the gradient of Gamma (X) is always 1 or less, the following (Expression 2) is established.
Gamma (R) ≤ R
Gamma (G) ≤ G
Gamma (R) ≤ B (Formula 2)

  That is, if the above condition is satisfied, the signal after nonlinear conversion becomes smaller than the original signal. However, in other cases, the signal after nonlinear transformation can be larger than the original signal. It is assumed that the signal becomes large due to nonlinear conversion and exceeds a desired upper limit level. At this time, if the upper limit clip is applied only to a signal having a desired upper limit level or more, the ratio of the R, G, and B signals is out of order, which causes a hue change.

  In order to avoid such a situation, the following technique is conventionally proposed in Patent Document 1.

  In this prior art, the luminance conversion calculation is performed when the luminance exceeds the upper limit level. In the luminance conversion calculation, the above-described processing of Expression (1) is performed using the luminance W as the representative value X. As a result, only the luminance is converted without changing the hue and saturation, and is below the upper limit level. FIG. 8 shows a circuit for realizing the luminance conversion calculation. In the figure, Ri, Gi, Bi are signals before conversion, and Ro, Go, Bo are signals after conversion. Each of the R, G, and B channels has a multiplier that multiplies the signal before conversion by a conversion coefficient.

The R, G, and B signals are distributed above and below the luminance W. That is, at least one channel of the R, G, and B signals is larger than the luminance W, and at least one channel is smaller than the luminance W. Therefore, even if the luminance conversion is performed, one or two channels of the R, G, and B signals may exceed the upper limit level. Therefore, in the conventional technique of Patent Document 1, when there is a signal that exceeds the upper limit level after luminance conversion, the following (Equation 3) is performed as the saturation conversion calculation.
Ro = Wi + Kc x (Ri-Wi)
Go = Wi + Kc x (Gi-Wi)
Bo = Wi + Kc x (Bi-Wi)
Wi: Luminance signal
Kc: Saturation conversion coefficient (Formula 3)

  In this saturation conversion calculation, only the color difference signal is multiplied by Kc, so that only the saturation is multiplied by Kc without changing the luminance and hue. This saturation conversion is performed so that the highest signal among R, G, and B is below the upper limit level. FIG. 9 shows a circuit for realizing the luminance conversion calculation. Each of the R, G, and B channels includes a subtracter that subtracts the luminance Wi from the signal before conversion, a multiplier that multiplies the subtraction result by the luminance conversion coefficient Kc, and an adder that adds the luminance signal Wi to the multiplication result. have.

As described above, the technique of Patent Document 1 can independently perform luminance conversion and saturation conversion without changing the hue. Using this, as a first step, the luminance level is suppressed to the upper limit or less by luminance conversion. If there is a channel (R, G, B) that still exceeds the upper limit after luminance conversion, as a second stage, the saturation is converted so that the highest level channel falls below the upper limit. Through these two stages of processing, non-linear processing is performed so that the signals of all channels are below the upper limit level without changing the hue.
JP 9-331539 A (page 8-12, FIG. 6)

  However, the conventional image processing apparatus described in Patent Document 1 performs nonlinear conversion for an image that is too bright such that the signal value exceeds the upper limit level by converting luminance and saturation without changing the hue. However, there is a problem that the image quality may be deteriorated when nonlinear conversion of a relatively dark image is performed. This point will be further described below.

  In the technique of Patent Document 1, when an image is too bright, first, the luminance is converted to an upper limit level or less by luminance conversion. If the signal value still exceeds the upper limit level after the luminance conversion, the saturation conversion is performed. That is, if all the R, G, and B signals after luminance conversion are equal to or lower than the upper limit level, saturation conversion is not performed.

  Here, consider a case where the luminance of a dark image is increased by nonlinear transformation. In this case, it is considered that in the image after luminance conversion, one of the R, G, and B signals often does not exceed the upper limit level. If there is no signal exceeding the upper limit level, the technique of Patent Document 1 does not perform saturation conversion. As described above, the nonlinear conversion according to Patent Document 1 may cause a phenomenon that the image becomes bright but the saturation does not change. When such conversion is performed, the color of the image may become light and the image quality may deteriorate.

  The present invention has been made to solve the conventional problems, and an object of the present invention is to perform non-linear conversion of luminance and saturation independently without changing the hue, and to perform non-linear conversion of a relatively dark image. An object of the present invention is to provide an image processing apparatus capable of improving the image quality in the case.

  The image processing apparatus of the present invention calculates an image signal input unit that inputs an image signal, and a luminance conversion coefficient and a saturation conversion coefficient used for nonlinear conversion processing of each target pixel that is a pixel to be processed in the image signal. A conversion coefficient calculation unit that performs non-linear conversion on the pixel signal of the pixel of interest based on the luminance conversion coefficient and the saturation conversion coefficient, and the luminance conversion coefficient and the saturation conversion coefficient without changing the hue of the image signal A signal level conversion unit that changes luminance and saturation respectively according to the brightness conversion coefficient, the conversion coefficient calculation unit is a value corresponding to the luminance conversion coefficient as the saturation conversion coefficient, and from the luminance conversion coefficient It is configured to obtain a small value.

  With this configuration, a value corresponding to the luminance conversion coefficient that is smaller than the luminance conversion coefficient is obtained as the saturation conversion coefficient. In the non-linear conversion, the luminance and the saturation are converted independently using the luminance conversion coefficient and the saturation conversion coefficient without changing the hue. Therefore, in the present invention, not only luminance conversion but also saturation conversion is performed even when nonlinear conversion processing is performed on a dark image. As a result, it is possible to avoid a situation in which the image quality deteriorates because the saturation is not converted. In addition, since the saturation conversion coefficient is set to a value smaller than the luminance conversion coefficient, the saturation is appropriately adjusted as the luminance increases. In this way, it is possible to provide an image processing apparatus that can perform non-linear conversion of luminance and saturation independently without changing the hue, and improve image quality when performing non-linear conversion of dark images.

  The image processing apparatus of the present invention further includes a representative value calculation unit that calculates a representative value of the target pixel, and the conversion coefficient calculation unit calculates the luminance conversion coefficient and the saturation conversion coefficient from the representative value. . A value representative of a pixel value in a predetermined representative value calculation range including the target pixel may be calculated. With this configuration, the luminance conversion coefficient and the saturation conversion coefficient can be suitably calculated by using the representative value. The representative value is, for example, the maximum value of the R, G, B signal, and is, for example, a luminance value calculated from the R, G, B signal. The representative values are preferably the following values.

  In the image processing apparatus of the present invention, a plurality of maximum value specifying ranges are set so as to be shifted from each other, each of the plurality of maximum value specifying ranges includes the target pixel, and the representative value calculating unit includes: An average of a plurality of maximum values respectively specified from the plurality of maximum value specifying ranges is calculated as the representative value. With this configuration, it is possible to prevent the representative value from changing suddenly when the target pixel is shifted one by one. Since the representative value changes smoothly, the nonlinear conversion coefficient also changes smoothly. Therefore, it is possible to prevent a sudden change in the pixel value in the converted image, suppress false coloring, and improve the image quality.

  The image processing apparatus according to the present invention includes a luminance value calculation unit that calculates a luminance value of the pixel of interest, and the signal level conversion unit based on the luminance value calculated by the luminance value calculation unit. A saturation conversion coefficient correction unit that corrects the saturation conversion coefficient in order to clip the pixel value to a predetermined upper limit level. The luminance value may be calculated from the pixel value in a predetermined luminance value calculation range including the target pixel. With this configuration, it is possible to improve the image quality when nonlinear conversion of a dark image is performed while suppressing the signal level below a desired upper limit level.

  The image processing method of the present invention includes an image signal input step for inputting an image signal, and a luminance conversion coefficient and a saturation conversion coefficient used for nonlinear conversion processing of each pixel of interest that is a pixel to be processed in the image signal. A conversion coefficient calculation step for calculating the luminance conversion coefficient, the saturation conversion coefficient and the saturation conversion coefficient, and a non-linear conversion of the pixel signal of the pixel of interest based on the luminance conversion coefficient and the saturation conversion coefficient, without changing the hue of the image signal A signal level conversion step for changing luminance and saturation according to the conversion coefficient, respectively, wherein the conversion coefficient calculation step sets the saturation conversion coefficient to a value according to the luminance conversion coefficient, The value is smaller than the luminance conversion coefficient. This method also provides the advantages of the present invention described above.

  In the image processing method of the present invention, the luminance value calculation step for calculating the luminance value of the target pixel, and the signal level conversion step based on the luminance value calculated in the luminance value calculation step. A saturation conversion coefficient correction step for correcting the saturation conversion coefficient in order to clip the pixel value of the target pixel to a predetermined upper limit level. This method also provides the advantages of the present invention described above.

  As described above, the present invention obtains a value corresponding to the luminance conversion coefficient that is smaller than the luminance conversion coefficient as the saturation conversion coefficient, and performs nonlinear conversion using these luminance conversion coefficient and saturation conversion coefficient. Provided is an image processing apparatus having an effect that by performing the non-linear conversion of luminance and saturation without changing the hue and improving the image quality when performing non-linear conversion of a dark image by providing a configuration to perform Can do.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  An image processing apparatus according to the first embodiment of the present invention is shown in FIG. The image processing apparatus 1 is provided in an imaging apparatus such as a video camera or a digital camera. The imaging device is a surveillance camera, for example. In FIG. 1, an image processing apparatus 1 is used for an image signal input unit 3 that inputs an image signal, a representative value calculation unit 5 that calculates a representative value of each pixel of interest in the image signal, and a nonlinear conversion process of the pixel of interest. A conversion coefficient calculation unit 7 that calculates a luminance conversion coefficient and a saturation conversion coefficient, and a signal level conversion unit 9 that nonlinearly converts the pixel signal of the pixel of interest based on the luminance conversion coefficient and the saturation conversion coefficient. The conversion coefficient calculation unit 7 includes a luminance conversion coefficient calculation unit 11 and a saturation conversion coefficient calculation unit 13 that calculate a luminance conversion coefficient and a saturation conversion coefficient, respectively.

  The image signal input unit 3 is configured to input an image signal. As described above, in the present embodiment, the image processing apparatus 1 is provided in the imaging apparatus. In this case, an image signal is generated by the imaging device and input by the image signal input unit 3. As image signals, R, G, and B3 primary color pixel signals are input. The image signal is a digital signal that has undergone AD conversion.

  The image processing apparatus 1 according to the present embodiment may be provided in a so-called three-panel camera or a single-panel camera. In the case of a three-plate camera, three image sensors generate R, G, and B signals. Then, R, G, and B signals having the same optical center are input in parallel. In the case of a single-panel camera, R, G, and B color filters are arranged in a mosaic pattern on the image sensor, and R, G, and B signals are generated by one image sensor. Then, the image signal input unit 3 uses the line memory (not shown) to input the R, G, B signals near the target pixel in synchronization.

  The representative value calculation unit 5 obtains a representative value from the R, G, and B3 signals input from the image signal input unit 3. In the image processing apparatus 1, each pixel is sequentially set as a target pixel, and processing of the target pixel is performed. Then, the representative value is also calculated for each target pixel.

  In the case of a single-panel camera, the representative value calculation unit 5 may calculate a representative value representative of the plurality of pixel signals from a plurality of pixel signals near the target pixel. In this process, a representative value calculation range including the target pixel is set in advance, and a representative value is calculated from a plurality of pixel signals included in the representative value calculation range. The representative value calculation range is, for example, a 3 × 3 pixel area centered on the target pixel. The representative value is, for example, the maximum value of the R, G, and B signals within the representative value calculation range. For example, the representative value is a luminance value calculated from R, G, and B signals in the representative value calculation range.

  More preferably, the representative value calculation unit 5 calculates the representative value as follows. In this processing, a plurality of predetermined maximum value specifying ranges that are shifted from each other are used. Each maximum value specifying range is a part of the representative value calculation range, is set so as to include the target pixel and include all R, G, and B pixels. These maximum value specifying ranges are preferably provided so as to make one round around the pixel of interest. More specifically, as in the example described later, a plurality of maximum value specifying ranges are biased in different directions (upper left, upper right, lower left, lower right) with respect to the target pixel so as to include the target pixel in the upper, lower, left, and right corners, respectively. May be set. The representative value calculation unit 5 specifies the maximum values of the R, G, and B signals from each of the plurality of maximum value specifying ranges. Then, the representative value calculation unit 5 uses an average value of a plurality of maximum values specified from a plurality of maximum value specifying ranges as a representative value.

  As a specific example, it is assumed that the representative value calculation range is a 3 × 3 pixel range centered on the target pixel. As the maximum value specifying range, a range of 2 × 2 pixels is suitable. That is, the four maximum value specifying ranges (2 × 2 pixels) are set so as to include the target pixel in the lower right, lower left, upper right, and upper left, respectively. A maximum value is specified from each of these maximum value specifying ranges, and an average of a plurality of specified maximum values is obtained.

  Such a representative value calculation process is advantageous for a single-panel camera or the like. When R, G, and B signals are obtained from a plurality of signals having different optical centers as in a single plate camera, the R, G, and B signals are sequentially arranged. A primary color Bayer array is known as a typical pixel array. According to the above processing, each maximum value specifying range includes at least one of each of R, G, and B (in a Bayer array, one red is one, one blue is a 2 × 2 pixel, 2 greens are included). Therefore, the maximum value of each maximum value specifying range is always a value selected from adjacent R, G, and B signals. And the average of these several maximum values is calculated | required. Since such processing is performed, the representative value of one target pixel and the representative value of the adjacent target pixel do not differ greatly. That is, according to the present embodiment, it is possible to prevent the representative value from changing suddenly when the target pixel is shifted one by one. Thereby, since the representative value changes smoothly, the non-linear conversion coefficient also changes smoothly. Accordingly, it is possible to prevent a sudden change in the pixel value in the image after nonlinear conversion described later, and to suppress false coloring.

  On the other hand, when each pixel of interest has all signals of R, G, and B as in the case of a three-panel camera, the above-described representative value calculation range and maximum value specifying range may not be used. An averaging process of a plurality of maximum values may not be performed. An appropriate representative value such as a maximum value or a luminance value may be obtained from the R, G, and B signals of each pixel of interest.

  In the conversion coefficient calculation unit 7, the luminance conversion coefficient calculation unit 11 and the saturation conversion coefficient calculation unit 13 calculate a luminance conversion coefficient and a saturation conversion coefficient from the representative values. Among these, the luminance conversion coefficient calculation unit 11 calculates a luminance conversion coefficient to be used for the nonlinear conversion processing of the target pixel from the representative value calculated by the representative value calculation unit 5.

FIG. 2 shows an example of the X-Gamma (X) characteristic of nonlinear transformation. X is a representative value, and Gamma (X) is a converted value (value after conversion). In this case, the luminance conversion coefficient kY for the representative value X is obtained by Equation 4.
kY = Gamma (X) / X (Formula 4)

  FIG. 3 shows the relationship between the representative value X and the luminance conversion coefficient kY. In this example, the luminance conversion coefficient kY is large in the region where the representative value X is small. Therefore, when the luminance conversion coefficient kY of FIG. 3 is used, the luminance of the dark part of the image is raised.

  As a specific configuration of the calculation process, the luminance conversion coefficient calculation unit 11 may calculate the luminance conversion coefficient kY from the luminance conversion coefficient characteristics of FIG. Alternatively, a luminance conversion coefficient table corresponding to FIG. 3 may be stored in a storage unit such as a ROM. This table is a table of representative values X and luminance conversion coefficients kY. Then, the luminance conversion coefficient calculation unit 11 reads out the luminance conversion coefficient kY corresponding to the representative value X from the luminance conversion coefficient table.

  Alternatively, the image processing apparatus 1 may store only the luminance conversion coefficient kY corresponding to a plurality of representative values X that are separated from each other. Then, the luminance conversion coefficient kY corresponding to the representative value X that is not stored may be obtained from the stored value by interpolation processing.

  The saturation conversion coefficient calculation unit 13 calculates a saturation conversion coefficient from the representative value. In the present embodiment, the saturation conversion coefficient calculation unit 13 is configured to obtain a value corresponding to the luminance conversion coefficient and smaller than the luminance conversion coefficient as the saturation conversion coefficient.

Specifically, for example, the saturation conversion coefficient calculation unit 13 calculates the saturation conversion coefficient kC from the luminance conversion coefficient kY by the following formula 5.
kC = (n × kY + m) / (n + m) (Formula 5)

  If n = 1 and m = 0, kY = kC, and the luminance conversion coefficient and the saturation conversion coefficient are equal. If n = 0 and m = 1, kC is always 1 and the saturation does not change. When n = 1 and m = 3 are set, the relationship between the luminance conversion coefficient kY and the saturation conversion coefficient kC is as shown in FIG. When the conversion is performed according to the coefficient setting in FIG. 4, the saturation increases to some extent as the luminance increases in the portion where the luminance increases. In particular, in the dark part of the image, the saturation increases moderately as the luminance increases. Thereby, it is possible to avoid a situation in which the saturation does not change even though the brightness is increased and brightened, and it is possible to prevent the image from being unnaturally thinned by nonlinear conversion, and a natural and good image can be obtained.

  In the above example, the saturation conversion coefficient kC is calculated from the luminance conversion coefficient kY. However, the saturation conversion coefficient kC may be calculated from the representative value completely independently of the luminance conversion coefficient kY. However, even in this case, in the calculation result, the saturation conversion coefficient kC must be a value corresponding to the luminance conversion coefficient kY and smaller than the luminance conversion coefficient kY.

  As a specific example of the independent processing, the saturation conversion coefficient may be calculated using a predetermined saturation conversion characteristic in the same manner as the luminance conversion characteristic of FIG. 2 is used for calculation of the luminance conversion coefficient. Further, a saturation conversion coefficient table corresponding to FIG. 4 may be stored in a storage unit such as a ROM. This table is a table of the representative value X and the saturation conversion coefficient kC. Then, the saturation conversion coefficient calculation unit 13 reads out the saturation conversion coefficient kC corresponding to the representative value X from the saturation conversion coefficient table. This saturation conversion coefficient table may be integrated with the above-described luminance conversion coefficient table (even in this case, since kY and kC are read out, the two coefficient calculation processes are independent).

  Similarly to the description of the luminance conversion coefficient calculation process, the image processing apparatus 1 may store only the saturation conversion coefficient kC corresponding to a plurality of representative values X separated from each other. Then, the saturation conversion coefficient kC corresponding to the representative value X that is not stored may be obtained from the stored value by interpolation processing.

  The signal level conversion unit 9 generates the signal (R) of the target pixel based on the luminance conversion coefficient kY and the saturation conversion coefficient kC calculated by the luminance conversion coefficient calculation unit 11 and the saturation conversion coefficient calculation unit 13 as described above. , G, B signals). The signal level conversion unit 9 changes the luminance and saturation according to the luminance conversion coefficient kY and the saturation conversion coefficient kC, respectively. In this processing, the signal level conversion unit 9 converts only the luminance according to the luminance conversion coefficient without changing the hue (that is, keeping the hue constant), and only the saturation according to the saturation conversion coefficient. Convert. Therefore, luminance and saturation are converted independently without changing the hue.

As a specific process, the signal level conversion unit 9 is configured to perform the calculation of the following Expression 6.
Gamma (R) = (0.3kY + 0.7kC) × R + (kY−kC) × 0.6G + (kY−kC) × 0.1B
Gamma (G) = (kY−kC) × 0.3R + (0.6kY + 0.4kC) × G + (kY−kC) × 0.1B
Gamma (B) = (kY−kC) × 0.3R + (kY−kC) * 0.6G + (0.1kY + 0.9kC) × B (Formula 6)

Here, the luminance Y is expressed by Equation 7.
Y = 0.3R + 0.6G + 0.1B (Formula 7)

By using the luminance Y, Gamma (R) is transformed as follows.
Gamma (R) = (0.3kY + 0.7kC) × R + (kY−kC) × 0.6G + (kY−kC) × 0.1B
= KC × R + (kY−kC) × (0.3R + 0.6G + 0.1B)
= KC x R + (kY-kC) x Y
= KY x Y + kC x (R-Y) (Formula 8)

Similarly, Gamma (G) and Gamma (B) are also modified as follows.
Gamma (G) = kY × Y + kC × (G−Y)
Gamma (B) = kY x Y + kC x (B−Y) (Formula 9)

  In these formulas 8 and 9, the luminance is obviously multiplied by kY and the color difference is multiplied by kC. In other words, the nonlinear conversion of Equation 7 can perform the nonlinear conversion of luminance and saturation at independent magnifications without changing the hue.

  The configuration of the image processing apparatus 1 according to the present embodiment has been described above together with the processing performed by each component. Next, the operation of the image processing apparatus 1 will be described with reference to FIG.

  As shown in FIG. 5, when an image signal is input by the image signal input unit 3, the representative value calculation unit 5 calculates a representative value from R, G, and B signals near the target pixel in the image signal (S1). ). As already described in detail, in the preferred process, the maximum value is specified in each of the four areas including the upper left, upper right, lower left, and lower right including the target pixel, and the average of the four maximum values obtained thereby is calculated. The Each area is 2 × 2 pixels, for example.

  Next, the luminance conversion coefficient calculation unit 11 calculates the luminance conversion coefficient kY (S3), and the saturation conversion coefficient calculation unit 13 calculates the saturation conversion coefficient kC (S5). The luminance conversion coefficient kY is obtained from the representative value (FIG. 3). As the saturation conversion coefficient kC, as shown in FIG. 4, a value corresponding to the luminance conversion coefficient kY and smaller than the luminance conversion coefficient kY is obtained. The saturation conversion coefficient kC may be calculated from the luminance conversion coefficient kY or may be directly obtained from the representative value. In the former case, the saturation conversion coefficient kC is indirectly calculated from the representative value via the luminance conversion coefficient kY. These luminance conversion coefficient calculation step and saturation conversion coefficient calculation step correspond to the conversion coefficient calculation step of the present invention.

  Next, the signal level conversion unit 9 performs nonlinear conversion processing using the luminance conversion coefficient kY and the saturation conversion coefficient kC (S7). Here, the above-described processing of (Expression 6) is performed. In a single panel camera, each pixel of interest has only one of R, G, and B signals. In such a case, the signal level conversion unit 9 executes only the conversion formula corresponding to the color of the pixel signal of the target pixel among the three formulas of (Formula 6) as the processing of each target pixel. By this non-linear conversion process, the luminance conversion coefficient is applied to the luminance, and the saturation conversion coefficient is applied to the saturation. Therefore, luminance and saturation can be converted at independent magnifications without changing the hue.

  Here, as shown in FIG. 4, in the present embodiment, the saturation conversion coefficient kC is a value corresponding to the luminance conversion coefficient kY and smaller than the luminance conversion coefficient kY. As the luminance conversion coefficient kY increases / decreases, the saturation conversion coefficient kC also increases / decreases. Therefore, when the luminance is converted, the saturation is also converted at a level smaller than the luminance conversion. It is possible to avoid a situation in which the saturation does not change even though the luminance is increased. In the example of FIG. 4, the brightness of the dark part of the image is greatly increased. At this time, the saturation of the dark part of the image is also raised moderately. Therefore, it is possible to avoid the situation that the color becomes light because the saturation does not change even though the image becomes bright, and the image quality can be improved.

  The image processing apparatus 1 according to the first embodiment of the present invention has been described above. According to the present embodiment, a value corresponding to the luminance conversion coefficient kY that is smaller than the luminance conversion coefficient kY is obtained as the saturation conversion coefficient kC. In the non-linear conversion, the luminance and the saturation are converted independently without changing the hue by using the luminance conversion coefficient kY and the saturation conversion coefficient kC. Therefore, even when nonlinear conversion processing is performed on a dark image, not only luminance conversion but also saturation conversion is performed. As a result, it is possible to avoid a situation in which the image quality deteriorates because the saturation is not converted. Moreover, since the saturation conversion coefficient kC is set to a value smaller than the luminance conversion coefficient kY, the saturation is appropriately adjusted as the luminance increases. In this way, it is possible to provide an image processing apparatus that can perform non-linear conversion of luminance and saturation independently without changing the hue, and improve image quality when performing non-linear conversion of dark images.

  Further, according to the present embodiment, the representative value of the target pixel is calculated, and the luminance conversion coefficient and the saturation conversion coefficient are calculated from the representative value. By using the representative value, it is possible to suitably calculate the luminance conversion coefficient and the saturation conversion coefficient.

  In the present embodiment, the plurality of maximum value specifying ranges are set so as to be shifted from each other, and each of the plurality of maximum value specifying ranges includes a target pixel. The representative value is an average of a plurality of maximum values respectively specified from a plurality of maximum value specifying ranges. By calculating the representative value in this way, it is possible to prevent the representative value from changing suddenly when the target pixel is shifted one by one. Since the representative value changes smoothly, the nonlinear conversion coefficient also changes smoothly. Therefore, it is possible to prevent a sudden change in the pixel value in the converted image, suppress false coloring, and improve the image quality.

  Next, an image processing apparatus according to the second embodiment of the present invention will be described. FIG. 6 shows the image processing apparatus 21 of the present embodiment. In the following, description of matters common to the first embodiment will be omitted, and differences from the first embodiment will be mainly described.

  As shown in FIG. 6, in addition to the configuration of the image processing apparatus 1 of the first embodiment, the image processing apparatus 21 of the present embodiment has a maximum luminance value calculation unit 23 and a saturation conversion coefficient correction unit. 25.

  The maximum value luminance value calculation unit 23 calculates the maximum value and luminance value of the signal of the target pixel. In the maximum value calculation process, a signal having the maximum signal level is selected from R, G, and B signals included in the maximum value selection range including the target pixel. In the luminance value calculation process, the R, G, and B signals in the luminance value calculation range including the target pixel are added at a ratio of 3: 6: 1. The maximum value selection range and the luminance value calculation range may be the same. Further, one or both of the maximum value selection range and the luminance value calculation range may be the same as the above-described representative value calculation range. The maximum value selection range and the luminance value calculation range are, for example, a 3 × 3 pixel range including the target pixel.

  Further, when each pixel of interest has all the signals of R, G, and B as in the case of a three-panel camera, the above-described maximum value selection range and luminance value calculation range need not be used. The maximum value of the R, G, and B signals of each pixel of interest is selected, and the luminance value is calculated from the R, G, and B signals of each pixel of interest.

  The saturation conversion coefficient correction unit 25 calculates an upper limit saturation conversion coefficient from the maximum value and the luminance value and a predetermined upper limit signal value set in advance. The saturation conversion coefficient correction unit 25 compares the saturation conversion coefficient kC calculated by the saturation conversion coefficient calculation unit 13 with the upper limit saturation conversion coefficient. Then, an upper limit clipping process to the upper limit saturation conversion coefficient is performed on the saturation conversion coefficient kC. That is, when the saturation conversion coefficient kC exceeds the upper limit saturation conversion coefficient, the saturation conversion coefficient kC is corrected to the upper limit saturation conversion coefficient. As a result, the pixel value of the target pixel is clipped to the upper limit level also in the nonlinear conversion process of the signal level conversion unit 9 using the saturation conversion coefficient kC.

  The upper limit saturation conversion coefficient is obtained so that the pixel value after conversion does not exceed a predetermined upper limit signal value (upper limit level). The upper limit saturation conversion coefficient kC includes the maximum value and the luminance value obtained by the maximum value luminance value calculation unit 23, the luminance conversion coefficient kY calculated by the luminance conversion coefficient calculation unit 11, and a predetermined upper limit signal set in advance. Calculated from the value. The calculation process will be described below.

If the maximum value of the R, G, and B signals obtained by the maximum luminance value calculation unit 25 is an R signal, the following equation 10 needs to be satisfied.
Upper limit signal value ≥ Gamma (R)
≧ (0.3kY + 0.7kC) × R + (kY−kC) × 0.6G + (kY−kC) × 0.1B
≧ kY × Y + kC × (R−Y) (Formula 10)

By transforming Equation 10, Equation 11 is obtained.
kC ≦ (Upper limit signal value−kY × Y) / (R−Y) (Formula 11)

Similarly, the following equation (12) is obtained for the B and G signals.
kC ≤ (Upper limit signal value-kY x Y) / (B-Y)
kC ≦ (Upper limit signal value − kY × Y) / (G−Y) (Formula 12)

KC in these equations corresponds to the upper limit saturation coefficient. After all, the upper saturation coefficient is expressed as follows.
Upper limit saturation conversion coefficient = (Upper limit signal value−kY × luminance value) / (maximum value−luminance value) (Formula 13)

  By clipping the saturation conversion coefficient kC with the upper limit saturation conversion coefficient, the pixel value in the nonlinear conversion is also clipped to the upper limit signal value. Therefore, it is possible to perform non-linear conversion of luminance and saturation with independent characteristics without causing a hue change, and to suppress the converted signal to a desired upper limit signal value or less.

  FIG. 7 shows the operation of the image processing apparatus 21 according to the present embodiment. Again, differences from the first embodiment will be mainly described. As shown in FIG. 7, in the present embodiment, in addition to the representative value calculation (S1) by the representative value calculation unit 5, the luminance value and the maximum value are calculated by the maximum value luminance value calculation unit 23 (S11, S13). ). Further, in addition to the calculation of the luminance conversion coefficient kY and the saturation conversion coefficient kC by the luminance conversion coefficient calculation unit 11 and the saturation conversion coefficient calculation unit 13 (S3, S5), the saturation conversion coefficient correction unit 25 performs the upper limit saturation conversion. A coefficient is calculated (S15). Then, the saturation conversion coefficient correction unit 25 compares the saturation conversion coefficient kC with the upper limit saturation conversion coefficient, and determines whether or not the saturation conversion coefficient kC is equal to or lower than the upper limit saturation conversion coefficient (S17). If the determination in S17 is Yes, the saturation conversion coefficient kC is used as it is, and the signal level conversion process is performed by the signal level conversion unit 9 (S7). If the determination in S17 is No, the saturation conversion coefficient kC is clipped to the upper limit saturation conversion coefficient (S19). Then, signal level conversion processing is performed using the upper limit saturation conversion coefficient (S7). Thereby, the signal after conversion can be suppressed below a desired upper limit signal value.

  The image processing apparatus 21 according to the second embodiment of the present invention has been described above. According to the present embodiment, the luminance value of the target pixel is calculated. Then, based on the luminance value, the saturation conversion coefficient is corrected in order to clip the pixel value of the target pixel to a predetermined upper limit level in the processing of the signal level conversion unit. Specifically, an upper limit saturation conversion coefficient at which the converted signal is equal to or lower than the upper limit level is calculated, and the saturation conversion coefficient is clipped below the upper limit saturation conversion coefficient. Thereby, the signal level can be suppressed to a desired upper limit level or less. Therefore, it is possible to improve the image quality of a dark image by performing non-linear conversion of luminance and saturation at independent magnifications without causing a hue change, and keep the converted signal below the desired maximum signal value. I can do it.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments, and it goes without saying that those skilled in the art can modify the above-described embodiments within the scope of the present invention.

  As described above, the image processing apparatus according to the present invention can perform nonlinear conversion of luminance and saturation independently without changing the hue, and can improve image quality when performing nonlinear conversion of a relatively dark image. And is useful as an image processing device for an imaging device.

1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention. Diagram showing luminance conversion characteristics as nonlinear characteristics Diagram of nonlinear transformation coefficient divided by luminance transformation coefficient by input signal Diagram showing the relationship between luminance conversion coefficient and saturation conversion coefficient The figure which shows operation | movement of the image processing apparatus in 1st Embodiment. The block diagram of the image processing apparatus in the 2nd Embodiment of this invention The figure which shows operation | movement of the image processing apparatus in 2nd Embodiment. Block diagram showing a luminance conversion arithmetic unit in the prior art Block diagram showing a saturation conversion computing unit in the prior art

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 3 Image signal input part 5 Representative value calculation part 7 Conversion coefficient calculation part 9 Signal level conversion part 11 Luminance conversion coefficient calculation part 13 Saturation conversion coefficient calculation part

Claims (6)

An image signal input unit for inputting an image signal;
A conversion coefficient calculation unit that calculates a luminance conversion coefficient and a saturation conversion coefficient used for nonlinear conversion processing of each target pixel that is a pixel to be processed in the image signal;
The pixel signal of the target pixel is nonlinearly converted based on the luminance conversion coefficient and the saturation conversion coefficient, and the luminance and the saturation are changed according to the luminance conversion coefficient and the saturation conversion coefficient without changing the hue of the image signal. And a signal level conversion unit for changing
The image processing apparatus, wherein the conversion coefficient calculation unit is configured to obtain a value corresponding to the luminance conversion coefficient that is smaller than the luminance conversion coefficient as the saturation conversion coefficient. A representative value calculating unit for calculating a representative value of the target pixel;
The image processing apparatus according to claim 1, wherein the conversion coefficient calculation unit calculates the luminance conversion coefficient and the saturation conversion coefficient from the representative value.   A plurality of maximum value specifying ranges are set to be shifted from each other, each of the plurality of maximum value specifying ranges includes the target pixel, and the representative value calculation unit specifies each of the plurality of maximum value specifying ranges from the plurality of maximum value specifying ranges. The image processing apparatus according to claim 2, wherein an average of a plurality of maximum values is calculated as the representative value. A luminance value calculation unit for calculating a luminance value of the target pixel;
Based on the brightness value calculated by the brightness value calculation unit, the saturation conversion for correcting the saturation conversion coefficient so as to clip the pixel value of the target pixel to a predetermined upper limit level in the processing of the signal level conversion unit A coefficient correction unit;
The image processing apparatus according to claim 1, further comprising: An image signal input step for inputting an image signal;
A conversion coefficient calculating step for calculating a luminance conversion coefficient and a saturation conversion coefficient used for nonlinear conversion processing of each target pixel which is a pixel to be processed in the image signal;
The pixel signal of the target pixel is nonlinearly converted based on the luminance conversion coefficient and the saturation conversion coefficient, and the luminance and the saturation are changed according to the luminance conversion coefficient and the saturation conversion coefficient without changing the hue of the image signal. A signal level conversion step for changing
And the conversion coefficient calculation step sets the saturation conversion coefficient to a value corresponding to the luminance conversion coefficient and smaller than the luminance conversion coefficient. A luminance value calculating step for calculating a luminance value of the target pixel;
Saturation for correcting the saturation conversion coefficient in order to clip the pixel value of the target pixel to a predetermined upper limit level in the processing of the signal level conversion step based on the luminance value calculated in the luminance value calculation step A conversion coefficient correction step;
The image processing method according to claim 5, further comprising:

Images