JP2008042392A - Image processor and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor capable of suppressing the unnatural raise of color saturation due to nonlinear transformation, while preserving a hue, and reducing false coloring. <P>SOLUTION: An image signal input part 101 inputs the pixel signals of three RGB primary colors. A representative value calculating part 102 calculates the maximum level representative value X, which expresses the maximum level of the pixel signals included in a representative value calculation range, as a value representing the pixel signals in the representative value calculation range near the pixel to be noted. A transformation coefficient calculating part 103 calculates a luminance transformation coefficient kY and a color saturation transformation coefficient kC to be used for nonlinear transformation processing in response to a luminance transformation coefficient characteristic and a color saturation transformation coefficient characteristic, which are different in coefficient change corresponding to the maximum level representative value. A signal level transforming part 104 performs the nonlinear transformation processing of the pixel signals, so as to respectively transform the luminance and color saturation in response to the luminance transformation coefficient kY and the hue transformation coefficient kC, and transform the luminance and color saturation by individual change ratios without accompanying hue change. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to R, G, B primary color image signal processing, and more particularly, to signal level nonlinear conversion processing.

  As is well known in the art, in camera signal processing, non-linear conversion processing of an image signal generally called gamma processing is performed. The gamma processing is for compensating in advance on the imaging side for non-linearity of the video level with respect to the signal of the receiver. Furthermore, the purpose of gamma processing is not limited to this. For example, the nonlinear conversion characteristics of gamma are made S-shaped to compress or expand a specific gradation, thereby reducing noise and improving gradation. Can be achieved.

In primary color signal processing using the three primary colors R, G, and B, if nonlinear signal processing is performed on each of the R, G, and B signals, the ratio of the R, G, and B signals changes, and the hue changes. Will be caused. Therefore, in the conventional nonlinear processing, the representative value X is calculated from the R, G, and B signals near the target pixel, the nonlinear conversion value Gamma (X) for the representative value X is obtained, and further, Gamma (X) is represented by the representative value X. The divided one is used as a non-linear conversion coefficient, and this non-linear conversion coefficient is added to each of the R, G, and B signals. That is, the non-linear transformation process can be expressed as (Equation 1) below.
Gamma (R) = R × Gamma (X) / X
Gamma (G) = G × Gamma (X) / X
Gamma (B) = B x Gamma (X) / X (Formula 1)

  In this way, the signal ratio of R, G, and B can be maintained, and nonlinear conversion processing can be performed without a change in hue.

  However, in the above processing, when any pixel signal of RGB is larger than the signal before conversion and exceeds a predetermined upper limit value, the upper limit clip is applied only to that signal, and R, G, B There is a possibility that the hue of the signal may change due to the signal ratio being out of order. As techniques for avoiding hue changes, for example, the following techniques are known.

The image processing apparatus of Patent Document 1 performs the above-described process (Equation 1) with the luminance as the representative value X in the first stage process, and further performs the following process in the second stage process. The process shown by (Formula 2) is performed.
Ro = Wi + Kc x (Ri-Wi)
Go = Wi + Kc x (Gi-Wi)
Bo = Wi + Kc x (Bi-Wi) (Formula 2)
Wi: Luminance signal
Kc: Coefficient to convert saturation

In the above (Expression 2), only the color difference signal is multiplied by Kc, so that only the saturation is multiplied by Kc without any change in luminance and hue. The prior art combines these two processes, and is configured to compress the luminance W level in the first stage and reduce the saturation until the highest level signal falls within the standard in the second stage. ing.
JP-A-9-331539

  However, the conventional nonlinear processing has the following problems. As already described, in the nonlinear processing, it is effective to increase the gradient of Gamma (X) in order to improve the gradation feeling by extending a specific gradation. However, when this processing is performed with the conventional technology, a large nonlinear conversion coefficient is applied to each of the R, G, and B signals in a specific gradation portion, and there is a problem that the saturation of the gradation portion is excessively emphasized. Can occur. Such a problem of saturation enhancement is not taken into consideration in the prior art of Patent Document 1 shown in (Expression 2).

  Further, in the configuration using the luminance value as the representative value as in the conventional technique shown in (Expression 2), the following problem may occur. As the electronic camera, a three-plate camera and a single-plate camera are known. If there are R, G, B three color signals in the same optical center pixel as in the case of a three-plate camera, it is not possible to obtain a nonlinear conversion coefficient using the luminance signal generated from the R, G, B signal as a representative value. Does not occur. However, in a single-plate camera, only one of R, G, and B signals exists in each pixel at the optical center, and luminance is generated from R, G, and B signals of a plurality of adjacent pixels. In that case, the luminance difference between adjacent pixels is significant in the portion where the signal level fluctuation of the image is severe, so when calculating the nonlinear transformation coefficient based on the luminance, the difference in the nonlinear transformation coefficient applied to each of the R, G, B signals. As a result, there is a problem that the signal ratio of R, G, B collapses and false coloring occurs.

  The present invention has been made in order to solve the conventional problems, the purpose of which can suppress an unnatural saturation increase in a portion where the slope of the nonlinear conversion characteristic is increased while preserving the hue, It is another object of the present invention to provide an image processing apparatus capable of reducing false coloring of a portion where the signal level fluctuation of the image is severe.

  The image processing apparatus according to the present invention includes an image signal input unit that inputs pixel signals of RGB three primary colors and a pixel signal included in the representative value calculation range as a value representative of the pixel signal in the representative value calculation range near the target pixel. Luminance conversion coefficient used for non-linear conversion processing in accordance with luminance conversion coefficient characteristics and chroma conversion coefficient characteristics with different coefficient changes according to the maximum level representative value, representative value calculating means for calculating a maximum level representative value representing the maximum level Conversion coefficient calculation means for calculating the saturation conversion coefficient, and signal level conversion means for performing non-linear conversion processing of the pixel signal so that luminance and saturation are respectively converted according to the luminance conversion coefficient and the saturation conversion coefficient And.

  With this configuration, the luminance conversion coefficient and the saturation conversion coefficient are given different characteristics, and the luminance and saturation are converted according to the luminance conversion coefficient and the saturation conversion coefficient according to the different characteristics, respectively. Conversion processing is performed. As a result, brightness and saturation can be converted at different rates without hue change, the rate of change in saturation can be controlled independently, and the unnatural color at the part where the slope of the nonlinear conversion characteristics is increased. Increase in the degree can be suppressed. In addition, a maximum level representative value representing the maximum level of the pixel signal included in the representative value calculation range is calculated as a value representing the pixel signal in the representative value calculation range near the target pixel, and the luminance conversion coefficient is calculated from the maximum level representative value. Since the saturation conversion coefficient is obtained, it is possible to reduce the false coloring of the portion where the signal level fluctuation is large.

  Further, in the image processing apparatus of the present invention, the luminance conversion coefficient characteristic and the saturation conversion coefficient characteristic are compared with a rise due to an increase in the inclination of the luminance conversion coefficient characteristic on the low frequency side of the maximum level representative value. The rise of the saturation conversion coefficient is set to be reduced.

  With this configuration, the saturation conversion coefficient rises smaller than the rise of the luminance conversion coefficient characteristic on the low frequency side of the maximum level representative value. The low-frequency part of the maximum level representative value corresponds to a dark part of the image, and it is effective to increase the slope of the nonlinear conversion characteristic, avoiding so-called blackening and improving the gradation. In this area, by varying the amount of rise as described above, the slope of the luminance conversion coefficient characteristic is increased, but the slope of the saturation conversion coefficient is prevented from becoming too large, and unnatural saturation rises. Can be suppressed.

  In the image processing apparatus according to the aspect of the invention, the representative value calculation unit may obtain an average of a plurality of maximum pixel values respectively obtained from a plurality of maximum value selection ranges set so as to be shifted from each other within the representative value calculation range. , The maximum level representative value.

  With this configuration, it is possible to appropriately calculate the maximum level representative value representing the maximum level of the RGB signal included in the representative value calculation range, thereby reducing the false coloring of the portion where the signal level fluctuation of the image is severe.

  In the image processing apparatus of the present invention, the signal level conversion unit may calculate a converted luminance obtained by converting a luminance value of each RGB pixel signal after nonlinear conversion using the luminance conversion coefficient, and a color difference between the pixel value and the luminance value. Non-linear conversion processing is performed to make the sum with the converted color difference converted by the saturation conversion coefficient.

  With this configuration, it is possible to adjust the saturation appropriately by performing non-linear conversion that converts brightness and saturation at different independent rates without changing the hue, and at the part where the slope of the non-linear conversion characteristic is increased Unnatural saturation rise can be suppressed.

  Further, the image processing apparatus of the present invention obtains a maximum pixel value from a maximum value detection range near the target pixel, and calculates a maximum value luminance value calculation unit that calculates a luminance value from a pixel signal in a luminance value calculation range near the target pixel. Then, an upper limit saturation conversion coefficient at which the signal value after nonlinear conversion becomes equal to or less than a predetermined upper limit signal value is obtained from the maximum pixel value, the luminance value, and the luminance conversion coefficient, and the saturation is reduced below the upper limit saturation conversion coefficient. Saturation conversion coefficient correction means for correcting the degree conversion coefficient.

  With this configuration, an upper limit saturation conversion coefficient at which the signal value after nonlinear conversion becomes equal to or lower than a predetermined upper limit signal value is obtained, and the saturation conversion coefficient is corrected to be equal to or lower than the upper limit saturation conversion coefficient. Therefore, while suppressing the signal level after conversion to a predetermined upper limit value or less, without causing a hue change, suppressing an unnatural saturation increase in a portion where the slope of the nonlinear conversion characteristic is increased, and further, the signal level of the image It is possible to reduce false coloration in a portion where fluctuation is severe.

  According to another aspect of the present invention, there is provided an image processing method, wherein the representative value calculation is performed by using an image signal input step of inputting RGB three primary color pixel signals and a value representative of a pixel signal in a representative value calculation range near the target pixel. A representative value calculating step for calculating a maximum level representative value representing the maximum level of the pixel signal included in the range, and a non-linear conversion according to a luminance conversion coefficient characteristic and a saturation conversion coefficient characteristic with different coefficient changes according to the maximum level representative value A conversion coefficient calculating step for calculating a luminance conversion coefficient and a saturation conversion coefficient used for processing, and a non-linear conversion of the pixel signal so that luminance and saturation are respectively converted according to the luminance conversion coefficient and the saturation conversion coefficient A signal level conversion step for performing processing. This aspect also provides the above-described advantages of the present invention.

  The image processing method of the present invention obtains a maximum pixel value from a maximum value detection range near a target pixel, and calculates a maximum value luminance value calculation step for calculating a luminance value from a pixel signal in a luminance value calculation range near the target pixel; From the maximum pixel value, the luminance value, and the luminance conversion coefficient, an upper limit saturation conversion coefficient that causes a signal value after nonlinear conversion to be equal to or lower than a predetermined upper limit signal value is obtained, and the saturation conversion is performed to be equal to or lower than the upper limit saturation conversion coefficient. A saturation conversion coefficient correction step for correcting the coefficient. This method also provides the advantages of the present invention described above.

  According to another aspect of the present invention, there is provided an image processing program, wherein the representative value calculation is performed using an image signal input step for inputting pixel signals of RGB three primary colors and a pixel signal in a representative value calculation range near the target pixel. A representative value calculating step for calculating a maximum level representative value representing the maximum level of the pixel signal included in the range, and a non-linear conversion according to a luminance conversion coefficient characteristic and a saturation conversion coefficient characteristic with different coefficient changes according to the maximum level representative value A conversion coefficient calculating step for calculating a luminance conversion coefficient and a saturation conversion coefficient used for processing, and a non-linear conversion of the pixel signal so that luminance and saturation are respectively converted according to the luminance conversion coefficient and the saturation conversion coefficient And causing a computer to execute a signal level conversion step for processing. This aspect also provides the above-described advantages of the present invention.

  In the present invention, the nonlinear conversion process is performed using the luminance conversion coefficient and the saturation conversion coefficient obtained from the maximum level representative value as described above, thereby maintaining the hue and increasing the slope of the nonlinear conversion characteristic. It is possible to provide an image processing apparatus that can suppress an unnatural increase in color saturation, and can reduce false coloring in a portion where the signal level fluctuation of the image is severe.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  An image processing apparatus according to the first embodiment of the present invention is shown in FIG. The image processing apparatus is provided in an imaging apparatus such as a digital camera and a video camera. In FIG. 1, an image processing apparatus includes an image signal input unit 101, a representative value calculation unit 102 that calculates a representative value from R, G, and B signals in the vicinity of a target pixel, and a luminance conversion that is used for nonlinear conversion processing from the representative value. A conversion coefficient calculation unit 103 that calculates the coefficient and the saturation conversion coefficient, and a signal level conversion unit 104 that nonlinearly converts the R, G, and B signals based on the luminance conversion coefficient and the saturation conversion coefficient.

  The configuration of each unit in FIG. 1 may be realized by a circuit or may be realized by software by a program. In the latter case, a program for realizing each element in FIG. 1 is stored and executed by the arithmetic unit.

  The image signal input unit 101 is configured to input pixel signals of R, G, B, and three primary colors. The image signal input unit 101 may include an image sensor that converts an optical image of a subject into an electrical signal. In the present embodiment, the image sensor is a single-plate camera type, color filters are arranged in a mosaic pattern, and each pixel has one of R, G, and B signals. The image signal input unit 101 inputs R, G, and B signals near the target pixel in synchronization using a line memory.

  Note that the image signal input unit 101 is not limited to the above configuration within the scope of the present invention. The image signal input unit 101 may be configured to input image data captured and recorded in the past.

  The representative value calculation unit 102 is configured to calculate a representative value that determines a conversion coefficient for nonlinear processing from R, G, and B signals near the target pixel. The representative value calculation unit 102 calculates a value representing a pixel signal in a predetermined representative value calculation range set near the target pixel. In this embodiment, the representative value calculation unit 102 is configured to calculate a value representing the maximum level of the pixel signal included in the representative value calculation range as the representative value. This representative value is referred to as a “maximum level representative value” in the present embodiment. In the following description, X is used as a symbol representing the maximum level representative value.

  FIG. 2 shows a representative value calculation process. In the example of FIG. 2, the representative value calculation range 201 is a 3 × 3 pixel range composed of the pixel of interest 202 and vertically and horizontally slanted pixels, that is, a range of nine pixels centered on the pixel of interest. Within the representative value calculation range, four maximum value selection ranges 203 to 206 are set so as to be shifted from each other. Each of the maximum value selection ranges 203 to 206 is a 2 × 2 pixel range including the target pixel 202. In the four maximum value selection ranges 203 to 206, the target pixel is located at the lower right, lower left, upper left, and upper right, respectively. In the representative value calculation process, the maximum pixel value (maximum value of four pixel values) is selected from each of the maximum value selection ranges 203 to 206. As a result, four maximum values are selected. Then, the average of the four maximum values is calculated as the maximum level representative value X.

  In this way, in the present embodiment, the representative value calculation unit 102 has a plurality of maximum pixel values respectively obtained from the plurality of maximum value selection ranges 203 to 206 set in the representative value calculation range 201 so as to be shifted from each other. The average is obtained and the maximum level representative value X is obtained. The calculated maximum level representative value X is supplied to the conversion coefficient calculation unit 103.

  The representative value calculation process is not limited to the above. For example, the representative value calculation process may simply be a process for obtaining the maximum value of nine pixel values in the representative value calculation range.

  The conversion coefficient calculation unit 103 includes a luminance conversion coefficient calculation unit 105 and a saturation conversion coefficient calculation unit 106. The luminance conversion coefficient calculation unit 105 and the saturation conversion coefficient calculation unit 106 are respectively a luminance conversion coefficient and a saturation conversion. Calculate the coefficient. Hereinafter, the luminance conversion coefficient is expressed as kY, and the saturation conversion coefficient is expressed as kC.

  The luminance conversion coefficient calculation unit 105 calculates the luminance conversion coefficient kY from the maximum level representative value calculated by the representative value calculation unit 102 as described below.

  FIG. 3 is an example of the X-Gamma (X) characteristic of the nonlinear transformation related to luminance. The horizontal axis is the maximum level representative value X, and the vertical axis is the converted value Gamma (X).

  FIG. 4 is a luminance conversion coefficient characteristic when the X-Gamma (X) characteristic is set as shown in FIG. In FIG. 4, the horizontal axis represents the maximum level representative value X, and the vertical axis represents the luminance conversion coefficient kY. The luminance conversion coefficient kY is Gamma (X) / X, that is, a value obtained by dividing Gamma (X) by the maximum level representative value X.

  As shown in FIG. 4, in the luminance conversion coefficient characteristic, as a whole, the luminance conversion coefficient kY decreases as the maximum level representative value X increases. On the high frequency side of the maximum level representative value X (region where the image is bright), the gradient of the luminance conversion coefficient characteristic is small, and the luminance conversion coefficient kY changes gently. On the other hand, on the low frequency side (region where the image is dark) of the maximum level representative value X, the gradient of the luminance conversion coefficient characteristic is large, the characteristic curve rises, and the luminance conversion coefficient kY changes greatly. This rising area is an area where the image is dark, and the gradation can be effectively improved by increasing the slope of the nonlinear characteristic. Therefore, the image quality can be improved by the characteristics shown in FIG.

  The luminance conversion coefficient calculation unit 105 calculates the luminance conversion coefficient in accordance with the luminance conversion coefficient characteristic of FIG. 4 set according to the maximum level representative value X. A table of maximum level representative value X and luminance conversion coefficient kY corresponding to FIG. 4 may be stored in a ROM or the like. The luminance conversion coefficient calculation unit 105 may acquire the maximum level representative value X calculated by the representative value calculation unit 102, and obtain the luminance conversion coefficient kY corresponding to the maximum level representative value X from the table.

  Further, for example, values of a plurality of representative points on the luminance conversion coefficient characteristic line in FIG. 4 may be stored. The value between the representative points may be calculated by interpolation processing.

  The luminance conversion coefficient calculation unit 105 supplies the luminance conversion coefficient kY obtained as described above to the signal level conversion unit 104.

  Next, the saturation conversion coefficient kC calculation process by the saturation conversion coefficient calculation unit 106 will be described. FIG. 5 shows the saturation conversion coefficient characteristics together with the luminance conversion coefficient characteristics. As in FIG. 4, also in FIG. 5, the horizontal axis is the maximum level representative value X, and the vertical axis is the conversion coefficient. The saturation conversion coefficient characteristic and the luminance conversion coefficient characteristic are set differently as described below.

  Similar to the luminance conversion coefficient characteristic, the saturation conversion coefficient characteristic as a whole has a smaller saturation conversion coefficient kC as the maximum level representative value X increases. Similarly to the luminance conversion coefficient characteristic, the slope of the saturation conversion coefficient characteristic is small on the high frequency side (region where the image is bright) of the maximum level representative value X. The difference from the luminance conversion coefficient characteristic is that the rise of the saturation conversion coefficient is reduced compared to the rise of the luminance conversion coefficient characteristic on the low-frequency side (the dark image area) of the maximum level representative value X. Thus, the slope of the saturation conversion coefficient is set smaller than the slope of the luminance conversion coefficient characteristic.

  In this way, when comparing the luminance conversion coefficient characteristic and the saturation conversion coefficient characteristic, in the region where the luminance conversion coefficient characteristic on the side where the maximum level representative value X is lower, the saturation conversion coefficient characteristic is matched to the luminance conversion coefficient characteristic. Although it is lifted, it is lifted at a certain level that is suppressed compared to the luminance conversion coefficient characteristics.

  The saturation conversion coefficient calculation unit 106 calculates the saturation conversion coefficient kC according to the saturation conversion coefficient characteristic of FIG. A table of maximum level representative value X and saturation conversion coefficient kC corresponding to FIG. 5 may be stored in a ROM or the like. The saturation conversion coefficient calculation unit 106 may acquire the maximum level representative value X calculated by the representative value calculation unit 102, and obtain the saturation conversion coefficient kC corresponding to the maximum level representative value X from the table.

  Further, for example, values of a plurality of representative points on the saturation conversion coefficient characteristic line of FIG. 5 may be stored. The value between the representative points may be calculated by interpolation processing.

In the above processing, the saturation conversion coefficient characteristic is stored separately from the luminance conversion coefficient characteristic, and the saturation conversion coefficient kC is obtained from the representative value completely independently of the luminance conversion coefficient kY. In another example, as described below, the saturation conversion coefficient kC may be calculated from the luminance conversion coefficient kY. The saturation conversion coefficient calculation unit 106 calculates the saturation conversion coefficient kC from the luminance conversion coefficient kY by, for example, the following (Equation 3). This calculation process also corresponds to the process of obtaining the saturation conversion coefficient kC according to the saturation conversion coefficient characteristic of FIG.
kC = (n × kY + m) / (n + m) (Formula 3)

  Here, if n = 1 and m = 0, kY = kC, and the luminance conversion coefficient kY and the saturation conversion coefficient kC are equal. If n = 0 and m = 1, the saturation conversion coefficient kC is always 1 and the saturation does not change. n and m are set so as to obtain the relationship between the two characteristics as shown in FIG. When n = 1 and m = 3 are set, the relationship between the luminance conversion coefficient kY and the saturation conversion coefficient kC is as shown in FIG. 5, and the portion where the luminance increases increases the saturation accordingly.

  In the above processing, the saturation conversion coefficient kC is calculated from the luminance conversion coefficient kY, but before that, the luminance conversion coefficient kC is calculated from the maximum level representative value X. Therefore, in this case, it can be said that the saturation conversion coefficient calculation unit 106 calculates the saturation conversion coefficient kC from the maximum level representative value X.

  The saturation conversion coefficient calculation unit 106 supplies the saturation conversion coefficient kC obtained as described above to the signal level conversion unit 104.

  The signal level conversion unit 104 performs nonlinear conversion of the RGB pixel signal based on the luminance conversion coefficient kY and the saturation conversion coefficient kC supplied from the luminance conversion coefficient calculation unit 105 and the saturation conversion coefficient calculation unit 106. As described below, the signal level conversion unit 104 performs non-linear conversion processing of the pixel signal so that the luminance and the saturation are respectively converted according to the luminance conversion coefficient and the saturation conversion coefficient, and thereby the hue is converted. The brightness and saturation are converted at different rate of change without any change.

Specifically, the signal level conversion unit 104 performs processing represented by the following (Equation 4).
Gamma (R) = (0.3kY + 0.7kC) × R + (kY−kC) × 0.6G + (kY−kC) × 0.1B
Gamma (G) = (kY−kC) × 0.3R + (0.6kY + 0.4kC) × G + (kY−kC) × 0.1B
Gamma (B) = (kY−kC) × 0.3R + (kY−kC) × 0.6G + (0.1kY + 0.9kC) × B (Formula 4)
Gamma (R), Gamma (G), and Gamma (B) are values of R, G, and B after conversion, respectively.

Here, Gamma (R) can be modified as follows.
Gamma (R) = (0.3kY + 0.7kC) × R + (kY−kC) × 0.6G + (kY−kC) × 0.1B
= kC × R + (kY−kC) × (0.3R + 0.6G + 0.1B)
= kC × R + (kY−kC) × Y
= kY × Y + kC (R−Y)
Similarly, Gamma (G) and Gamma (B) can be modified as follows.
Gamma (G) = kY × Y + kC (G−Y)
Gamma (B) = kY × Y + kC (B−Y)

  That is, it can be said that the pixel value after conversion is the sum of the converted luminance and the converted color difference. The converted luminance is a value obtained by converting the luminance value Y with the luminance conversion coefficient kY, and is specifically a product. The converted color difference is a value obtained by converting the difference between the pixel values R, G, and B and the luminance value Y with the saturation conversion coefficient kC, and is specifically a product.

  From the above, the converted value is clearly a value obtained by multiplying the luminance by kY and the color difference by kC. Therefore, by the nonlinear conversion process of (Equation 4), the luminance and the saturation can be converted at different independent change rates (magnifications) without changing the hue.

  In the present embodiment, the independent change rate means that conversion is performed with conversion coefficients having different luminance and color difference as described above. To supplement the explanation regarding this point, as shown in (Equation 3), the saturation conversion coefficient kC may be calculated from the luminance conversion coefficient kY. In this case, the two conversion coefficients kY and kC are linked. However, even if the two conversion coefficients kY and kC are linked, the two conversion coefficients kY and kC are different coefficients according to the different characteristics of FIG. 5, and these different coefficients are used in the nonlinear conversion of (Equation 4). used. Therefore, even in the case of this coefficient interlocking control, it can be said that the luminance and the saturation are converted at an independent rate of change.

  The realization of such independent conversion ratios of luminance and saturation is advantageous in the following points. The rate of change in luminance and saturation is a value corresponding to the luminance conversion coefficient characteristic and the saturation conversion coefficient characteristic shown in FIG. In FIG. 5, the luminance conversion coefficient characteristic rises greatly on the low frequency side of the maximum level representative value X. The low frequency portion of the maximum level representative value X corresponds to the dark portion of the image. Therefore, according to the luminance conversion coefficient characteristic of FIG. 5, the gradient of the luminance conversion rate becomes large in a dark image area, so-called blackout can be avoided, and the gradation feeling can be effectively improved.

  However, if the same conversion coefficient as the luminance is applied to the saturation, the dark part may become unnaturally vivid or the color noise component may be emphasized. In this embodiment, in consideration of this point, the saturation conversion coefficient characteristic is set as shown in FIG. 5, and the saturation conversion coefficient characteristic is matched with the luminance conversion coefficient characteristic on the low frequency side of the maximum level representative value X. However, the rise amount is small and the slope of the characteristic is small compared to the luminance conversion coefficient characteristic. Therefore, it is possible to avoid the saturation conversion rate from becoming too large at the portion where the luminance conversion rate is large, and to avoid the saturation being emphasized too much.

  FIG. 6 is a flowchart showing the operation of the image processing apparatus according to the present embodiment, and corresponds to the image processing method according to the present embodiment. The image processing method of FIG. 6 includes a representative value calculation step (S601), a luminance conversion coefficient calculation step (S602), a saturation conversion coefficient calculation step (S603), and a signal level conversion processing step (S604).

  In the representative value calculating step (S601), a representative value is obtained from the R, G, B signals. Here, the representative value is calculated from the RGB signal of the primary color Bayer array in the single panel camera. At this time, as described with reference to FIG. 2, the maximum value is selected from each of the four 2 × 2 pixel regions including the target pixel and located on the top, bottom, left, and right, and the four maximum values are averaged to obtain the maximum level representative. The value X is calculated.

  In the luminance conversion coefficient calculation step (S602) and the saturation conversion coefficient calculation step (S603), as described with reference to FIGS. 2 to 5, the luminance conversion coefficient kY and the saturation conversion coefficient kC from the maximum level representative value X. Is calculated. The saturation conversion coefficient kC may be obtained from a table of saturation conversion coefficient characteristics, or may be calculated from the luminance conversion coefficient kY by (Equation 3).

  In the signal level conversion processing step (S604), non-linear conversion processing is performed by the calculation formula of (Formula 4). When each pixel has only one of R, G, and B signals as in a single-panel camera, only the conversion formula corresponding to the color of the corresponding pixel signal is selected from the three conversion formulas in (Formula 4). Just do it.

  The image processing apparatus according to the first embodiment of the present invention has been described above. According to the present embodiment, as described with reference to FIG. 5 and (Equation 4), different characteristics are given to the luminance conversion coefficient kY and the saturation conversion coefficient kC, and the luminance conversion according to such different characteristics. Non-linear conversion processing of the pixel signal is performed so that the luminance and the saturation are respectively converted according to the coefficient kY and the saturation conversion coefficient kC. As a result, brightness and saturation can be converted at different rates without hue change, the rate of change in saturation can be controlled independently, and the unnatural color at the part where the slope of the nonlinear conversion characteristics is increased. Increase in the degree can be suppressed.

  Further, according to the present embodiment, the maximum level representative value X representing the maximum level of the pixel signal included in the representative value calculation range is calculated as a value representing the pixel signal in the representative value calculation range near the target pixel. . Then, the luminance conversion coefficient kY and the saturation conversion coefficient kC are obtained from the maximum level representative value X. As a result, it is possible to reduce false coloration in a portion where the signal level of the image varies greatly.

  In this regard, in the prior art, luminance is used as a representative value. In this case, in the portion where the signal level fluctuation of the image is severe, the difference between adjacent pixels becomes severe, and the representative value is also greatly different. When the nonlinear transformation coefficient is calculated based on such representative values, the difference between the nonlinear transformation coefficients applied to each of the R, G, and B signals becomes large. As a result, the signal ratio of R, G, and B collapses, and false coloring occurs. It can happen. Since the present embodiment uses the maximum level representative value X as described above, false coloring can be reduced.

  As described above, according to the present embodiment, it is possible to suppress an unnatural saturation increase at a portion where the slope of the nonlinear conversion characteristic is increased while preserving the hue, and further, a portion where the signal level fluctuation of the image is severe. The false coloring can be reduced, and these can provide a high-quality image.

  Further, according to the present embodiment, in the luminance conversion coefficient characteristic and the saturation conversion coefficient characteristic, the saturation conversion coefficient is compared with the rise due to the increase in the inclination of the luminance conversion coefficient characteristic on the low frequency side of the maximum level representative value X. The rising edge is set to be reduced. As a result, the saturation conversion coefficient rises smaller on the lower side of the maximum level representative value X than the rise of the luminance conversion coefficient characteristic. The low-frequency part of the maximum level representative value X corresponds to the dark part of the image, and it is effective to increase the slope of the nonlinear conversion characteristics, avoiding so-called black crushing and improving the tone of gradation. . In this area, by varying the amount of rise as described above, the slope of the luminance conversion coefficient characteristic is increased, but the slope of the saturation conversion coefficient is prevented from becoming too large, and unnatural saturation rises. Can be suppressed.

  Further, according to the present embodiment, the representative value calculation unit 102 obtains an average of a plurality of maximum pixel values respectively obtained from a plurality of maximum value selection ranges set so as to be shifted from each other within the representative value calculation range. It is configured. Thereby, the maximum level representative value X representing the maximum level of the RGB signal included in the representative value calculation range can be appropriately calculated, and false coloring of a portion where the signal level fluctuation of the image is severe can be reduced.

  Further, according to the present embodiment, the signal level conversion unit 104, as shown in (Expression 4) and its modified expression, each RGB pixel signal after nonlinear conversion, the luminance value as the luminance conversion coefficient, And the sum of the converted luminance converted by the saturation conversion coefficient and the color difference between the pixel value and the luminance value. This makes it possible to perform non-linear conversion that converts luminance and saturation at different independent rates without changing the hue, so that the saturation can be adjusted appropriately, and the non-uniformity in the portion where the slope of the non-linear conversion characteristic is increased. Reduces natural saturation.

  Next, an image processing apparatus according to the second embodiment of the present invention will be described.

  FIG. 7 shows an image processing apparatus according to this embodiment. The image processing apparatus in FIG. 7 includes a maximum value luminance value calculation unit 701 and a saturation conversion coefficient correction unit 702 in addition to the configuration of the image processing apparatus in FIG. 1, and in this respect, the second embodiment. Is different from the first embodiment. Other configurations are the same as those of the first embodiment. Hereinafter, description of parts common to the first embodiment will be omitted, and parts different from the first embodiment will be described.

  The maximum value luminance value calculation unit 701 obtains the maximum pixel value from the maximum value detection range near the target pixel. The maximum value detection range is, for example, a range of a pixel of interest 202 and pixels that are diagonally up, down, left, and right, that is, a range of 9 pixels (3 × 3 pixels) centering on the pixel of interest. The maximum pixel value is a signal having the maximum signal level among the R, G, and B signals of 9 pixels in the maximum value detection range.

  The maximum value luminance value calculation unit 701 calculates a luminance value from the pixel signal in the luminance value calculation range near the target pixel. In the example of the present embodiment, the luminance value calculation range is set to a 3 × 3 pixel range including the target pixel, like the maximum value detection range. The maximum value luminance value calculation unit 701 calculates the luminance value by adding the R, G, B signals in the luminance value calculation range in a ratio of 3: 6: 1.

  The maximum pixel value is the maximum value of nine pixel values. On the other hand, the maximum level representative value X that is the basis for determining the conversion coefficient in the first embodiment is an average of four maximum values obtained from four 2 × 2 pixel regions. Therefore, the maximum pixel value is different from the maximum level representative value described above.

  However, as described in the first embodiment, the maximum level representative value may simply be the maximum value of the pixel values near the target pixel. In this case, the maximum level representative value matches the maximum pixel value. Therefore, one configuration may be used for calculating the maximum level representative value and the maximum pixel value.

  The maximum pixel value and the luminance value are supplied from the maximum luminance value calculation unit 701 to the saturation conversion coefficient correction unit 702. Further, the saturation conversion coefficient correction unit 702 is supplied with the saturation conversion coefficient kC from the saturation conversion coefficient calculation unit 106 of the conversion coefficient calculation unit 104. Further, the luminance conversion coefficient kY calculated by the luminance conversion coefficient calculation unit 105 is supplied to the saturation conversion coefficient correction unit 702. The saturation conversion coefficient correction unit 702 corrects the saturation conversion coefficient kC as follows using the maximum pixel value, the luminance value, and the luminance conversion coefficient kY.

  The saturation conversion coefficient correction unit 702 obtains an upper limit saturation conversion coefficient that causes the signal value after nonlinear conversion to be equal to or less than a predetermined upper limit signal value from the maximum pixel value, the luminance value, and the luminance conversion coefficient kC. The predetermined upper limit signal value is an allowable upper limit value of the RGB pixel value. Then, the saturation conversion coefficient correction unit 702 corrects the saturation conversion coefficient kC so as to be equal to or less than the upper limit saturation conversion coefficient. Here, the saturation conversion coefficient kC calculated by the saturation conversion coefficient calculation unit 106 is compared with the upper limit saturation conversion coefficient, and the upper limit clipping process to the upper limit saturation conversion coefficient is performed.

  The upper limit saturation conversion coefficient is calculated as follows. In this process, the saturation conversion coefficient corresponding to the upper limit signal value is obtained by back calculation using the maximum pixel value, the luminance value, and the luminance conversion coefficient kY.

  First, it is assumed that the maximum pixel value obtained by the maximum value luminance value calculation unit 701 is an R signal. In this case, when the R signal after nonlinear conversion is to be made lower than the upper limit signal value, the saturation conversion coefficient is expressed by the following equation.

Upper limit signal value ≥ Gamma (R)
= (0.3kY + 0.7kC) × R + (kY−kC) × 0.6G + (kY−kC) × 0.1B
≧ kY × Y + kC × (R−Y)
kC ≤ (Upper limit signal value-kY x Y) / (R-Y)

Similarly, when the maximum pixel value is the G and B signals, the saturation conversion coefficient is expressed by the following equation if the signal after nonlinear conversion is tried to be equal to or lower than the upper limit signal value.
kC ≤ (Upper limit signal value-kY x Y) / (G-Y)
kC ≤ (Upper limit signal value-kY x Y) / (B-Y)

After all, the upper limit saturation conversion coefficient is expressed by the following (formula 5).
Upper limit saturation conversion coefficient = (upper limit signal value−kY × luminance value) / (maximum pixel value−luminance value) (Formula 5)

  The saturation conversion coefficient correction unit 702 calculates the upper limit saturation conversion coefficient by (Equation 5), and performs correction processing to clip the saturation conversion coefficient kC with the upper limit saturation conversion coefficient. The corrected saturation conversion coefficient kC is supplied to the signal level conversion unit 104 and used for nonlinear processing. Thereby, the signal after conversion can be suppressed below the upper limit signal value.

  FIG. 8 is a flowchart showing the operation of the image processing apparatus according to the present embodiment, and corresponds to the image processing method according to the present embodiment. In the image processing method of FIG. 8, the representative value calculation step (S801), the luminance conversion coefficient calculation step (S804), the saturation conversion coefficient calculation step (S805), and the signal level conversion processing step (S809) are the representative values of FIG. This is the same as the calculation step (S601), the luminance conversion coefficient calculation step (S602), the saturation conversion coefficient calculation step (S603), and the signal level conversion processing step (S604). In FIG. 8, a luminance value calculation step (S802), a maximum value calculation step (S803), an upper limit saturation conversion coefficient calculation step (S806), a saturation conversion coefficient comparison step (S807), and a saturation conversion coefficient clip processing step (S808). ) Has been added.

  In the luminance value calculation step (S802) and the maximum value calculation step (S803), as described above, the maximum value luminance value calculation unit 701 calculates the luminance value and the maximum pixel value near the target pixel.

  In the upper limit saturation value calculation step (S806), the saturation conversion coefficient correction unit 702 calculates the upper limit saturation conversion coefficient. The upper limit saturation conversion coefficient is calculated from the luminance value and the maximum pixel value, the luminance conversion coefficient kY calculated by the luminance conversion coefficient calculation unit 105 in the luminance conversion coefficient calculation step (S804), and a predetermined upper limit signal value. ).

  In the saturation conversion coefficient comparison step (S807), the saturation conversion coefficient correction unit 702 calculates the upper limit saturation conversion coefficient by the saturation conversion coefficient calculation unit 106 in the saturation conversion coefficient calculation step (S805). Compared with degree conversion coefficient kC. If the saturation conversion coefficient kC is larger than the upper limit saturation conversion coefficient (S807, No), upper limit clipping is performed in the saturation conversion coefficient clip processing step (S808). Then, the corrected saturation conversion coefficient kC is used in the signal level conversion processing step (S809). If the saturation conversion coefficient kC is equal to or less than the upper limit saturation conversion coefficient (S807, Yes), the saturation conversion coefficient kC is used for nonlinear processing without being corrected.

  The image processing apparatus according to the second embodiment of the present invention has been described above. According to the present embodiment, the maximum pixel value and luminance value near the target pixel are calculated. Then, from the maximum pixel value, the luminance value, and the luminance conversion coefficient kY, an upper limit saturation conversion coefficient that makes the signal value after nonlinear conversion equal to or less than a predetermined upper limit signal value is obtained. Then, the saturation conversion coefficient kC is corrected to be equal to or lower than the upper limit saturation conversion coefficient. Therefore, while suppressing the signal level after conversion to a predetermined upper limit value or less, without causing a hue change, suppressing an unnatural saturation increase in a portion where the slope of the nonlinear conversion characteristic is increased, and further, the signal level of the image It is possible to reduce false coloration in a portion where fluctuation is severe.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments, and it goes without saying that those skilled in the art can modify the above-described embodiments within the scope of the present invention.

  As described above, the image processing apparatus according to the present invention has an effect that it is possible to suppress unnatural saturation increase due to non-linear transformation while preserving the hue, and to further reduce false coloring. It is useful as an image processing device for an imaging device.

1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention. Diagram showing maximum level representative value calculation processing The figure which shows the example of the luminance conversion characteristic in non-linear conversion The figure which shows the luminance conversion coefficient characteristic which is the nonlinear conversion coefficient characteristic which divided the luminance conversion characteristic by the input signal Diagram showing saturation conversion coefficient characteristics together with luminance conversion coefficient characteristics FIG. 3 is a flowchart showing the operation of the image processing apparatus according to the first embodiment. The block diagram of the image processing apparatus in the 2nd Embodiment of this invention FIG. 9 is a flowchart showing the operation of the image processing apparatus according to the second embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image signal input part 102 Representative value calculation part 103 Conversion coefficient calculation part 104 Signal level conversion part 105 Luminance conversion coefficient calculation part 106 Saturation conversion coefficient calculation part 701 Maximum value luminance value calculation part 702 Saturation conversion coefficient correction part

Claims (8)

Image signal input means for inputting pixel signals of RGB three primary colors;
Representative value calculating means for calculating a maximum level representative value representing a maximum level of a pixel signal included in the representative value calculation range as a value representing a pixel signal in a representative value calculation range near the target pixel;
Conversion coefficient calculation means for calculating a luminance conversion coefficient and a saturation conversion coefficient used for nonlinear conversion processing according to a luminance conversion coefficient characteristic and a saturation conversion coefficient characteristic that differ in coefficient change according to the maximum level representative value;
Signal level conversion means for performing non-linear conversion processing of pixel signals so that luminance and saturation are respectively converted according to the luminance conversion coefficient and the saturation conversion coefficient;
An image processing apparatus comprising:   In the luminance conversion coefficient characteristic and the saturation conversion coefficient characteristic, the rising edge of the saturation conversion coefficient is reduced compared to the rising edge due to an increase in the inclination of the luminance conversion coefficient characteristic on the low frequency side of the maximum level representative value. The image processing apparatus according to claim 1, wherein the image processing apparatus is set.   The representative value calculating means obtains an average of a plurality of maximum pixel values respectively obtained from a plurality of maximum value selection ranges set to be shifted from each other within the representative value calculation range, and sets the average as the maximum level representative value. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus.   The signal level conversion means includes a converted luminance obtained by converting a luminance value of each RGB pixel signal after nonlinear conversion using the luminance conversion coefficient, and a converted color difference obtained by converting a color difference between the pixel value and the luminance value using the saturation conversion coefficient. The image processing apparatus according to claim 1, wherein non-linear transformation processing is performed to make the sum of A maximum value luminance value calculating means for calculating a luminance value from a pixel signal in a luminance value calculation range near the target pixel and obtaining a maximum pixel value from a maximum value detection range near the target pixel;
From the maximum pixel value, the luminance value, and the luminance conversion coefficient, an upper limit saturation conversion coefficient that causes a signal value after nonlinear conversion to be equal to or less than a predetermined upper limit signal value is obtained, and the saturation is reduced to the upper limit saturation conversion coefficient or less. A saturation conversion coefficient correction means for correcting the conversion coefficient;
The image processing apparatus according to claim 1, further comprising: An image signal input step for inputting pixel signals of RGB three primary colors;
A representative value calculating step for calculating a maximum level representative value representing the maximum level of the pixel signal included in the representative value calculation range as a value representing a pixel signal in the representative value calculation range near the target pixel;
A conversion coefficient calculation step for calculating a luminance conversion coefficient and a saturation conversion coefficient used for the nonlinear conversion processing according to a luminance conversion coefficient characteristic and a saturation conversion coefficient characteristic that vary in coefficient according to the maximum level representative value;
A signal level conversion step for performing nonlinear conversion processing of a pixel signal so that luminance and saturation are respectively converted according to the luminance conversion coefficient and the saturation conversion coefficient;
An image processing method comprising: A maximum value luminance value calculating step for calculating a luminance value from a pixel signal in a luminance value calculation range near the target pixel, and obtaining a maximum pixel value from the maximum value detection range near the target pixel;
From the maximum pixel value, the luminance value, and the luminance conversion coefficient, an upper limit saturation conversion coefficient that causes a signal value after nonlinear conversion to be equal to or less than a predetermined upper limit signal value is obtained, and the saturation is reduced to the upper limit saturation conversion coefficient or less. A saturation conversion coefficient correction step for correcting the conversion coefficient;
The image processing method according to claim 6, further comprising: An image signal input step for inputting pixel signals of RGB three primary colors;
A representative value calculating step for calculating a maximum level representative value representing the maximum level of the pixel signal included in the representative value calculation range as a value representing a pixel signal in the representative value calculation range near the target pixel;
A conversion coefficient calculation step for calculating a luminance conversion coefficient and a saturation conversion coefficient used for the nonlinear conversion processing according to a luminance conversion coefficient characteristic and a saturation conversion coefficient characteristic that vary in coefficient according to the maximum level representative value;
A signal level conversion step for performing nonlinear conversion processing of a pixel signal so that luminance and saturation are respectively converted according to the luminance conversion coefficient and the saturation conversion coefficient;
An image processing program for causing a computer to execute.

Images