JP4047154B2 - Image processing apparatus and image processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for processing an image on which saturation enhancement processing has been performed.
[0002]
[Prior art]
A so-called digital camera is a representative example of an image processing system that processes an image acquired by an image sensor and records image data on a recording medium. A conventional digital camera generates an image having a predetermined color reproduction characteristic by performing predetermined image processing on captured image data.
[0003]
However, the characteristics of these images are aimed at generating an average scene and a satisfactory picture for the subject. Not generated. For example, when an object with low saturation is photographed indoors, the entire image data has a color tone with low saturation. In order to cope with such a case, in the shooting mode of the digital camera, in addition to the normal shooting mode that performs optimum processing for an average scene, image processing that outputs higher saturation (saturation enhancement processing) In some cases, the digital camera has a mode for performing the above (saturation emphasis mode), and a device is devised so that the user can select these modes by an external switch.
[0004]
[Problems to be solved by the invention]
However, when shooting a scene with high saturation in the saturation enhancement mode, data that exceeds the color space of the recorded image set to record the captured image appears in the captured image. However, since these data are projected to the reproduction range of the color space at the time of recording, there is a problem that desired color reproduction cannot be performed.
[0005]
The present invention has been made in view of the above problems, and an object of the present invention is to perform processing without impairing the color reproducibility of an image with enhanced saturation as much as possible.
[0006]
[Means for Solving the Problems]
In order to achieve the object of the present invention, for example, an image processing apparatus of the present invention comprises the following arrangement.
[0007]
That is, an image processing apparatus having a mode for enhancing the saturation of an image,
Color correction means for performing color correction on image data representing the image ;
When a mode for enhancing the saturation of the image is designated, saturation enhancement means for performing saturation enhancement processing on the image data subjected to the color correction ;
In the saturation enhancement processing an image represented by the image data subjected to the counting means for counting the number of pixels that do not fit into the first color space definitive in the image,
Color space conversion means for converting the color space of image data;
Have
The color space conversion means includes
When the mode is designated, if the number of counted pixels by the counting means is smaller than the threshold converts the image data to which the saturation enhancement processing is performed on the image data in the first color space, wherein If the number of counted pixels by the counting means is greater than the threshold value in the image data in the second color space is a wider color space than the image data to which the saturation enhancement processing is performed first color space It is characterized by converting .
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described in detail according to preferred embodiments with reference to the accompanying drawings.
[0009]
[First Embodiment]
FIG. 1 is a block diagram illustrating a functional configuration of the image capturing apparatus according to the present embodiment. In addition, each part of 102 thru | or 110 comprises the image imaging device in this embodiment, 200 is a recording medium which can be attached or detached with respect to the image imaging device in this embodiment.
[0010]
Reference numeral 101 denotes an imaging unit, an imaging lens, an imaging device such as a CCD, a gain adjustment circuit that adjusts the gain of an output signal from the imaging device, and an A / D conversion circuit that converts the output signal of the imaging device after gain adjustment into a digital signal The image capturing unit 101 is configured by a memory or the like that temporarily stores the converted digital signal, and outputs the captured image to the subsequent white balance correction unit 103 as digital image data.
[0011]
An exposure adjustment unit 102 refers to the image data output from the imaging unit 101, and in accordance with a predetermined program diagram so as to obtain an appropriate brightness based on the luminance of a predetermined part in the image. The speed and aperture value are determined, and the exposure time of the image sensor included in the imaging unit 101 and the aperture of the photographing lens are controlled. A white balance correction unit 103 adjusts the white balance of the image data output from the imaging unit 101.
[0012]
A pixel interpolation unit 104 interpolates R, G, and B values, which are pixel values of each pixel of the image after correction by the white balance correction unit 103, according to the color filter array of the image sensor, and outputs color image data To do.
[0013]
A color correction unit 105 performs color correction on the color image data output from the pixel interpolation unit 104 so that the color reproduction characteristic becomes a desired characteristic. A processing control unit 130 controls branching of processing to each subsequent unit in accordance with a mode (details will be described later) input from the outside by a switch or the like. Reference numeral 106 denotes a saturation enhancement processing unit. When the saturation enhancement mode is input to the processing control unit 130 as a shooting mode, the processing control unit 130 controls the operation of the saturation enhancement processing unit 106, and the saturation enhancement processing unit 106. Performs a saturation enhancement process on the image color-corrected by the color correction unit 105.
[0014]
Reference numeral 107 denotes an edge emphasizing unit that performs processing for emphasizing the contour portion (edge) of the subject in the image by the color correcting unit 105. A color space encoding unit 108 converts the image data from the color correction unit 105 or the image data from the edge enhancement unit 107 into image data in a different color space from the color space (RGB) to which the image data belongs. .
[0015]
An image compression unit 109 performs compression coding processing on the color image data by the color space coding unit 108 using a compression method such as JPEG.
[0016]
A recording / reproducing unit 110 adds predetermined auxiliary information to the image data compressed by the image compression unit 109 and records the image data in the recording medium 200 as recorded image data according to a predetermined format. Is also displayed on a display device such as a liquid crystal monitor (not shown). Reference numeral 200 denotes a recording medium such as a compact flash (registered trademark) card or an optical disc that can be attached to and detached from the image pickup apparatus according to the present embodiment.
[0017]
Next, the operation of the image pickup apparatus in each of two shooting modes that can be set in the image pickup apparatus having the above configuration will be described.
[0018]
<Normal shooting mode>
A case where a normal shooting mode (hereinafter referred to as a normal mode), that is, a mode without saturation enhancement is input from an input unit (not shown) and a signal indicating this mode is input to the processing control unit 130 will be described.
[0019]
When the user presses a shutter button (not shown) to photograph the subject (when the shutter button is not completely pressed), the imaging unit 101 detects this and captures an image. The exposure adjusting unit 102 refers to this image, determines a shutter speed and an aperture value according to a predetermined program diagram so that the image has an appropriate brightness based on the luminance of a predetermined part in the image, and an imaging unit 101 controls the exposure time of the image sensor included in 101 and the aperture of the taking lens. When the shutter button is further pressed deeper, an image is taken according to the controlled exposure time and the aperture of the taking lens. Hereinafter, the captured image is a processing target.
[0020]
The white balance correction unit 103 performs statistical analysis such as calculation of an average value using the image data output from the imaging unit 101 so that a predetermined statistic as a statistical analysis result becomes a predetermined value. Next, a coefficient for adjusting the white balance is obtained. At this time, when a plurality of white balance modes can be set to correspond to a plurality of light sources, there is a target value of a statistic according to each mode, and an optimum coefficient is obtained for each mode. These coefficients are obtained for each RGB channel.
[0021]
The pixel interpolation unit 104 interpolates R, G, and B channel color image data from the image data output from the imaging unit 101 using the coefficient obtained by the white balance correction unit 103. Here, the interpolation method in the present embodiment will be described.
[0022]
FIG. 2 is a diagram illustrating an arrangement of color filters of the image sensor of the imaging unit 101 according to the present embodiment. Since only one of R, G, and B channels can be arranged at each pixel position of the filter shown in the figure, it is necessary to obtain pixel values of channels different from the filter color by interpolation. At this time, in the pixel position where the color of the filter is G, when the pixel value of the G channel is obtained, a value obtained by adding the coefficient corresponding to the G channel to the pixel value becomes the output value. Further, when obtaining the pixel value of the R or B channel, the coefficient corresponding to each channel is added to the pixel values of two adjacent pixels, and the average value becomes the output value. Further, at the pixel position where the filter color is R or B, when obtaining the pixel value of the channel of the same color as the filter, the value obtained by integrating the pixel value with the coefficient corresponding to each channel is the output value. Further, when obtaining pixel values of channels having a color different from that of the filter, the coefficient corresponding to each channel is added to the pixel values of the four adjacent pixels, and the average value becomes the output value.
[0023]
Note that the above-described pixel interpolation method is an example. For example, when a pixel value of an R or B channel is obtained at a pixel position where the filter color is G, pixels of corresponding pixels separated by a predetermined distance. A coefficient corresponding to each channel may be added to the value, and a weighted average value based on the distance may be used as the output value. The image data output from the pixel interpolation unit 104 is temporarily stored in an image memory (not shown) in the pixel interpolation unit 104.
[0024]
The color correction unit 105 performs matrix correction on the image data output from the pixel interpolation unit 104 and performs color correction processing as shown in (Equation 1) below.
[0025]
C ′ = AC (Formula 1)
Here, C = (R, G, B) ^t , C ′ = (R ′, G ′, B ′) ^t , (t represents transposition), and represents RGB values before and after color correction, respectively. A is a matrix of 3 rows and 3 columns, and is mainly a matrix for correcting the hue of image data.
[0026]
Since the signal indicating the normal mode in which the saturation enhancement is not performed is input to the processing control unit 130 as described above, the processing by the color enhancement unit 105 is not performed without performing the processing by the subsequent saturation enhancement processing unit 106. Control is performed so that the data is directly output to the color space encoding unit 108.
[0027]
Next, the edge emphasizing unit 107 performs processing for emphasizing a contour portion (edge) in the image using the image data from the color correcting unit 105. As processing for emphasizing edges, first, filtering is performed on an image, and contour portions in a predetermined direction are extracted from the image, respectively. Then, edge enhancement is performed on the extracted contour portion by applying an optimum edge enhancement filter for each direction. At this time, a filter process having a low-pass effect is performed on an isolated noise component with no contour directionality.
[0028]
The image data processed by the edge enhancement unit 107 is output to the color space encoding unit 108. The color space encoding unit 108 encodes the input image data assuming a standard color space. For example, when the standard color space is assumed to be the sRGB color space described in IEC 61966-2-1, RGB values are output according to the description of IEC 61966-2-1. Here, (0, 0, 0), (1, 1, 1), (1, 0, 0), (0, 1, 0), (0, 0, 1) are black, white, red, Assuming that green and blue are represented and the values of R, G, and B are represented by real numbers from 0 to 1, the voltage-luminance characteristics of the display for each RGB channel of the input image data. Γ conversion is performed with reference to the table for the main purpose of matching (γ characteristics), and clipping is performed so that the R, G, and B values fall outside the range of 0 to 1 and fall within 0 to 1. (0 or less is set to 0, and 1 or more is set to 1), normalized to the number of output bits (8 bits for sRGB), and output.
[0029]
The image data output from the color space encoding unit 108 is compressed by the image compression unit 109 according to the desired image quality and compression rate, and output to the recording / reproducing unit 110. The recording / reproducing unit 110 records information at the time of photographing together with the compressed image data on the recording medium 200 as one image data according to a predetermined image format.
[0030]
<Saturation enhancement mode>
Next, a case where a mode for enhancing the saturation of the captured image (saturation enhancement mode) is input from an input unit (not shown) and a signal indicating this mode is input to the processing control unit 130 will be described.
[0031]
Even when the saturation enhancement mode is input, the normal mode is selected for the processing in each of the imaging unit 101, the exposure adjustment unit 102, the white balance correction unit 103, the pixel interpolation unit 104, and the color correction unit 105. Since it is the same as the case, description here is omitted.
[0032]
The saturation enhancement unit 106 performs saturation enhancement processing on the image data output from the color correction unit 105 by matrix calculation shown in the following (Equation 2).
[0033]
C ″ = A′C ′ (Formula 2)
Here, C ″ = (R ″, G ″, B ″) ^t , where t represents transposition, and represents the RGB value after saturation enhancement. A ′ is a matrix of 3 rows and 3 columns, and is a parameter for performing saturation enhancement. Since matrix calculation for saturation enhancement is a well-known technique, description thereof is omitted here.
[0034]
The edge enhancement unit 107 performs the same processing as when the normal mode is selected. The color space encoding unit 108 encodes the RGB values in a color space wider than the color space encoded in the normal mode for the image data (RGB values) from the edge enhancement unit 107. For example, RGB values are output to a 12-bit non-linear scRGB color space described in IEC 61966-2-2. Here, (0, 0, 0), (1, 1, 1), (1, 0, 0), (0, 1, 0), (0, 0, 1) are black, white, red, Assuming that green and blue are represented and the values of R, G, and B are represented by real numbers from 0 to 1, γ conversion is performed on each RGB channel of the image data by referring to the table, and each R, G, B values outside the range of -0.5 to 7.5 are clipped so that they fall within -0.5 to 7.5, normalized to the number of output bits, and output.
[0035]
As described above, in the saturation enhancement mode, colors can be expressed in a wider color space than in the normal mode, and therefore, the probability that the color is clipped is lower than in the normal mode. As a result, even if the range of colors used by performing saturation emphasis increases, the color to be expressed can be expressed almost accurately.
[0036]
Also, for example, the range of RGB values is 0 to 1 which is the same as the standard color space, but the RGB values are output to a color space defined by chromaticity coordinates having higher RGB saturation than the standard color space. Also good. For example, the RGB value of the standard color space is associated with the RGB value of a wider color space by the process shown in (Expression 2) (that is, the saturation on the data is reduced by the inverse process of (Expression 2)), the standard If the encoding is performed in the same manner as that of the color space, it is possible to encode the chroma value-enhanced RGB value into a color space wider than the standard.
[0037]
Further, the processing performed by the image compression unit 109 and the recording / reproducing unit 110 is the same as that in the case where the normal mode is instructed, and thus description thereof is omitted.
[0038]
As described above, the image capturing apparatus according to the present embodiment outputs the image data (RGB values) subjected to saturation enhancement processing to a wide color space, and thus is forcibly projected within the color space range. Therefore, the desired color reproduction can be achieved as compared with the prior art.
[0039]
[Second Embodiment]
The image capturing apparatus according to the present embodiment performs the same operation as the image capturing apparatus according to the first embodiment when the normal mode is instructed. However, when the saturation enhancement mode is instructed, the first embodiment is performed. An operation different from the form is performed. Therefore, processing performed by the image capturing apparatus according to the present embodiment when the saturation enhancement mode is instructed will be described below.
[0040]
FIG. 3 is a block diagram showing a functional configuration of the image pickup apparatus in the present embodiment. The same parts as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted.
[0041]
When the user presses a shutter button (not shown) to shoot a subject (completely pressed), image data whose brightness is controlled by the exposure adjustment unit 102 is output from the imaging unit 101, and the white balance Input to the correction unit 103. As described above, the white balance correction unit 103 obtains a coefficient for adjusting the white balance. The pixel interpolation unit 104 performs the interpolation process using the coefficient obtained by the white balance correction unit 103.
[0042]
The color correction unit 105 performs the color correction process according to the above (Equation 1) on the image data output from the pixel interpolation unit 104. Since the signal indicating the saturation enhancement mode is input to the processing control unit 130 as described above, the processing by the subsequent saturation enhancement processing unit 106 is performed. The saturation enhancement processing unit 106 performs the saturation enhancement processing according to the above (Equation 2) on the image data color-corrected by the color correction unit 105.
[0043]
Next, the image analysis unit 121 first determines whether the RGB values (input RGB values) of the image data output from the saturation enhancement unit 106 fall within the color gamut of the standard color space when projected onto the standard color space. I do. For example, (0,0,0), (1,1,1), (1,0,0), (0,1,0), (0,0,1) are black, white, red, and green, respectively. , Blue, and each R, G, B value is represented by a real number from 0 to 1, whether or not the input RGB value is in the range of 0 to 1 is determined for each of the image data. The process is sequentially performed on the pixels, and the number of pixels outside the range is counted.
[0044]
The color space selection unit 122 has a predetermined count value that the count value counted by the image analysis unit 121, that is, the number of pixels whose RGB values exit the color gamut of the standard color space is stored in the color space selection unit 122 in advance. A standard color space is selected when the threshold value is below the threshold value, and a color space wider than the standard color space is selected when the predetermined threshold value is exceeded.
[0045]
The color space encoding unit 108 encodes the RGB values in the color space selected by the color space selection unit 122 using the RGB values of the image data in which the edge portion of the image is enhanced by the edge enhancement unit 107 as input RGB values. The encoding process to the color space includes the encoding process in the normal mode described above when encoding in the standard color space, and the encoding process in the saturation enhancement mode described above when encoding in the wide color space. It is the same.
[0046]
Further, the processing performed by the image compression unit 109 and the recording / reproducing unit 110 is the same as that in the case where the normal mode is instructed, and thus description thereof is omitted.
[0047]
Thus, in the image capturing apparatus according to the present embodiment, the image analysis unit 121 counts the number of pixels exceeding the standard color space when the saturation enhancement is performed, and the color space selection unit 122 performs the image analysis unit 121. Since the image data is encoded in a wide color space when the count value of the above exceeds a predetermined threshold, and the other is encoded in the standard color space even when saturation enhancement is performed, an efficient color Can encode into space.
[0048]
That is, when the color distribution of the image after saturation enhancement can be reproduced in a standard color space, the image is encoded in the standard color space, so that the saturation enhancement mode is designated as in the first embodiment. Even if this is done, the image data is not encoded in an unnecessarily wide color space. For example, in the case where the pixel value is encoded with 8 bits in the standard color space and 12 bits in the wide color space for each RGB channel, if the encoding is performed in the wide color space, the memory of the memory that stores the encoded data is stored. A large size is required. However, according to the present embodiment, even when the saturation emphasis mode is specified, the size of the memory for holding the encoded data can be made smaller than that in the first embodiment.
[0049]
In addition, even when recording with the same number of bits in the standard and wide color spaces, as a result of saturation enhancement, images with many pixels in the color gamut of the standard color space are encoded in the standard color space. Therefore, since the pixels in the color gamut of the standard color space that occupies most of the image can be recorded in a wider range than when encoding in a wide color space, color reproducibility is good.
[0050]
[Third Embodiment]
In the first and second embodiments, the color space encoding unit 108 performs encoding in any one of the two color spaces, but selects one from two or more color spaces. The encoding specification is also good. For example, in the first embodiment, in addition to the normal mode, when there are a plurality of chroma enhancement shooting modes having different enhancement levels, a color space to be recorded in advance may be associated with each mode. .
[0051]
In the first and second embodiments, when the image data is recorded on the recording medium 200 by the recording / reproducing unit 110, the information of the color space used for encoding together with the image data may be recorded as supplementary information. good. For example, chromaticity coordinates of typical colors such as black, white, red, green, and blue may be recorded in association with RGB values when the colors are recorded in the color space. Further, in the case of a color space whose definition is clear in advance, such as sRGB and scRGB, a code representing each color space may be defined in advance and recorded. Thereby, for example, when printing image data, the recorded color space can be referred to, so that the color of the photographed subject can be faithfully reproduced.
[0052]
Further, although each processing in the first and second embodiments has been described as being performed by an image capturing device such as a digital camera, for example, the data output from the photographing unit 101 is directly used by the recording / reproducing unit 110 as the recording medium 200. And the recorded data is read by a general-purpose PC (Personal Computer) or the like, and the white balance correction unit 103 to the color space encoding unit 108 (in addition to this in the second embodiment, the image analysis unit 121, the color Each process performed by each of the space selection units 122) may be executed by a PC application program. In this case, in the configuration shown in FIGS. 1 and 3, the configuration excluding the imaging unit 101, the exposure adjustment unit 102, the recording / reproducing unit 110, and the recording medium 200 indicates the functional configuration of the PC that executes the application program. Become.
[0053]
[Other Embodiments]
An object of the present invention is to supply a storage medium (or recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus. Needless to say, this can also be achieved by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.
[0054]
Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.
[0055]
The storage medium includes a communication medium such as a communication cable used for a network such as the Internet or a LAN. That is, when the program code of the above-described embodiment is held in a server device on a network, the program can be introduced into the computer by downloading from the server device to the computer via the network. Therefore, the installed program is executed by a control circuit such as a CPU or MPU on the computer, and as a result, the computer realizes the function in the above-described embodiment. It goes without saying that a communication medium such as a communication cable is included.
[0068]
【The invention's effect】
As described above, according to the present invention, the color reproducibility of an image with enhanced saturation can be processed without impairing as much as possible.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a functional configuration of an image capturing apparatus according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating an arrangement of color filters of an image sensor of the imaging unit 101. FIG.
FIG. 3 is a block diagram illustrating a functional configuration of an image capturing apparatus according to a second embodiment of the present invention.

Claims (6)

An image processing apparatus having a mode for enhancing the saturation of an image,
Color correction means for performing color correction on image data representing the image ;
When a mode for enhancing the saturation of the image is designated, saturation enhancement means for performing saturation enhancement processing on the image data subjected to the color correction ;
In the saturation enhancement processing an image represented by the image data subjected to the counting means for counting the number of pixels that do not fit into the first color space definitive in the image,
Color space conversion means for converting the color space of image data;
Have
The color space conversion means includes
When the mode is designated, if the number of counted pixels by the counting means is smaller than the threshold converts the image data to which the saturation enhancement processing is performed on the image data in the first color space, wherein If the number of counted pixels by the counting means is greater than the threshold value in the image data in the second color space is a wider color space than the image data to which the saturation enhancement processing is performed first color space An image processing apparatus for converting . 2. The image according to claim 1, wherein the color space conversion unit converts the image data subjected to the color correction into image data in the first color space when the mode is not designated. Processing equipment. The image processing apparatus according to claim 1, further comprising an edge enhancement processing unit configured to perform an edge enhancement process on the image data on which the color correction has been performed. The image processing apparatus according to any one of claims 1 to 3, wherein the first color space is a sRGB color space. An image processing method performed by an image processing apparatus having a mode for enhancing the saturation of an image,
A color correction step of performing color correction on the image data indicating the image ;
When a mode for enhancing the saturation of the image is designated, a saturation enhancement step for performing saturation enhancement processing on the image data subjected to the color correction ;
In the saturation enhancement processing an image represented by the image data which has been subjected, a counting step of counting the number of pixels that do not fit into the first color space definitive in the image,
A color space conversion process for converting the color space of the image data;
Have
In the color space conversion step,
When the mode is designated, if the number of counted pixels by the counting step is less than the threshold value, converts the image data to which the saturation enhancement processing is performed on the image data in the first color space, wherein the image data in the second color space when the number of counted pixels by the count process is larger than the threshold value is a wider color space than the image data to which the saturation enhancement processing is performed first color space An image processing method characterized by converting . A program for causing a computer to execute the image processing method according to claim 5 .

Images