JP2006202091A - Image processing method, apparatus, and program, and image pickup device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method which generates viewing image reference data with appropriate hue rotation correction performed therein without excessively increasing a file capacity to be stored in an information recording medium concerning the image processing method for performing correction processing to correct a hue fluctuation relative to raw image data. <P>SOLUTION: Hue correction information is generated, based on luminance data Y, color difference data Cr, and color difference data Cb, which are obtained by converting the raw image data generated by picking up the image of a subject. Then the hue correction information and the raw image data are recorded in the information recording medium. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing method, an image processing apparatus, an image processing program, and an imaging apparatus.

  Digital image data (hereinafter referred to as image data) captured by an imaging apparatus is distributed via an information recording medium such as a CD-R (Compact Disc Recordable), a floppy (registered trademark) disk, a memory card, or the Internet, or the CRT ( Cathode Ray Tube), displayed on a display device such as a liquid crystal display or a plasma display or a small liquid crystal monitor of a mobile phone, or printed as a hard copy image using an output device such as a digital printer, an inkjet printer, or a thermal printer. The display / printing methods have been diversified.

  Also, when displaying and outputting image data for viewing purposes, gradation adjustment, brightness adjustment, color balance adjustment, and sharpness so that the desired image quality can be obtained on a display monitor or hard copy used for viewing. In general, various image processing typified by emphasis is performed to generate appreciation image reference data.

  As an example of an image processing method performed on image data, after converting the image data into luminance data and color difference data in the Y-Cr-Cb space, the color difference ratio is constant regardless of the luminance when performing the color difference matrix calculation. It has been proposed to correct a hue variation that has occurred at the time of overexposure or underexposure of skin color by determining a coefficient depending on luminance so as to satisfy (for example, see Patent Document 1). As a result, for an image displayed on a display monitor such as a CRT, an effect of temporarily suppressing the occurrence of hue variation can be expected.

On the other hand, when the output of the scene reference raw data is specified by the operation unit, the scene reference raw data depending on the imaging device characteristics of the imaging device is generated by imaging, and the standardized scene reference is performed on the scene reference raw data. It has been proposed to generate auxiliary reproduction data when performing imaging device characteristic correction processing for generating image data, attach it to the scene reference raw data, and further record it on a medium (for example, see Patent Document 2). . As a result, the captured image can be recorded on the media without any information loss and provided to an external device.
JP 2003-61109 A JP 2004-96500 A

  When further image processing is performed on the image data that has been subjected to the processing described in Patent Document 1, the correction method for correcting the hue variation is greatly different for each digital camera. The correction value had to be changed for each camera, and the load was large. In particular, in business applications where data from a variety of digital cameras are brought in, it is very heavy to hold correction values for all digital cameras in advance, and hue fluctuations (both hue hues) are common to all digital cameras. Could not be efficiently and appropriately suppressed.

  Furthermore, when the correction image data such as the viewing image reference data is also generated in the digital camera, the correction image data cannot be reproduced from the raw image data such as the scene reference raw data. When recording both image data, data association and directory hierarchy management when storing the information recording medium have become very complicated.

  On the other hand, in the imaging device described in Patent Document 2, when all the reproduction auxiliary data is added and recorded on the information recording medium, the capacity becomes enormous, and the capacity of the scene reference raw data is further increased. Free space is constrained and the transfer time during communication is increased. Furthermore, the processing load on the digital camera increases, leading to a decrease in recording speed and a decrease in battery life.

  Therefore, in view of the above-described problems, the present invention provides an image processing method for performing correction processing (hereinafter, also referred to as hue rotation correction) for correcting hue variation on raw image data, and reducing the file capacity stored in the information recording medium. It is possible to generate appreciation image reference data that has undergone appropriate hue correction without being excessively increased, and without depending on the captured digital camera, and hue correction performed by the captured digital camera. An object of the present invention is to provide an image processing method capable of generating appreciation image reference data that reproduces.

  The object of the present invention can be achieved by the following constitution.

The invention according to claim 1 is a raw image data input step of inputting raw image data generated by imaging a subject;
A color space data conversion step of converting the raw image data into luminance data Y and color difference data Cr and color difference data Cb in the Y-Cr-Cb space;
A correction information generating step for generating hue correction information based on the luminance data Y, the color difference data Cr, and the color difference data Cb;
A hue correction step of generating corrected image data based on the hue correction information and the raw image data;
And an information recording step of recording the hue correction information and the raw image data on an information recording medium.

  The invention according to claim 2 is the image processing method according to claim 1, wherein the information recording step records the corrected image data on an information recording medium.

In the invention according to claim 3, in the hue correction step, the color difference data Cr and the color difference data Cb are respectively converted into color differences by performing a matrix operation using any one of the following formulas (1) to (4). Correction to data Cr ′ and color difference data Cb ′;
3. The image processing method according to claim 1, wherein the hue correction information includes a correction coefficient K used in any one of the following formulas (1) to (4).
(1) Cr ′ = Cr × A / K + Cb × B,
Cb ′ = Cr × C + Cb × D
(2) Cr ′ = (Cr × A + Cb × B) / K,
Cb ′ = Cr × C + Cb × D
(3) Cr ′ = Cr × A + Cb × B,
Cb ′ = Cr × C + Cb × D × K
(4) Cr ′ = Cr × A + Cb × B,
Cb ′ = (Cr × C + Cb × D) × K
However, A, B, C and D are matrix coefficients, K is a function of the luminance data Y, and the color difference ratio represented by Cr ′ / Cb ′ is constant regardless of the value of the luminance data Y. The correction coefficient is determined to be greater than 0.

The invention according to claim 4 is the image processing method according to claim 3, wherein the correction coefficient K is calculated by the following equation (5).
(5) K = (Cr / Cb) / (Cr / Cb) _{Y = Y0}
However, (Cr / Cb) _{Y = Y0} is the color difference ratio at the time of proper exposure obtained by changing the exposure amount for the skin-colored subject.

  According to a fifth aspect of the invention, in the information recording step, the hue correction information and the raw image data are associated and recorded on an information recording medium. This is an image processing method.

The invention according to claim 6 is a correction area specifying step of specifying a correction area in one image;
Including a position information generation step of generating position information indicating the position of the correction region,
The color space data conversion step, the correction information generation step, and the hue correction step are performed on the raw image data in the correction region,
6. The image processing method according to claim 1, wherein the information recording step records the hue correction information and the position information in an information recording medium in association with each other.

The invention according to claim 7 is a raw image data input means for inputting raw image data generated by imaging a subject;
Color space data conversion means for converting the raw image data into luminance data Y and color difference data Cr and color difference data Cb in the Y-Cr-Cb space;
Correction information generating means for generating hue correction information based on the luminance data Y, the color difference data Cr, and the color difference data Cb;
Hue correction means for generating corrected image data based on the hue correction information and the raw image data;
An image processing apparatus comprising: information recording means for recording the hue correction information and the raw image data on an information recording medium.

  The invention according to claim 8 is the image processing apparatus according to claim 7, wherein the information recording means records the corrected image data on an information recording medium.

In the invention according to claim 9, the hue correction means performs the matrix operation using any one of the following formulas (1) to (4), thereby obtaining the color difference data Cr and the color difference data Cb, respectively. The color difference data Cr ′ and the color difference data Cb ′ are corrected.
The image processing apparatus according to claim 7, wherein the hue correction information includes a correction coefficient K used in any one of the following formulas (1) to (4).
(1) Cr ′ = Cr × A / K + Cb × B,
Cb ′ = Cr × C + Cb × D
(2) Cr ′ = (Cr × A + Cb × B) / K,
Cb ′ = Cr × C + Cb × D
(3) Cr ′ = Cr × A + Cb × B,
Cb ′ = Cr × C + Cb × D × K
(4) Cr ′ = Cr × A + Cb × B,
Cb ′ = (Cr × C + Cb × D) × K
However, A, B, C and D are matrix coefficients, K is a function of the luminance data Y, and the color difference ratio represented by Cr ′ / Cb ′ is constant regardless of the value of the luminance data Y. The correction coefficient is determined to be greater than 0.

The invention according to claim 10 is the image processing apparatus according to claim 9, wherein the correction coefficient K is calculated by the following equation (5).
(5) K = (Cr / Cb) / (Cr / Cb) _{Y = Y0}
However, (Cr / Cb) _{Y = Y0} is the color difference ratio at the time of proper exposure obtained by changing the exposure amount for the skin-colored subject.

  The invention according to claim 11 is characterized in that the information recording means records the hue correction information and the raw image data in an information recording medium in association with each other. This is an image processing apparatus.

The invention according to claim 12 is a correction area specifying means for specifying a correction area in one image;
Position information generating means for generating position information indicating the position of the correction region;
The color space data conversion means and the correction information generation means execute processing on the raw image data of the correction area,
12. The image processing apparatus according to claim 7, wherein the information recording unit records the hue correction information and the position information in an information recording medium in association with each other.

  A thirteenth aspect of the present invention is an image processing program that causes a computer to execute the image processing method according to any one of the first to sixth aspects.

  An invention according to a fourteenth aspect is an image processing program that causes a computer to function as each means according to any one of the seventh to twelfth aspects.

  According to a fifteenth aspect of the present invention, there is provided an imaging apparatus comprising the image processing apparatus according to any one of the seventh to twelfth aspects.

  According to the first or seventh aspect of the present invention, since the hue correction information and the raw image data, which are highly effective in correcting the hue variation, are recorded in the information recording medium among the reproduction auxiliary data, the file stored in the information recording medium It is possible to generate appreciation image reference data that has been corrected for hue rotation without excessively increasing the capacity and without depending on the captured digital camera, and for the hue rotation performed by the captured digital camera. It is possible to easily generate appreciation image reference data that reproduces the correction.

  According to the second or eighth aspect of the invention, since the corrected image data generated based on the hue correction information and the raw image data is recorded on the information recording medium, the hue around correction is performed on the display device or the output medium. It is possible to easily appreciate a good image made.

  According to the third, fourth, ninth, or tenth aspect of the present invention, the hue correction information includes a correction coefficient K that varies greatly between the digital camera and the image processing apparatus that perform the hue rotation correction. It is possible to easily generate appreciation image reference data that has been corrected for hue rotation without excessively increasing the file capacity to be recorded and without depending on the captured digital camera.

  According to the invention described in claim 5 or 11, since the hue correction information and the raw image data are associated and recorded on the information recording medium, it is possible to easily generate the appreciation image reference data reproducing the hue correction. It becomes.

  According to the invention described in claim 6 or 12, hue correction is performed on the raw image data of the correction area specified in one image, and position information indicating the position of the correction area and hue correction information are obtained. Since the information is recorded in association with the information recording medium, it is possible to easily generate appreciation image reference data in which hue correction is reproduced without excessively increasing the capacity of the file stored in the information recording medium.

  Hereinafter, details of various data and color space according to the present invention will be sequentially described.

[Raw image data]
The raw image data refers to image data in a state before performing hue correction based on the luminance data and color difference data of the raw image data, for example, scene reference raw data depending on imaging device characteristics corresponds to this. .

[Scene reference raw data]
“Scene-referenced raw data depending on imaging device characteristics” is a raw output signal directly recorded on an imaging device in which information faithful to a subject is recorded. For example, data digitized by an A / D converter included in a digital camera This means data that has been corrected for noise such as fixed pattern noise and dark current noise, and is generally referred to as RAW data or “Digital Negative (DNG)” advocated by Adobe Systems. The image data having the image format is representative.

  The RAW data corresponds to a captured image signal form described in, for example, Japanese Patent Laid-Open No. 11-261933, and such image data is referred to as dedicated application software ("development software"). ) Can be converted into viewing image reference data for display / printing (referred to as “electronic development” or simply “development”). Since RAW data stores all information at the time of shooting, it is possible to recreate corrected image data. If another color system file such as CMYK is created directly, the color gamut difference from the display monitor (sRGB) Due to this, the color is not inadvertently changed.

  The information amount (for example, the number of gradations) of the scene reference raw data is preferably equal to or greater than the information amount (for example, the number of gradations) required for the viewing image reference data in accordance with the performance of the A / D converter. . For example, when the number of gradations of the viewing image reference data is 8 bits per channel, the number of gradations of the scene reference raw data is preferably 12 bits or more, more preferably 14 bits or more, and even more preferably 16 bits or more.

[Corrected image data]
The corrected image data refers to image data that has been subjected to hue correction based on the luminance data and color difference data of the raw image data. For example, standardized scene reference image data and appreciation image reference data correspond to this.

[Scene reference image data]
“Standardized scene reference image data” is image data in which the signal intensity of each color channel based on at least the spectral sensitivity of the image sensor itself is mapped to the standard color space. In the scene reference image data, image processing for modifying the data contents is omitted in order to improve the effect at the time of image viewing such as gradation conversion, sharpness enhancement, and saturation enhancement. Typical examples of the standardized scene reference image data include image data mapped to a standard color space of “RIMM RGB” (Reference Input Medium Metric RGB) or “ERIMM RGB” (Extended Reference Input Medium Metric RGB). Details of RIMM RGB and ERIMM RGB are described in Journal of Imaging Science and Technology, Vol. 45, pages 418-426 (2001).

  In addition, the scene reference image data includes the photoelectric conversion characteristics of the imaging device (opt-electronic conversion function defined by ISO1452; for example, “Fine Imaging and Digital Photography” of Corona Co., Ltd., page 449 of the Japan Photographic Society Publishing Committee). It is preferable that correction has been performed. The information amount (for example, the number of gradations) of the standardized scene reference image data is equal to or more than the information amount (for example, the number of gradations) required for the viewing image reference data in accordance with the performance of the A / D converter. It is preferable. For example, when the number of gradations of the viewing image reference data is 8 bits per channel, the number of gradations of the scene reference image data is preferably 12 bits or more, more preferably 14 bits or more, and even more preferably 16 bits or more.

[Appreciation image reference data]
The appreciation image reference data is an image used for a display device such as a CRT, a liquid crystal display, or a plasma display, or an image used by an output device for generating a hard copy image on an output medium such as a silver salt photographic paper, an inkjet paper, or a thermal printer paper. Means data. Appreciation image reference data is luminance data of raw image data so that an optimal image can be obtained on a display device such as a CRT, a liquid crystal display, a plasma display, and an output medium such as a silver halide photographic paper, an inkjet paper, and a thermal printer paper. And image data subjected to optimization processing including hue correction based on the color difference data.

[Y-Cr-Cb space]
"Y-Cr-Cb space" is ITU-R (International Telecommunication Union Radiocommunication Division) recommendation BT. This is a standardized color space defined by the luminance information Y, the color difference Cb of the blue (Blue) component, and the color difference Cr of the red (Red) component, defined in 601.

[Hue correction information]
“Hue correction information” is information created based on the luminance data and color difference data of raw image data, and correction for correcting hue fluctuation at the time of overexposure or underexposure of a subject, that is, correction around the hue. Information to do. Therefore, it is possible to perform the hue correction process on the raw image data using only the hue correction information.

  The hue correction information includes, for example, information necessary for converting the color difference data Cr and the color difference data Cb of the raw image data into the color difference data Cr ′ and the color difference data Cb ′, that is, the color difference inherently performed in the image processing apparatus of the present invention. A matrix coefficient of an arithmetic expression used in the data conversion process is described.

  For example, when only the model name of the imaging device to which the image processing device is applied is described, the image processing device or the image recording device that reproduces the hue correction according to the present invention includes the model name of the imaging device and the matrix. Since there is a possibility of not having a coefficient correspondence table, it cannot be said to be hue correction information. In addition, for example, even if the information sufficient for performing the hue correction according to the present invention is not directly described, a URL indicating the location of the information on the Internet is described. It can be regarded as data sufficient to perform such hue correction. The hue correction information is preferably recorded on the information recording medium as tag information written in a header portion in the image file or a metafile associated with the tag information.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  A functional configuration of an imaging apparatus to which an image processing apparatus according to the present invention is applied will be described with reference to FIG. FIG. 1 is a block diagram showing a functional configuration of an imaging apparatus 1 according to the present invention.

  The imaging device 1 includes a lens 2, an aperture 3, an imaging device 4, an analog processing circuit 5, a temporary storage memory 6, an image processing unit 7, a header information processing unit 8, a storage device 9, an imaging device drive circuit 10, a control unit 11, The A / D converter 13, the operation unit 14, the display unit 15, the strobe driving circuit 16, the strobe 17, the focal length adjustment circuit 18, the automatic focus driving circuit 19, the motor 20 and the like are configured.

  The optical system of the imaging apparatus 1 includes a lens 2, a diaphragm 3, and an imaging element 4.

  The lens 2 adjusts the focus and forms an optical image of the subject. The diaphragm 3 adjusts the amount of light beam that has passed through the lens 2. The image sensor 4 photoelectrically converts subject light imaged on the light receiving surface by the lens 2 into an electrical signal (imaging signal) of an amount corresponding to the amount of incident light for each sensor in the image sensor 4. . The image sensor 4 is sequentially controlled by the timing pulse output from the image sensor drive circuit 10 to sequentially output the image signal to the analog processing circuit 5.

  The image pickup device 4 may be any device such as a two-dimensional CCD image sensor, a C-MOS image sensor, or a CID image sensor, but a primary color filter is used in the following embodiment. An interline type CCD image sensor will be described as an example.

  The analog processing circuit 5 performs R, G, B signal amplification, noise reduction processing, and the like on the image signal input from the image sensor 4. The processing in the analog processing circuit 5 can be switched ON / OFF via the control unit 11 in accordance with an operation signal from the operation unit 14.

  The A / D converter 13 converts the imaging signal input from the analog processing circuit 5 into image data and outputs the image data.

  The temporary storage memory 6 is a buffer memory or the like, and temporarily stores the image data output from the A / D converter 13.

  The image processing unit 7 performs image quality improvement processing such as tone correction of image data used for display on the display unit 15, crosstalk correction of spectral sensitivity, dark current noise suppression, sharpening, white balance adjustment, and saturation adjustment. Processing such as image size change, trimming, aspect conversion, and hue correction according to the present invention is performed. In the processing in the image processing unit 7, the image processing content can be switched via the control unit 11 in accordance with an operation signal from the operation unit 14.

  The header information processing unit 8 headers the hue correction information generated by the hue correction information means (not shown) of the image processing unit 7 or the association information with the hue correction information with respect to the image data stored in the temporary storage memory 6. Write processing as information.

  The storage device 9 is configured by a non-volatile semiconductor memory or the like, and includes an information recording medium such as a memory card that records captured image data, and a readable memory that stores a control program for the imaging apparatus 1. It is configured.

  The information recording medium may be, for example, a compact flash (registered trademark), memory stick, smart media, multimedia card, hard disk, floppy (registered trademark) disk, magnetic information recording medium (MO), or CD-R. . In addition, the unit for writing to the information recording medium may be a writing unit connected in a wired state via a cord, an independent or remote place connected in a wireless state via communication or the Internet, even if it is integrated with the photographing apparatus. Any form such as an installed unit may be used. Further, when the imaging device and the information recording medium writing unit are in a connected state, the image processing device or the image recording device directly receives “reproduction auxiliary data when performing imaging device characteristic correction processing” and “required data” from the imaging device. It may be an aspect having a function of reading out “ The file format when “recording on the media” is preferably recorded in a standardized general-purpose file format such as TIFF or JPEG, not a format unique to the imaging apparatus.

  The image sensor drive circuit 10 outputs a timing pulse based on the control signal output from the control unit 11 and controls the drive of the image sensor 4.

  The control unit 11 is configured by a CPU (Central Processing Unit) or the like, reads a control program of the image pickup apparatus 1 stored in the storage device 9, and controls the entire image pickup apparatus 1 according to the read program. Specifically, the control unit 11 controls the motor 20 for adjusting the focal length and focus (focus) of the lens 2 in accordance with an operation signal from the operation unit 14, and a focal length adjustment circuit. 18, the image sensor driving circuit 10, the analog processing circuit 5, the temporary storage memory 6, the image processing unit 7, the operation unit 14, the display unit 15 and the strobe driving circuit 16 are controlled to perform photographing.

  Further, the control unit 11 performs so-called automatic exposure control in which, for example, an image is divided into a plurality of blocks, and the aperture diameter of the diaphragm 3 and the charge accumulation time by the image sensor 4 are adjusted based on the representative luminance value of each block.

  The operation unit 14 includes a release button (not shown), various function buttons such as a power ON / OFF button, a zoom button, and cursor keys. The operation signal corresponding to each button or key is input to the control unit 11 as an input signal. Output. In the present embodiment, the operation unit 14 includes a function button for designating output of scene reference raw data.

  The display unit 15 displays an image based on the image data in accordance with a control signal from the control unit 11 and also displays information for the user of the imaging device 1 to confirm settings and conditions related to shooting.

The strobe drive circuit 16 drives and controls the strobe 17 to emit light when the subject brightness is low according to a control signal from the control unit 11.
The strobe 17 boosts the battery voltage to a predetermined high voltage and stores it as a charge in a capacitor. When driven by the strobe driving circuit 16, the xenon tube is caused to emit light by the electric charge stored in the capacitor, and the subject is irradiated with auxiliary light.

The focal length adjustment circuit 18 controls the motor 20 for adjusting the focal length by moving the lens 2 in accordance with a control signal from the control unit 11.
The automatic focus drive circuit 19 controls the motor 20 for adjusting the focus (focus) by moving the lens 2 in accordance with a control signal from the control unit 11.

  In the imaging apparatus 1 shown in FIG. 1, the temporary storage memory 6, the image processing unit 7, the control unit 11, and the storage device 9 cooperate to function as the image processing apparatus according to the present invention. That is, the temporary storage memory 6 functions as temporary storage means, the control unit 11 functions as control means, and the storage device 9 functions as information recording means.

  Details of the image processing unit 7 of the imaging apparatus 1 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a block diagram illustrating an internal configuration of the image processing unit 7 of the imaging device 1 of FIG.

  The image processing unit 7 includes an input unit 71 and an output unit 72, and includes a color space conversion unit 701, a hue correction information generation unit 702, a hue correction unit 703, an appreciation image reference data generation unit 704, a correction region specification unit 705, and a position. It also has a function as information generation means 706.

  Image data output from the temporary storage memory 6, that is, raw image data, and various control data output from the control unit 11 are input to the input unit 71.

  The output unit 72 outputs the image data and various control data processed by each unit of the image processing unit 7 to each unit connected to the image processing unit 7.

The color space conversion unit 701 converts the raw image data input to the input unit 71 into luminance data Y, color difference data Cr, and color difference data Cb in the Y-Cr-Cb space to generate hue correction information. . When the raw image data is RGB data, conversion to Y, Cr, Cb is performed according to, for example, ITU-R (International Telecommunication Union Radiocommunication Division) recommendation BT. Conversion is performed according to the following formula defined in 601.
Y = 0.299R + 0.587G + 0.114B
Cb = −0.172R−0.339G + 0.511B
Cr = 0.511R-0.428G-0.083B
The hue correction information generation unit 702 generates hue correction information that is information for performing a hue rotation correction based on the luminance data Y, the color difference data Cr, and the color difference data Cb converted by the color space conversion unit 701.

  The hue correction unit 703 performs a correction process for correcting a hue variation on the raw image data input to the input unit 71 based on the hue correction information generated by the hue correction information generation unit 702, that is, a hue rotation correction process. The corrected image data is output to the output unit 72 or the appreciation image reference data generation means 704.

  The appreciation image reference data generation unit 704 outputs the image data corrected by the hue correction unit 704 at the output destination based on the operation information on the type of the storage device 9 and the display unit 15 input from the operation unit 14 in FIG. Optimization processing for obtaining an optimal image is performed, and the image is output to the output unit 72 as viewing image reference data. Optimization processing includes, for example, compression to the output destination color gamut, gradation compression from 16 bits to 8 bits, reduction of the number of output pixels, and processing to handle output characteristics (LUT) of output devices and display devices. It is.

  The correction area specifying unit 704 analyzes the raw image data input to the input unit 71, and specifies a correction area in the raw image data to be subjected to the hue around correction process. The correction area is specified for each area obtained by dividing one image into a predetermined number. For example, when the main image of the area is a skin color that tends to occur around the hue, the correction area is set. Further, instead of the automatic area selection by the correction area specifying unit 704, the user can specify the correction area by operating the operation unit 14 while viewing the image displayed on the display unit 15. In this case, the operation unit 14 functions as a correction area specifying unit.

The position information generating unit 705 generates position information of the area specified by the correction area specifying unit 704. For example, when the correction area is a circular area, the position information is described by the center coordinates of the circle and its diameter. When the correction area is a rectangular area, the position information is described by the rectangular center coordinates and the size of the rectangular area. Is done. A description example is shown below.
In the case of a circular area: X coordinate value, Y coordinate value, in the case of a rectangular area: X coordinate value, Y coordinate value, width of rectangular area, height of rectangular area Next, the present invention will be described with reference to FIGS. An example of the operation of the imaging apparatus to which the image processing apparatus according to the invention is applied will be described.

  FIG. 3 is a flowchart showing a flow of imaging processing executed by the imaging device 1 shown in FIGS. 1 and 2.

  When the release button of the operation unit 14 is pressed, in step S1, the control unit 11 controls each unit of the imaging device 1 to image the subject. That is, the image pickup signal output from the image pickup device 4 by image pickup is converted into image data by the A / D converter 13, and raw image data is generated and held in the temporary storage memory 6.

  In step S <b> 2, the image processing unit 7 performs a correction process for correcting a hue variation with respect to the raw image data, that is, a hue rotation correction process. The image data that has undergone the hue correction process in step S2 is handled as scene reference image data. In step S2, hue correction information is also generated.

  In step S3, it is determined in advance whether or not the operation unit 14 or the like has been set to perform optimization processing for obtaining an optimal image at the output destination, that is, to create appreciation image reference data. If it is set to create appreciation image reference data (step S3; YES), the process of step S4 is executed. If appreciation image reference data is not created (step S3; NO), the process of step S5 is executed. The

  In step S4, appreciation image reference data generating means 704 generates appreciation image reference data that has been subjected to optimization processing for obtaining an optimal image at the output destination for the image data that has been corrected for hue rotation in step S2. Is done.

  In step S5, the scene reference image data generated in step S2, the appreciation image reference data generated in step S3, the image data such as the raw image data generated in step S1, and the hue correction information generated in step S2. Control data such as position information, which will be described later, is output from the image processing unit 7 and is recorded in the information recording medium 90 by the storage device 9, and the process ends. The header information is added to the image data by the header information processing unit 8. Further, instead of recording on the information recording medium 90, recording may be performed on a recording medium of another information processing apparatus via a communication medium (not shown).

  In step S5 of FIG. 3, the image data to which the header information is added and recorded on the information recording medium 90 is recorded as a file 91 including the raw image data 910, as shown in FIG. In 911, imaging device characteristic correction data d2 including hue correction information d1 is recorded. As another example, as shown in FIG. 4B, the identification information d3 of the metafile 93 associated with the raw image data 910 is described in the header area 921 of the file 92, and the identification information is displayed in the header area 931. Imaging device characteristic correction data d2 including hue correction information d1 is recorded in the metafile 93 having d3.

  By recording in this way, the information recording medium 90 is taken out from the imaging device 1 and attached to an external device such as another image processing device or an image recording device, thereby depending on the processing performed by the imaging device. In addition, it is possible to perform output with appropriate hue correction processing performed, and to reproduce the hue correction processing performed by the imaging apparatus.

  Details of the hue correction processing executed in step S4 of FIG. 3 will be described with reference to FIG. FIG. 5 is a flowchart showing the hue correction process executed in step S4 of FIG.

  In step S11, the raw image data held in the temporary storage memory 6 in step S1 of FIG. 3 is input to the input unit 71 of the image processing unit 7 shown in FIG.

  In step S12, γ conversion processing for correcting gradation is performed on the raw image data.

  In step S13, the raw image data input to the input unit 71 by the color space conversion unit 701 shown in FIG. 2 is converted into luminance data Y, color difference data Cr, and color difference data in the Y-Cr-Cb space for internal processing. Converted to Cb. When the raw image data is RGB data, conversion to Y, Cr, and Cb is performed. However, when the raw image data is YCrCb data, the conversion process is not performed and the process of step S14 is performed. .

  In step S <b> 14, it is determined whether or not the setting for selecting the area around the hue correction process, that is, the correction area, has been made in advance by the operation unit 14 or the like. If the setting for selecting the correction area has been made (step S14; YES), the process of step S15 is executed. If the correction area is not selected (step S14; NO), the process of step S17 is executed.

  In step S15, the correction area specifying unit 705 shown in FIG. 2 specifies and selects an area for performing the hue correction process.

  In step S16, the position information generating unit 706 shown in FIG. 2 generates position information of the correction area selected in step S15.

  In step S17, the correction coefficient K used in the matrix calculation for hue correction is calculated. The correction coefficient K is a function of the luminance data Y converted in step S13, and the color difference ratio represented by Cr / Cb determined based on the color difference data converted in step S13 depends on the value of the luminance data Y. The correction coefficient is greater than 0, which is determined to be constant.

Specifically, the correction coefficient K is calculated by the following equation (5).
(5) K = (Cr / Cb) / (Cr / Cb) _{Y = Y0}
In the above equation (5), (Cr / Cb) _{Y = Y0} is a predetermined value that can be uniquely determined by the imaging apparatus 1, for example, at the time of proper exposure obtained by changing the exposure amount for a skin-colored subject. Color difference ratio. That is, in the above equation (5), as long as the color difference data Cr and the color difference data Cb of the raw image data are within the appropriate exposure range, the correction coefficient K is near 1, and the hue rotation correction is not substantially performed. On the other hand, when one color component in the raw image data is saturated, the correction coefficient K becomes a value other than 1, and the hue circumference correction becomes effective. The process by which the above formula (5) is derived is described in detail in Japanese Patent Laid-Open No. 2003-61109.

Regarding (Cr / Cb) _{Y = Y0} , the individual difference of the color difference ratio is large even if it is the flesh color at the time of proper exposure, and it may be difficult to make the image apparatus 1 to be the only specified value. Therefore, in advance stores a plurality of values of (Cr / Cb) Y = _Y0 advance, selected is determined that the most preferred by environment during imaging the (Cr / Cb) Y = _Y0, in the selected value The correction coefficient K may be obtained.

  In step S18, a matrix arithmetic expression to be executed on the luminance data Y, the color difference data Cr and the color difference data Cb converted in step S13 is determined. In determining the matrix calculation formula, the most preferable calculation formula is selected based on, for example, whether the image is a skin color, the value of the luminance data Y, the value of the color difference data Cr or the color difference data Cb, and the like.

  In step S19, the position information generated in step S16 by the hue correction information generation unit 702 shown in FIG. 2, the correction coefficient K calculated in step S17, the type of matrix calculation formula determined in step S18, and the matrix Hue correction information is generated based on matrix coefficients and the like used in the arithmetic expression.

  In step S20, the color difference data of the raw image data converted in step S13 based on the correction coefficient K calculated in step S17 and the matrix arithmetic expression determined in step S18 by the hue correction unit 703 shown in FIG. Cr and color difference data Cb are corrected to color difference data Cr ′ and color difference data Cb ′.

  In step S21, the luminance data Y and the color difference data Cr ′ and color difference data Cb ′ corrected in step S20 are converted into output image data (for example, RGB data), and then the image processing unit 7 shown in FIG. The data is output to the output unit 72, stored in the temporary storage memory 6 or the storage device 9, and the process ends. Further, the hue correction information generated in step S19 is also output.

  Details of the matrix arithmetic expression determination process executed in step S18 of FIG. 5 will be described with reference to FIG. FIG. 6 is a flowchart showing the flow of the matrix arithmetic expression determination process executed in step S18 of FIG.

In this embodiment, the color difference converted in step S12 by performing matrix calculation using any one of the following formulas (1), (2), (3), (4), and (6). Data Cr and color difference data Cb are corrected to color difference data Cr ′ and color difference data Cb ′, respectively. In determining the matrix calculation formula, the most preferable calculation formula is selected based on, for example, whether the image is a skin color, the value of the luminance data Y, the value of the color difference data Cr or the color difference data Cb, and the like.
(1) Cr ′ = Cr × A / K + Cb × B,
Cb ′ = Cr × C + Cb × D
(2) Cr ′ = (Cr × A + Cb × B) / K,
Cb ′ = Cr × C + Cb × D
(3) Cr ′ = Cr × A + Cb × B,
Cb ′ = Cr × C + Cb × D × K
(4) Cr ′ = Cr × A + Cb × B,
Cb ′ = (Cr × C + Cb × D) × K
(6) Cr ′ = Cr × A + Cb × B,
Cb ′ = Cr × C + Cb × D
In the above formulas (1), (2), (3), (4) and (6), A, B, C and D are predetermined matrix coefficients, and K is the correction calculated in step S17 in FIG. It is a coefficient.

  In step S101, based on the color difference data Cr and the color difference data Cb, it is determined whether the raw image data is a skin color. Specifically, threshold values for Cr and Cb can be determined in advance, and if Cr and Cb are within the threshold values, the skin color can be determined. If it is determined that the skin color is selected (step S101; YES), the process of step S102 is executed. If it is not the skin color (step S101; NO), the process of step S109 is executed.

  In step S102, it is determined whether the luminance data Y is greater than a predetermined threshold Y0. When it is determined that the luminance data Y is greater than the threshold Y0 (step S102; YES), the process of step S103 is executed, and when it is less than the threshold Y0 (step S102; NO), the process of step S106 is executed.

  In step S103, it is determined whether or not the color difference data Cb is larger than a predetermined threshold Cb0. When it is determined that the color difference data Cb is larger than the threshold value Cb0 (step S103; YES), the process of step S104 is executed. When the color difference data Cb is smaller than the threshold value Cb0 (step S103; NO), the process of step S105 is executed.

  In step S104, equation (4) is determined as a matrix calculation equation, and equation (3) is determined in step S105.

  In step S106, it is determined whether or not the color difference data Cr is greater than a predetermined threshold value Cr0. When it is determined that the color difference data Cr is larger than the threshold value Cr0 (step S106; YES), the process of step S107 is executed, and when it is smaller than the threshold value Cr0 (step S106; NO), the process of step S108 is executed.

  In step S107, equation (2) is determined as a matrix operation equation, equation (1) is determined in step S108, and equation (6) is determined in step S109.

  In the matrix arithmetic expression determination process, the expressions (1) and (2) increase Cr in the high luminance region and sometimes protrude out of the color gamut. For this reason, the phenomenon of yellowing the skin with extremely high brightness is improved but not perfect. Therefore, it is suitable when the yellowing is small but the luminance is high. In particular, the expression (2) compensates for the hue variation of the Cb component with Cr, and is suitable for correcting an image of a person having a dark skin color under a low color temperature such as tungsten light.

  Further, the expressions (3) and (4) are for reducing Cb in the high luminance region, and do not protrude outside the color gamut, and the hue circumference can be completely corrected. However, since the saturation of the yellow system in the high luminance range is slightly lowered, it is suitable when the luminance is low. In particular, the equation (4) compensates for the hue variation of the Cr component with Cb, and is suitable for correcting an image obtained by photographing a person with light skin color under a high color temperature such as daylight.

  Furthermore, the color difference matrix calculation may be performed by combining any one of the formula (1) or the formula (2) and any one of the formula (3) or the formula (4). When only one of the coefficients is used without combining the two expressions, it is particularly effective to use the combined expression when an unfavorable influence is exerted on general colors other than the skin color.

  In this case, K1 used for Cr ′ and K2 used for Cb ′ may be divided in any way as long as they are divided while maintaining the relationship of K1 × K2 = K. Of course, in order to reduce the burden of the arithmetic processing, it is preferable to use only one of the coefficients, which is a trade-off between improving the color reproducibility and simplifying the arithmetic processing. Moreover, although Formula (1) and Formula (2) are the division by the coefficient K, the multiplication using the reciprocal number of K may be sufficient.

  As the calculations from the above formulas (1) to (4), for example, a calculation may be performed using a look-up table (LUT) indicating the relationship between the luminance Y and the K value. Alternatively, for example, in the case of Equation (1), a two-dimensional LUT that directly reads the value of Cr ′ from the value of luminance Y and color difference Cr may be used. By using the look-up table, it is possible to reduce the signal processing time and the manufacturing cost of the camera.

  As a reference example, a hue correction reproduction device that reproduces hue rotation correction based on data created by the imaging apparatus 1 and its operation will be described with reference to FIGS. FIG. 7 is a block diagram illustrating a functional configuration of a printer which is an embodiment of the hue correction correction reproduction apparatus.

  The printer 30 includes a control unit 31, an image processing unit 32, a printing unit 33, an operation display unit 34, a storage unit 35, a memory interface 36, a communication unit 37, and the like.

  The control unit 31 includes a CPU (central processing unit) (not shown), a work memory, and the like, reads a program stored in the storage unit 35 into the work memory, and centrally controls each unit of the printer 30 according to the program. .

  The image processing unit 32 performs image processing optimal for the printing unit 33 to print the image data received by the communication unit 37 or read by the memory interface 36.

  The printing unit 33 includes an image recording mechanism (not shown), an image recording mechanism using an inkjet method, a recording paper transport mechanism, and the like, and visualizes an image based on image data included in the image file on the recording paper.

  The operation display unit 34 includes, for example, a pressure-sensitive (resistive film pressure) touch panel in which transparent electrodes are arranged in a grid and various operation buttons, displays read image data, and displays various guidance displays to the user. It has a function of displaying the status and receiving various operations from the user, and detects the XY coordinates of the power point pressed by a finger or a dedicated touch pen and the button operation, and outputs them to the control unit 31 as an operation signal.

  The storage unit 35 is configured by a nonvolatile semiconductor memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory), and stores setting contents according to various programs and functions that can be executed by the printer 30.

  The memory interface 36 reads and writes various data including image data from the information information recording medium 361 mounted in a slot (not shown). The information information recording medium 361 is a detachable memory with respect to the imaging device 1, the printer 30, and the like. For example, a removable card-shaped or stick-shaped semiconductor memory such as a flash memory is applied, and image data is stored. Store various data including it.

  The communication unit 37 is an externally-acquired output terminal configured by, for example, a USB terminal or a wireless LAN card, and is connected to a computer (not shown) or the imaging device 1 through a wired or wireless communication medium N and includes image data. Communication such as sending and receiving various data.

  An example of the operation of the printer 20 shown in FIG. 7 will be described with reference to FIG. FIG. 8 is a flowchart showing the flow of print processing executed by the printer 20 of FIG. In the flowchart of FIG. 8, it is assumed that the printer 30 is powered on and a print execution command is issued to the printer 30. The print execution command is transmitted from, for example, a computer (not shown), the imaging apparatus 1 or the like via the communication unit 37, or the control unit 31 recognizes that the information information recording medium 361 is mounted in a slot (not shown). Is generated.

  In step S31, the raw image data and the hue correction information are read out via the communication unit 37 or the memory interface 36.

  In step S32, the image processing unit 32 generates appreciation image reference data optimized for the operation display unit 34 based on the raw image data and hue correction information read in step S31, and the preview image is displayed on the operation display unit 34. Is displayed.

  In step S33, the image processing unit 32 determines whether or not to print the preview image displayed in step S32, that is, an image that has been subjected to the same hue correction as that during imaging. The determination is made, for example, by a selection input to the operation display unit 34 with respect to a display that prompts the operator to select whether or not to print the same image as at the time of imaging when a preview image is displayed. If it is determined to print an image that has been subjected to the same correction as that at the time of imaging (step S33; YES), the process of step S34 is executed, and an image that has been subjected to correction different from that at the time of imaging is printed. Is determined (step S33; NO), the process of step S35 is executed.

  In step S34, the image processing unit 32 generates appreciation image reference data optimized for the image printed by the printing unit 33 based on the raw image data and the correction information read in step S31.

  In step S35, the image processing unit 32 performs a correction process unique to the printer 30 or a correction process intended by the operator on the raw image data read in step S31, and then the corrected image data. Based on the above, image data optimized for the image formed by the printing unit 33 is generated.

  In step S36, the printing unit 33 performs printing based on the image data generated in step S34 or step S35.

  As described above, in this reference example, it is possible to produce both an image that has been subjected to appropriate hue correction without depending on the captured digital camera or an image that reproduces the hue correction performed by the captured digital camera. Is possible.

  In the present embodiment, a digital camera has been described as an example of the image processing apparatus. However, the present invention can also be applied to an image reading apparatus such as a scanner or a photographer's photographer.

  An object of the present invention is to supply an information recording medium recording a software program for realizing the functions of the above-described embodiments to a system or apparatus, and the computer (CPU or MPU) of the system or apparatus uses the information recording medium. Needless to say, this can also be achieved by reading and executing the stored program.

  In this case, the program itself read from the information recording medium realizes the functions of the above-described embodiment, and the program and the information recording medium storing the program constitute the present invention.

  As an information recording medium for supplying the program, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like is used. Can do.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program is actual. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the processing and the processing is included.

  Furthermore, after the program read from the information recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program. Needless to say, the CPU of the function expansion unit or the like performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  In addition, the detailed configuration and detailed operation of each component constituting the image processing apparatus according to the present invention can be changed as appropriate without departing from the spirit of the present invention.

It is a block diagram which shows the functional structure of the imaging device to which the image processing apparatus which concerns on this invention was applied. It is a block diagram which shows the internal structure of the image process part of FIG. It is a flowchart which shows the flow of the imaging process performed by the imaging device which concerns on this invention. FIG. 4A is a diagram showing a recording example of the hue correction information according to the present invention, and FIG. 4B is a diagram showing another recording example of the hue correction information according to the present invention. It is a flowchart which shows the flow of the hue periphery correction | amendment process performed with the flowchart of FIG. It is a flowchart which shows the flow of the matrix arithmetic expression determination process performed with the flowchart of FIG. It is a block diagram which shows the functional structure of the printer which is one Embodiment of the hue correction | amendment reproduction apparatus which concerns on a reference example. It is a flowchart which shows the flow of the printing process performed by the printer of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Lens 3 Aperture 4 Image sensor 5 Analog processing circuit 6 Temporary storage memory 7 Image processing part 71 Input part 72 Output part 701 Color space conversion means 702 Hue correction information generation means 703 Hue correction reference means 704 Appreciation image reference data generation means 705 Correction area specifying means 706 Position information generating means 8 Header information processing section 9 Storage device 91, 92 File 93 Metafile 910 Raw image data 911, 921, 931 Header area 11 Control section 13 A / D converter 14 Operation section 15 Display Unit 30 printer 31 control unit 32 image processing unit 33 print unit 34 operation display unit 35 storage unit 36 memory interface 37 communication unit d1 hue correction information d2 imaging device characteristic correction data d3 identification information

Claims (15)

A raw image data input step of inputting raw image data generated by imaging a subject;
A color space data conversion step of converting the raw image data into luminance data Y and color difference data Cr and color difference data Cb in the Y-Cr-Cb space;
A correction information generating step for generating hue correction information based on the luminance data Y, the color difference data Cr, and the color difference data Cb;
A hue correction step of generating corrected image data based on the hue correction information and the raw image data;
And an information recording step of recording the hue correction information and the raw image data on an information recording medium. The image processing method according to claim 1, wherein the information recording step records the corrected image data on an information recording medium. In the hue correction step, the color difference data Cr and the color difference data Cb are converted into the color difference data Cr ′ and the color difference data Cb ′ by performing matrix calculation using any one of the following formulas (1) to (4), respectively. To correct
The image processing method according to claim 1, wherein the hue correction information includes a correction coefficient K used in any one of the following formulas (1) to (4).
(1) Cr ′ = Cr × A / K + Cb × B,
Cb ′ = Cr × C + Cb × D
(2) Cr ′ = (Cr × A + Cb × B) / K,
Cb ′ = Cr × C + Cb × D
(3) Cr ′ = Cr × A + Cb × B,
Cb ′ = Cr × C + Cb × D × K
(4) Cr ′ = Cr × A + Cb × B,
Cb ′ = (Cr × C + Cb × D) × K
However, A, B, C and D are matrix coefficients, K is a function of the luminance data Y, and the color difference ratio represented by Cr ′ / Cb ′ is constant regardless of the value of the luminance data Y. The correction coefficient is determined to be greater than 0. The image processing method according to claim 3, wherein the correction coefficient K is calculated by the following equation (5).
(5) K = (Cr / Cb) / (Cr / Cb) _{Y = Y0}
However, (Cr / Cb) _{Y = Y0} is the color difference ratio at the time of proper exposure obtained by changing the exposure amount for the skin-colored subject. 5. The image processing method according to claim 1, wherein in the information recording step, the hue correction information and the raw image data are associated and recorded on an information recording medium. A correction area specifying step for specifying a correction area in one image;
Including a position information generation step of generating position information indicating the position of the correction region,
The color space data conversion step, the correction information generation step, and the hue correction step are performed on the raw image data in the correction region,
6. The image processing method according to claim 1, wherein the information recording step records the hue correction information and the position information in association with each other on an information recording medium. Raw image data input means for inputting raw image data generated by imaging a subject;
Color space data conversion means for converting the raw image data into luminance data Y and color difference data Cr and color difference data Cb in the Y-Cr-Cb space;
Correction information generating means for generating hue correction information based on the luminance data Y, the color difference data Cr, and the color difference data Cb;
Hue correction means for generating corrected image data based on the hue correction information and the raw image data;
An image processing apparatus comprising: information recording means for recording the hue correction information and the raw image data on an information recording medium. The image processing apparatus according to claim 7, wherein the information recording unit records the corrected image data on an information recording medium. The hue correction unit performs matrix calculation using any one of the following formulas (1) to (4) to thereby convert the color difference data Cr and the color difference data Cb into the color difference data Cr ′ and the color difference data Cb, respectively. Is to correct
The image processing apparatus according to claim 7, wherein the hue correction information includes a correction coefficient K used in any one of the following formulas (1) to (4).
(1) Cr ′ = Cr × A / K + Cb × B,
Cb ′ = Cr × C + Cb × D
(2) Cr ′ = (Cr × A + Cb × B) / K,
Cb ′ = Cr × C + Cb × D
(3) Cr ′ = Cr × A + Cb × B,
Cb ′ = Cr × C + Cb × D × K
(4) Cr ′ = Cr × A + Cb × B,
Cb ′ = (Cr × C + Cb × D) × K
However, A, B, C and D are matrix coefficients, K is a function of the luminance data Y, and the color difference ratio represented by Cr ′ / Cb ′ is constant regardless of the value of the luminance data Y. The correction coefficient is determined to be greater than 0. The image processing apparatus according to claim 9, wherein the correction coefficient K is calculated by the following equation (5).
(5) K = (Cr / Cb) / (Cr / Cb) _{Y = Y0}
However, (Cr / Cb) _{Y = Y0} is the color difference ratio at the time of proper exposure obtained by changing the exposure amount for the skin-colored subject. The image processing apparatus according to any one of claims 7 to 10, wherein the information recording unit records the hue correction information and the raw image data in an information recording medium in association with each other. Correction area specifying means for specifying a correction area in one image;
Position information generating means for generating position information indicating the position of the correction region;
The color space data conversion means and the correction information generation means execute processing on the raw image data of the correction area,
The image processing apparatus according to claim 7, wherein the information recording unit records the hue correction information and the position information in an information recording medium in association with each other. An image processing program that causes a computer to execute the image processing method according to claim 1. An image processing program for causing a computer to function as each means according to any one of claims 7 to 12. An imaging apparatus comprising the image processing apparatus according to claim 7.

Images